Server 8.10 HIMPORT support

Directory.Build.props

@@ -10,7 +10,7 @@
     <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
     <CodeAnalysisRuleset>$(MSBuildThisFileDirectory)Shared.ruleset</CodeAnalysisRuleset>
     <MSBuildWarningsAsMessages>NETSDK1069</MSBuildWarningsAsMessages>
-    <NoWarn>$(NoWarn);NU5105;NU1507;SER001;SER002;SER003;SER004;SER005;SER006</NoWarn>
+    <NoWarn>$(NoWarn);NU5105;NU1507;SER001;SER002;SER003;SER004;SER005;SER006;SER008</NoWarn>
     <PackageReleaseNotes>https://stackexchange.github.io/StackExchange.Redis/ReleaseNotes</PackageReleaseNotes>
     <PackageProjectUrl>https://stackexchange.github.io/StackExchange.Redis/</PackageProjectUrl>
     <PackageLicenseExpression>MIT</PackageLicenseExpression>

docs/HImport.md

@@ -0,0 +1,76 @@
+Hash Import
+===
+
+The `HIMPORT` command (Redis 8.10 and later) is a fast, session-based way to create many hashes that share a common
+set of field names — for example, importing a batch of records where every record has the same columns. Rather than
+sending the field names again for every hash (as a series of `HSET` calls would), the field names are declared once per
+connection and each hash then supplies only its values, positionally matched to those fields.
+
+On the wire this is a *connection-sticky* container command with several sub-commands: `HIMPORT PREPARE` registers a
+named *fieldset* (the ordered field names) on the current connection, `HIMPORT SET` creates one hash from a row of
+values against that fieldset, and `HIMPORT DISCARD` releases the fieldset. The fieldset lives only on the connection
+that prepared it and disappears when that connection is reset or closed.
+
+Because StackExchange.Redis is a multiplexer that owns the connection lifetime on your behalf — you do not control
+*which* physical connection a given command travels on<sup>&dagger;</sup> — it does **not** expose the individual `HIMPORT` sub-commands.
+Managing a `PREPARE`/`SET`/`DISCARD` sequence yourself would require pinning them all to one connection, which is
+exactly the detail the multiplexer abstracts away. Instead, the library exposes a single bulk operation,
+`IDatabase.HashImport` (and `HashImportAsync`), that performs the whole import for you: it generates a private fieldset,
+issues the `PREPARE`, one `SET` per entry, and the terminating `DISCARD`, all guaranteed to land on a single connection.
+
+<sup>&dagger;</sup>: *in theory* because of multiplexing there is a single connection, but with automatic reconnects, cluster
+slot migrations, active-active / retries, etc: *in reality* it is more complicated than that.
+
+Usage
+---
+
+You supply the shared field names once, and then one `HashImportEntry` per hash — each carrying the target key and that
+hash's values, in the same order as the field names:
+
+``` c#
+IDatabase db = muxer.GetDatabase();
+
+// performance note: you may use leased/oversized buffers throughout, see Notes
+
+// the field names shared by every hash we are importing
+var fields = new RedisValue[] { "name", "email", "age" };
+
+// one entry per hash: the key, plus its values positionally matching the fields above
+var entries = new HashImportEntry[]
+{
+    new("user:1", new RedisValue[] { "alice", "a@example.com", 30 }),
+    new("user:2", new RedisValue[] { "bob",   "b@example.com", 25 }),
+    new("user:3", new RedisValue[] { "carol", "c@example.com", 42 }),
+};
+
+var failures = await db.HashImportAsync(fields, entries);
+foreach (var failure in failures) // empty array on full success
+{
+    Console.WriteLine($"entry {failure.Index} ({failure.Key}) failed: {failure.Message}");
+}
+```
+
+After this completes, `user:1`, `user:2` and `user:3` each exist as a hash with the `name`, `email` and `age` fields
+set to their respective values.
+
+Notes
+---
+
+- `ReadOnlyMemory<T>` is used in the API (rather than arrays) so that you can pass slices of larger buffers without copying — useful
+  when importing in chunks from a pooled or reused backing array.
+  - In particular, only 2 leases are needed: a buffer of type `HashImportEntry` of length `entries`,
+    and a buffer of type `RedisValue` of length `(entries + 1) * fields`, using a [stride](https://en.wikipedia.org/wiki/Stride_of_an_array) of
+    length `fields` per entry, plus a leading header row of the field names.
+- Every entry must supply exactly as many values as there are field names; a mismatch throws (`ArgumentException`)
+  before anything is sent.
+- Each entry **replaces** any existing hash at its key (it does not merge into it) — this is import, not `HSET`.
+- The import is **not atomic**. Per-entry failures — for example a key that already holds a non-hash value, which
+  yields a `WRONGTYPE` error — are returned as `HashImportFailure[]` (each with the entry `Index`, `Key`, and server
+  `Message`); an empty array means full success, and other entries still apply. A *setup* failure (an invalid field
+  list) is thrown instead. If you need all-or-nothing semantics, this is not the right tool.
+- A zero-entry import is a no-op. There is no single-row shortcut: one entry uses the same `HIMPORT` path so its
+  replace-semantics are identical to a multi-entry import.
+- Inside a `MULTI`/`EXEC` transaction (`CreateTransaction`), the import is unrolled into individual queued commands
+  (`PREPARE`, one `SET` per entry, then `DISCARD`) so the whole import executes atomically as part of the transaction.
+  Batches are *not* supported.
+- Being connection-sticky, it is not cluster-aware; in a cluster, all supplied keys must map to a single node.

docs/exp/SER008.md

@@ -0,0 +1,24 @@
+Redis 8.10 is currently in preview and may be subject to change.
+
+New features in Redis 8.10:
+
+- Session-based bulk hash import via fieldsets (`HIMPORT`), surfaced as `IDatabase.HashImport`/`HashImportAsync`
+
+The corresponding library features must also be considered subject to change:
+
+1. Existing bindings may cease working correctly if the underlying server API changes.
+2. Changes to the server API may require changes to the library API, manifesting in either/both of build-time
+   or run-time breaks. 
+
+While this seems *unlikely*, it must be considered a possibility. If you acknowledge this, you can suppress
+this warning by adding the following to your `csproj` file:
+
+```xml
+<NoWarn>$(NoWarn);SER008</NoWarn>
+```
+
+or more granularly / locally in C#:
+
+``` c#
+#pragma warning disable SER008
+```

docs/index.md

@@ -48,6 +48,7 @@ Documentation
 - [Streams](Streams) - how to use the Stream data type
 - [Arrays](Arrays) - how to use Redis Arrays as sparse arrays of values
 - [Vector Sets](VectorSets) - how to use Vector Sets for similarity search with embeddings
+- [Hash Import](HImport) - bulk-importing many hashes that share a common set of field names
 - [Where are `KEYS` / `SCAN` / `FLUSH*`?](KeysScan) - how to use server-based commands
 - [Profiling](Profiling) - profiling interfaces, as well as how to profile in an `async` world
 - [Scripting](Scripting) - running Lua scripts with convenient named parameter replacement

src/RESPite/Shared/Experiments.cs

@@ -14,6 +14,7 @@ internal static class Experiments
         public const string Respite = "SER004";
         public const string UnitTesting = "SER005";
         public const string Server_8_8 = "SER006";
+        public const string Server_8_10 = "SER008";
 
         // ReSharper restore InconsistentNaming
 

src/StackExchange.Redis/APITypes/HashImportEntry.cs

@@ -0,0 +1,38 @@
+using System;
+using System.Diagnostics.CodeAnalysis;
+using RESPite;
+
+namespace StackExchange.Redis;
+
+/// <summary>
+/// Describes a single hash to be created during a bulk <see cref="IDatabase.HashImport"/> operation: a key plus the
+/// field values for that key, supplied positionally against the shared field-name list of the import.
+/// </summary>
+/// <remarks>
+/// The <see cref="Values"/> are matched positionally to the <c>fields</c> passed to the import, so
+/// <see cref="ReadOnlyMemory{T}.Length"/> must equal the number of fields.
+/// </remarks>
+[Experimental(Experiments.Server_8_10, UrlFormat = Experiments.UrlFormat)]
+public readonly struct HashImportEntry
+{
+    /// <summary>
+    /// Initializes a <see cref="HashImportEntry"/> value.
+    /// </summary>
+    /// <param name="key">The key of the hash to create.</param>
+    /// <param name="values">The field values, in the same order as the field names supplied to the import.</param>
+    public HashImportEntry(RedisKey key, ReadOnlyMemory<RedisValue> values)
+    {
+        Key = key;
+        Values = values;
+    }
+
+    /// <summary>
+    /// The key of the hash to create.
+    /// </summary>
+    public RedisKey Key { get; }
+
+    /// <summary>
+    /// The field values for this hash, positionally matched to the shared field names of the import.
+    /// </summary>
+    public ReadOnlyMemory<RedisValue> Values { get; }
+}

src/StackExchange.Redis/APITypes/HashImportFailure.cs

@@ -0,0 +1,35 @@
+using System.Diagnostics.CodeAnalysis;
+using RESPite;
+
+namespace StackExchange.Redis;
+
+/// <summary>
+/// Describes a single entry that failed during a bulk <see cref="IDatabase.HashImport"/> operation. The import is not
+/// atomic, so individual entries can fail (for example, when a target key already holds a non-hash value) while others
+/// succeed; <see cref="IDatabase.HashImport"/> returns the failures (an empty array indicates a fully successful import).
+/// </summary>
+[Experimental(Experiments.Server_8_10, UrlFormat = Experiments.UrlFormat)]
+public readonly struct HashImportFailure
+{
+    internal HashImportFailure(int index, RedisKey key, string message)
+    {
+        Index = index;
+        Key = key;
+        Message = message;
+    }
+
+    /// <summary>
+    /// The zero-based index of the failing entry within the entries supplied to the import.
+    /// </summary>
+    public int Index { get; }
+
+    /// <summary>
+    /// The key of the failing entry, in the caller's key-space (any keyspace-isolation prefix has been removed).
+    /// </summary>
+    public RedisKey Key { get; }
+
+    /// <summary>
+    /// The server error describing why the entry failed (for example, a <c>WRONGTYPE</c> message).
+    /// </summary>
+    public string Message { get; }
+}

src/StackExchange.Redis/Enums/RedisCommand.cs

@@ -98,6 +98,7 @@ internal enum RedisCommand
     HGETEX,
     HGETDEL,
     HGETALL,
+    HIMPORT,
     HINCRBY,
     HINCRBYFLOAT,
     HKEYS,
@@ -363,6 +364,7 @@ internal static bool IsPrimaryOnly(this RedisCommand command)
             case RedisCommand.HEXPIREAT:
             case RedisCommand.HGETDEL:
             case RedisCommand.HGETEX:
+            case RedisCommand.HIMPORT:
             case RedisCommand.HINCRBY:
             case RedisCommand.HINCRBYFLOAT:
             case RedisCommand.HMSET:

src/StackExchange.Redis/Interfaces/IDatabase.cs

@@ -788,6 +788,32 @@ public partial interface IDatabase : IRedis, IDatabaseAsync
         /// <remarks><seealso href="https://redis.io/commands/hscan"/></remarks>
         IEnumerable<RedisValue> HashScanNoValues(RedisKey key, RedisValue pattern = default, int pageSize = RedisBase.CursorUtils.DefaultLibraryPageSize, long cursor = RedisBase.CursorUtils.Origin, int pageOffset = 0, CommandFlags flags = CommandFlags.None);
 
+        /// <summary>
+        /// Bulk-creates many hashes that share a common set of field names, using the server-side session fieldset
+        /// mechanism (<c>HIMPORT</c>). Each entry supplies a key and the field values for that key, matched positionally
+        /// against <paramref name="fields"/>.
+        /// </summary>
+        /// <param name="fields">The field names shared by every hash being imported; every entry must supply exactly this many values.</param>
+        /// <param name="entries">The hashes to create; each carries the key and its field values.</param>
+        /// <param name="flags">The flags to use for this operation.</param>
+        /// <returns>The entries that failed to import (an empty array on full success).</returns>
+        /// <remarks>
+        /// <para>
+        /// The import is not atomic: it is unrolled into a session-local <c>HIMPORT PREPARE</c>, one <c>HIMPORT SET</c>
+        /// per entry, and a terminating <c>HIMPORT DISCARD</c>, all issued on a single connection. Each <c>SET</c>
+        /// <em>replaces</em> any existing hash at that key. Per-entry failures (such as a key holding a non-hash value)
+        /// are returned as <see cref="HashImportFailure"/> values rather than thrown; setup failures (an invalid field
+        /// list) are thrown. A zero-entry import is a no-op.
+        /// </para>
+        /// <para>
+        /// Inside a <c>MULTI</c>/<c>EXEC</c> transaction the import is unrolled into individual queued commands so it
+        /// participates atomically; it is not supported inside a batch. Being connection-sticky, it is not cluster-aware.
+        /// </para>
+        /// <para><seealso href="https://redis.io/commands/himport"/></para>
+        /// </remarks>
+        [Experimental(Experiments.Server_8_10, UrlFormat = Experiments.UrlFormat)]
+        HashImportFailure[] HashImport(ReadOnlyMemory<RedisValue> fields, ReadOnlyMemory<HashImportEntry> entries, CommandFlags flags = CommandFlags.None);
+
         /// <summary>
         /// Sets the specified fields to their respective values in the hash stored at key.
         /// This command overwrites any specified fields that already exist in the hash, leaving other unspecified fields untouched.

src/StackExchange.Redis/Interfaces/IDatabaseAsync.cs

@@ -180,6 +180,10 @@ public partial interface IDatabaseAsync : IRedisAsync
         /// <inheritdoc cref="IDatabase.HashScanNoValues(RedisKey, RedisValue, int, long, int, CommandFlags)"/>
         IAsyncEnumerable<RedisValue> HashScanNoValuesAsync(RedisKey key, RedisValue pattern = default, int pageSize = RedisBase.CursorUtils.DefaultLibraryPageSize, long cursor = RedisBase.CursorUtils.Origin, int pageOffset = 0, CommandFlags flags = CommandFlags.None);
 
+        /// <inheritdoc cref="IDatabase.HashImport(ReadOnlyMemory{RedisValue}, ReadOnlyMemory{HashImportEntry}, CommandFlags)"/>
+        [Experimental(Experiments.Server_8_10, UrlFormat = Experiments.UrlFormat)]
+        Task<HashImportFailure[]> HashImportAsync(ReadOnlyMemory<RedisValue> fields, ReadOnlyMemory<HashImportEntry> entries, CommandFlags flags = CommandFlags.None);
+
         /// <inheritdoc cref="IDatabase.HashSet(RedisKey, HashEntry[], CommandFlags)"/>
         Task HashSetAsync(RedisKey key, HashEntry[] hashFields, CommandFlags flags = CommandFlags.None);
 

src/StackExchange.Redis/KeyspaceIsolation/KeyPrefixed.cs

@@ -183,6 +183,9 @@ public Task<bool> HashSetAsync(RedisKey key, RedisValue hashField, RedisValue va
         public Task<long> HashStringLengthAsync(RedisKey key, RedisValue hashField, CommandFlags flags = CommandFlags.None) =>
             Inner.HashStringLengthAsync(ToInner(key), hashField, flags);
 
+        public async Task<HashImportFailure[]> HashImportAsync(ReadOnlyMemory<RedisValue> fields, ReadOnlyMemory<HashImportEntry> entries, CommandFlags flags = CommandFlags.None) =>
+            ToOuter(await Inner.HashImportAsync(fields, ToInner(entries), flags).ForAwait());
+
         public Task HashSetAsync(RedisKey key, HashEntry[] hashFields, CommandFlags flags = CommandFlags.None) =>
             Inner.HashSetAsync(ToInner(key), hashFields, flags);
 
@@ -859,6 +862,29 @@ public void WaitAll(params Task[] tasks) =>
         protected internal RedisKey ToInner(RedisKey outer) =>
             RedisKey.WithPrefix(Prefix, outer);
 
+        // reverse of ToInner: strip the leading key-prefix bytes so a key that came back from the inner database
+        // (for example, inside a HashImport failure) is reported to the caller in their own, un-prefixed key-space.
+        protected internal RedisKey ToOuter(RedisKey inner)
+        {
+            var prefix = Prefix;
+            if (prefix is null || prefix.Length == 0) return inner;
+            byte[]? full = inner;
+            if (full is null || full.Length < prefix.Length) return inner; // not prefixed as expected; leave as-is
+            var outer = new byte[full.Length - prefix.Length];
+            Buffer.BlockCopy(full, prefix.Length, outer, 0, outer.Length);
+            return outer;
+        }
+
+        // maps failure keys back to the caller's key-space, in place (we own the array returned by the inner database)
+        private protected HashImportFailure[] ToOuter(HashImportFailure[] failures)
+        {
+            for (int i = 0; i < failures.Length; i++)
+            {
+                failures[i] = new HashImportFailure(failures[i].Index, ToOuter(failures[i].Key), failures[i].Message);
+            }
+            return failures;
+        }
+
         protected RedisKey ToInnerOrDefault(RedisKey outer) =>
             (outer == default(RedisKey)) ? outer : ToInner(outer);
 
@@ -914,6 +940,18 @@ protected RedisKey ToInnerOrDefault(RedisKey outer) =>
         protected KeyValuePair<RedisKey, RedisValue> ToInner(KeyValuePair<RedisKey, RedisValue> outer) =>
             new KeyValuePair<RedisKey, RedisValue>(ToInner(outer.Key), outer.Value);
 
+        protected ReadOnlyMemory<HashImportEntry> ToInner(ReadOnlyMemory<HashImportEntry> outer)
+        {
+            if (outer.IsEmpty) return outer;
+            var span = outer.Span;
+            var inner = new HashImportEntry[span.Length];
+            for (int i = 0; i < span.Length; i++)
+            {
+                inner[i] = new HashImportEntry(ToInner(span[i].Key), span[i].Values);
+            }
+            return inner;
+        }
+
         [return: NotNullIfNotNull("outer")]
         protected KeyValuePair<RedisKey, RedisValue>[]? ToInner(KeyValuePair<RedisKey, RedisValue>[]? outer)
         {

src/StackExchange.Redis/KeyspaceIsolation/KeyPrefixedDatabase.cs

@@ -174,6 +174,9 @@ public bool HashSet(RedisKey key, RedisValue hashField, RedisValue value, When w
         public long HashStringLength(RedisKey key, RedisValue hashField, CommandFlags flags = CommandFlags.None) =>
             Inner.HashStringLength(ToInner(key), hashField, flags);
 
+        public HashImportFailure[] HashImport(ReadOnlyMemory<RedisValue> fields, ReadOnlyMemory<HashImportEntry> entries, CommandFlags flags = CommandFlags.None) =>
+            ToOuter(Inner.HashImport(fields, ToInner(entries), flags));
+
         public void HashSet(RedisKey key, HashEntry[] hashFields, CommandFlags flags = CommandFlags.None) =>
             Inner.HashSet(ToInner(key), hashFields, flags);