Skip to content

fix: Queue batch commands to backlog during MOVED reconnection#3134

Open
advaMosh wants to merge 2 commits into
StackExchange:mainfrom
advaMosh:fix/batch-backlog-during-reconnect
Open

fix: Queue batch commands to backlog during MOVED reconnection#3134
advaMosh wants to merge 2 commits into
StackExchange:mainfrom
advaMosh:fix/batch-backlog-during-reconnect

Conversation

@advaMosh

@advaMosh advaMosh commented Jul 21, 2026

Copy link
Copy Markdown

Problem

When a MOVED-to-same-endpoint triggers reconnection (as implemented in PR #3003), batch commands (IBatch) issued during the ~50-100ms reconnection window fail immediately with RedisConnectionException: NoConnectionAvailable. This is because:

  1. RedisBatch.Execute() calls SelectServer() which returns null for disconnected/reconnecting servers.
  2. PhysicalBridge.TryEnqueue() returns false when !IsConnected, unlike TryWriteAsync/TryWriteSync which queue to the backlog via QueueOrFailMessage()BacklogEnqueue().

This creates an inconsistency: individual async commands (e.g. db.StringSetAsync()) are queued and succeed after reconnection, but batch commands throw immediately during the same reconnection window.

Impact

  • Applications using IBatch see 1-3 RedisConnectionException errors per MOVED redirect, while individual async commands see zero errors in the same scenario.
  • This affects deployments where Redis is behind DNS records, load balancers, or proxies that use MOVED-to-same-endpoint.

Fix

RedisBatch.Execute(): When SelectServer() returns null and BacklogPolicy.QueueWhileDisconnected is enabled, retry with allowDisconnected: true to locate the server endpoint. This mirrors the pattern already used in PrepareToPushMessageToBridge().

PhysicalBridge.TryEnqueue(): When !IsConnected || NeedsReconnect and BacklogPolicy.QueueWhileDisconnected is enabled, queue messages to the backlog instead of returning false. This matches the existing behavior of TryWriteSync() and TryWriteAsync() for individual commands.

Testing

Tested with a proxy that sends MOVED-to-same-endpoint followed by connection close:

  • Before fix: 1-3 NoConnectionAvailable errors per redirect event (batch.Execute throws during ~50-100ms reconnection window)
  • After fix: 0 errors — batch commands are queued to backlog and sent after reconnection completes

The fix only activates when BacklogPolicy.QueueWhileDisconnected is true (the default), preserving existing behavior for users who have disabled backlog queuing.

Related

When a MOVED-to-same-endpoint triggers reconnection (PR StackExchange#3003), batch
commands issued during the ~50-100ms reconnection window fail immediately
with NoConnectionAvailable. This is because:

1. RedisBatch.Execute() calls SelectServer() which returns null for
   disconnected/reconnecting servers.
2. PhysicalBridge.TryEnqueue() returns false when !IsConnected, unlike
   TryWriteAsync/TryWriteSync which queue to the backlog.

This creates an inconsistency where individual async commands succeed
(queued via BacklogPolicy) but batch commands throw during the same
reconnection window.

Fix:
- RedisBatch.Execute(): When SelectServer returns null and
  BacklogPolicy.QueueWhileDisconnected is enabled, retry with
  allowDisconnected=true to find the server endpoint.
- PhysicalBridge.TryEnqueue(): When disconnected/reconnecting and
  BacklogPolicy.QueueWhileDisconnected is enabled, queue messages to
  the backlog instead of returning false.

This matches the existing behavior of individual command paths and
ensures batch commands are transparently queued and sent after
reconnection completes.

Tested with proxy redirect scenario (MOVED-to-same-endpoint + disconnect):
- Before: 1-3 NoConnectionAvailable errors per MOVED redirect
- After: 0 errors, batch waits for reconnection via backlog
@advaMosh
advaMosh force-pushed the fix/batch-backlog-during-reconnect branch 2 times, most recently from 4d15422 to f33c283 Compare July 21, 2026 11:34
Add two integration tests verifying batch commands are queued to the
backlog during MOVED-to-same-endpoint reconnection:

- MovedToSameEndpoint_BatchCommands_QueuedDuringReconnect: Verifies
  that a batch containing a MOVED-triggering command completes
  successfully after reconnection (commands queued to backlog).

- MovedToSameEndpoint_SubsequentBatch_QueuedDuringReconnect: Verifies
  that a second batch issued during the reconnection window is queued
  rather than throwing NoConnectionAvailable (fire-and-forget scenario).
@advaMosh
advaMosh force-pushed the fix/batch-backlog-during-reconnect branch from f33c283 to 917ae81 Compare July 21, 2026 12:10
@advaMosh

Copy link
Copy Markdown
Author

hey @mgravell , thank you for the quick response !
I see the failing tests in the failing check are flaky regardless to this PR. can / will it be merged in that state?
also - Do you have an estimate for when this PR is expected to be merged, and which SE.R versions it’s likely to be included in?

@mgravell

Copy link
Copy Markdown
Collaborator

The CI has a bit of noise, sadly. I've poked it. I'll do another eyeball tomorrow, but there's no reason to delay this any - we tend to build fairly regularly when there's reason to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants