-
Notifications
You must be signed in to change notification settings - Fork 2k
fix(cluster): make extractAllCommands drain the write queue #3364
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| import assert from 'node:assert'; | ||
| import RedisCommandsQueue from './commands-queue'; | ||
|
|
||
| describe('RedisCommandsQueue', () => { | ||
| function createQueue() { | ||
| return new RedisCommandsQueue(3, null, () => {}, 'test-client'); | ||
| } | ||
|
|
||
| describe('extractAllCommands', () => { | ||
| it('returns and removes every queued command, not just the first one', () => { | ||
| const queue = createQueue(); | ||
| for (let i = 0; i < 5; i++) { | ||
| queue.addCommand([`CMD${i}`]).catch(() => {}); | ||
| } | ||
|
|
||
| const extracted = queue.extractAllCommands(); | ||
|
|
||
| assert.strictEqual(extracted.length, 5); | ||
| assert.deepStrictEqual( | ||
| extracted.map(command => command.args?.[0]), | ||
| ['CMD0', 'CMD1', 'CMD2', 'CMD3', 'CMD4'], | ||
| ); | ||
| assert.strictEqual(queue.extractAllCommands().length, 0); | ||
| }); | ||
| }); | ||
| }); |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -665,15 +665,16 @@ export default class RedisCommandsQueue { | |
| } | ||
|
|
||
| /** | ||
| * Gets all commands from the write queue without removing them. | ||
| * Extracts all commands from the write queue. | ||
| */ | ||
| extractAllCommands(): CommandToWrite[] { | ||
| const result: CommandToWrite[] = []; | ||
| let current = this.#toWrite.head; | ||
| while (current) { | ||
| result.push(current.value); | ||
| this.#toWrite.remove(current); | ||
| const toRemove = current; | ||
| current = current.next; | ||
| this.#toWrite.remove(toRemove); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a remaining command has an abort signal or timeout and a full-shard migration calls this method, its listener still closes over the original source queue and linked-list node created in Useful? React with 👍 / 👎.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Real, but pre-existing — goes back to the V5 rewrite, not something this PR touches. |
||
| } | ||
| return result; | ||
| } | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When a source is split across multiple destinations while a transaction or pipeline is still queued,
_executeMultiand_executePipelineadd their commands withoutslotNumber, soextractCommandsForSlotsignores the entire batch. This newly complete drain then letscluster-slots.tsprepend all of those commands tolastDestNode, which is merely the final destination in notification order and may not own the batch's key slot; the commands consequently receiveMOVEDerrors or an aborted transaction instead of surviving the handoff. Preserve the selected batch slot on these queue entries or otherwise route the extracted chain to the destination owning that slot.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also pre-existing —
_executeMulti/_executePipelinenot settingslotNumbergoesback to the V5 rewrite too, not something this PR touches. Fixing it properly means
threading slot info through batch commands, bigger than this PR. Filing a separate
issue.