Skip to content

[fix][broker] Shared subscription stalls forever on a chunked message - #26396

Open
0asys wants to merge 1 commit into
apache:masterfrom
0asys:fix-26395-chunked-message
Open

[fix][broker] Shared subscription stalls forever on a chunked message#26396
0asys wants to merge 1 commit into
apache:masterfrom
0asys:fix-26395-chunked-message

Conversation

@0asys

@0asys 0asys commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Fixes #26395

Motivation

A Shared subscription stops dispatching when it runs into a chunk of a chunked message whose chunk 0 has already been acknowledged.

SharedConsumerAssignor pins all chunks of a message to one consumer through the in-memory uuidToConsumer map, which is only written when chunk 0 is assigned. If the mapping is gone by the time a later chunk is dispatched (consumer restart, topic unload, broker restart), it hits:

if (metadata.getChunkId() != 0) {
// Not the first chunk, skip it
return null;
}

Modifications

SharedConsumerAssignor:

  • A chunk with chunkId != 0 and no cached mapping is now dispatched to the default consumer
    instead of being returned as unassignable. The mapping is only ever seeded by chunk 0, and the
    cursor always re-reads chunk 0 ahead of later chunks while it is still unacked, so a missing
    mapping means chunk 0 is already acked and the message can never be reassembled. Such an entry
    can never become assignable, so replaying it stalls the subscription, while dispatching it lets
    the client discard the incomplete chunked message and move on. Logged at WARN, the old path was
    completely silent.
  • Added removeConsumer(Consumer) and clear() to drop stale uuid mappings.

PersistentDispatcherMultipleConsumers and PersistentDispatcherMultipleConsumersClassic:

  • Call assignor.removeConsumer() in removeConsumer() and assignor.clear() in
    clearComponentsAfterRemovedAllConsumers(). Both PersistentStickyKeyDispatcherMultipleConsumers
    variants call super.removeConsumer(), so Key_Shared gets the fix as well.

The broker deliberately does not acknowledge the orphan chunk. Dropping an incomplete chunked
message is the client's decision (autoAckOldestChunkedMessageOnQueueFull,
expireTimeOfIncompleteChunkedMessage, AutoAckIncompleteChunk in the Go client), the broker's
job here is just to stop refusing to deliver it. With auto-ack disabled on the client the entry
stays unacked and gets redelivered and discarded on every reconnect, but the subscription keeps
making progress.

Verifying this change

  • Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

  • SharedConsumerAssignorTest.testOrphanChunkIsDispatchedInsteadOfReplayedForever covers the
    actual bug: a chunk with chunkId != 0 and an empty uuid cache must be dispatched and must not
    end up in the replay queue, for a last chunk as well as a middle one. It fails on the current
    code with lists don't have the same size expected [1] but found [0], since the entry is
    replayed instead of dispatched.
  • SharedConsumerAssignorTest.testUuidMappingIsRemovedWhenConsumerIsRemoved covers dropping the
    mapping of a removed consumer and dispatching the remaining chunk to another one.
  • SharedConsumerAssignorTest.testClearRemovesAllUuidMappings covers the leak.

The last two exercise methods that do not exist before this change.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Shared subscription stalls forever on a chunked message whose first chunk is already acked

1 participant