Skip to content

[fix][broker] Fix BucketDelayedDeliveryTracker recovery after LightProto migration#26160

Merged
nodece merged 2 commits into
apache:masterfrom
nodece:fix/bucket-snapshot-lightproto-recovery
Jul 10, 2026
Merged

[fix][broker] Fix BucketDelayedDeliveryTracker recovery after LightProto migration#26160
nodece merged 2 commits into
apache:masterfrom
nodece:fix/bucket-snapshot-lightproto-recovery

Conversation

@nodece

@nodece nodece commented Jul 7, 2026

Copy link
Copy Markdown
Member

Motivation

PR #25337 migrated broker delayed-delivery protos from protobuf-java to LightProto. Unlike protobuf-java which eagerly copies bytes fields, LightProto lazily references the parsed ByteBuf. The existing parseSnapshotMetadataEntry released the buffer immediately after parsing, so any later access to delayed_index_bit_map entries (via forEachDelayedIndexBitMap) threw IllegalReferenceCountException (refCnt: 0).

This caused BucketDelayedDeliveryTracker recovery to fail on every broker restart, silently falling back to InMemoryDelayedDeliveryTracker — all bucket snapshots became unreachable, bucket-level metrics disappeared, and new snapshots were no longer created.

Modifications

  • Force materialization of delayed_index_bit_map entries into standalone byte[] copies before releasing the entry buffer in parseSnapshotMetadataEntry.
  • Add testParseSnapshotMetadataEntryMaterializesLazyBytes that reproduces the bug by mocking LedgerEntry with a refCnt=1 buffer (matching real BK, unlike MockedBookKeeper which masks the bug with an extra reference).

Verifying

The test fails without the fix (IllegalReferenceCountException: refCnt: 0) and passes with it.

@merlimat

merlimat commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

@nodece There's a test failing, related to this change

@lhotari lhotari added this to the 5.0.0-M2 milestone Jul 8, 2026
@nodece nodece closed this Jul 9, 2026
@nodece nodece reopened this Jul 9, 2026
@nodece nodece merged commit a74268d into apache:master Jul 10, 2026
83 of 86 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants