[fix][broker] Prevent stale service unit callbacks from dropping active lookup and cleanup jobs#26146
Open
void-ptr974 wants to merge 1 commit into
Open
Conversation
void-ptr974
force-pushed
the
fix/service-unit-channel-stale-futures
branch
2 times, most recently
from
db37eac to
ca16bea
Compare
…futures ServiceUnitStateChannel keeps deferred owner lookups and inactive broker cleanup jobs in maps keyed by service unit or broker. Completion callbacks removed entries by key, so a stale completion could remove a newer future installed after a retry. Guard getOwnerRequests and cleanupJobs removals with the captured future. Also guard broker-creation cleanup cancellation so a delayed health-check result cannot cancel a newer cleanup job. Add tests that reproduce stale get-owner and cleanup-job completion removing newer futures.
void-ptr974
force-pushed
the
fix/service-unit-channel-stale-futures
branch
from
ca16bea to
1605aee
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
ServiceUnitStateChannelImplstores pending owner lookups and inactive-broker cleanup jobs in maps keyed by service unit or broker. These futures can be replaced when a lookup is retried, when an Owned event completes a waiting lookup, or when cleanup is rescheduled.Some completion and cancellation paths removed entries by key only. A stale callback could remove or cancel a newer future for the same key, causing owner lookups to wait for timeout or delaying inactive-broker ownership cleanup.
Modifications
getOwnerRequestsremovals with the captured request future.cleanupJobsremovals with the captured cleanup future.Verifying this change
./gradlew :pulsar-broker:test --tests org.apache.pulsar.broker.loadbalance.extensions.channel.ServiceUnitStateChannelTest.testCompletedGetOwnerRequestDoesNotRemoveNewRequest --tests org.apache.pulsar.broker.loadbalance.extensions.channel.ServiceUnitStateChannelTest.testSkippedEventDoesNotRemoveNewGetOwnerRequest --tests org.apache.pulsar.broker.loadbalance.extensions.channel.ServiceUnitStateChannelTest.testCompletedCleanupJobDoesNotRemoveNewCleanupJob --tests org.apache.pulsar.broker.loadbalance.extensions.channel.ServiceUnitStateChannelTest.handleBrokerCreationEventDoesNotCancelNewCleanupJobTest./gradlew :pulsar-broker:checkstyleMain :pulsar-broker:checkstyleTestgit diff --check