NIFI-15954 Honor property overrides when verifying Kafka Topics step#11264
Open
rfellows wants to merge 1 commit into
Open
NIFI-15954 Honor property overrides when verifying Kafka Topics step#11264rfellows wants to merge 1 commit into
rfellows wants to merge 1 commit into
Conversation
KafkaToS3.verifyTopicsExists now reads the specified Topic Names list from the overridden ConnectorConfigurationContext built from the verify request, rather than from the persisted FlowContext, so a verification request that narrows the selection is no longer evaluated against stale persisted state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
KafkaToS3.verifyTopicsExists now reads the specified Topic Names list from the overridden ConnectorConfigurationContext built from the verify request, rather than from the persisted FlowContext, so a verification request that narrows the selection is no longer evaluated against stale persisted state.
NIFI-15954 Honor property overrides when verifying Kafka Topics step
Summary
NIFI-15954
This is a bug fix for the
KafkaToS3connector'sVerify Kafka topics existstep.Motivation
KafkaToS3.verifyConfigurationStep(...)(nifi-connectors/nifi-kafka-to-s3-bundle/nifi-kafka-to-s3-connector/src/main/java/org/apache/nifi/connectors/kafkas3/KafkaToS3.java) already constructs an overriddenConnectorConfigurationContextfrom the verification request's property overrides viaworkingFlowContext.getConfigurationContext().createWithOverrides(stepName, propertyValueOverrides), and feeds that overridden context into theVersionedExternalFlowit builds for verification.The
verifyTopicsExists(...)helper, however, was reading theTopic Nameslist directly fromworkingFlowContext.getConfigurationContext()— i.e. from the persistedFlowContextrather than from the overridden context. As a result, a verify request that narrowed (or otherwise altered) the topic selection was evaluated against stale persisted state instead of the request's own values. The companionverifyKafkaParsability(...)was already correct because it operates on theflowargument, which is built from the overridden context.Changes
nifi-connectors/nifi-kafka-to-s3-bundle/nifi-kafka-to-s3-connector/src/main/java/org/apache/nifi/connectors/kafkas3/KafkaToS3.javaverifyTopicsExists(...)now accepts the overriddenConnectorConfigurationContextas a second argument and readsKafkaTopicsStep.TOPIC_NAMESfrom it, so the list of specified topics matches what the verification request asked to verify.verifyConfigurationStep(...)was updated to pass the already-built overriddenconfigurationContextthrough toverifyTopicsExists(...).verifyTopicsExists(...)andgetAvailableTopics(...)was relaxed fromprivateto package-private to allow unit-test subclasses to stub the broker lookup. No external visibility changed.verifyTopicsExists(...), the if/else order was flipped so the success branch is handled first and the failure branch second. This is purely stylistic; the producedConfigVerificationResultvalues and outcomes are unchanged.nifi-connectors/nifi-kafka-to-s3-bundle/nifi-kafka-to-s3-connector/src/test/java/org/apache/nifi/connectors/kafkas3/KafkaToS3Test.java(new)verifyTopicsExists(...). Uses Mockito to stubFlowContext,ConnectorConfigurationContext, andConnectorPropertyValue, and uses two small private subclasses ofKafkaToS3to stubgetAvailableTopics(FlowContext)so the test can drive both the success path and a broker-lookup failure without standing up a real connection service.StandardConnectorTestRunnerharness is intentionally not used here because it requires a packaged NAR and a running broker, which is the responsibility of thenifi-kafka-to-s3-integration-testsmodule.Verification
Three unit tests were added in
KafkaToS3Test:verifyTopicsExistsUsesOverriddenTopicList— the specified-topic list read by the step comes from the overriddenConnectorConfigurationContext, and a topic present in the broker passes (Outcome.SUCCESSFUL).verifyTopicsExistsReportsTopicMissingFromBroker— a topic supplied via the override that is absent from the broker is reported asOutcome.FAILED, and the explanation references the missing topic.verifyTopicsExistsReturnsSkippedWhenAvailableTopicsLookupFails— when the broker lookup throws, the step is reported asOutcome.SKIPPEDand the explanation propagates the underlying failure message.Risk & Scope
nifi-kafka-to-s3-connectormodule; no public API surfaces outside this module are touched.ConnectorConfigurationContextverifyTopicsExists(...)readsTopic Namesfrom. Outcome values, step name, and explanation strings on the success and failure paths are unchanged.private→ package-private) are confined to two internal helpers used only within the connector package; no new constructors, fields, or types are exposed.