Skip to content

NIFI-15671 - Flow import/export with stateful components state#10986

Open
pvillard31 wants to merge 1 commit intoapache:mainfrom
pvillard31:NIFI-15671
Open

NIFI-15671 - Flow import/export with stateful components state#10986
pvillard31 wants to merge 1 commit intoapache:mainfrom
pvillard31:NIFI-15671

Conversation

@pvillard31
Copy link
Contributor

@pvillard31 pvillard31 commented Mar 9, 2026

Summary

NIFI-15671 - Flow import/export with stateful components state

This extends the JSON-based "Download flow definition" / "Upload flow definition" workflow to optionally include component state (processors and controller services annotated with @Stateful). A new includeComponentState query parameter on the download endpoint controls this. The feature does not apply to registry-based flow versioning.

Key behavioral decisions:

  • Opt-in via query parameter. GET /process-groups/{id}/download?includeComponentState=true includes state; the default is false, preserving backward compatibility. The existing ObjectMapper configuration (FAIL_ON_UNKNOWN_PROPERTIES = false) means older NiFi versions silently ignore the new field.
  • Never registry versioning. FlowMappingOptions.DEFAULT_OPTIONS (used by the registry path) has mapComponentState=false. This is architecturally enforced, not just a convention.
  • Components must be stopped. Exporting with state requires all processors in the group to be stopped and all controller services disabled. If any are running/enabled, the endpoint returns 409 Conflict. This guarantees state consistency.
  • Authorization escalation. Exporting with state requires WRITE permission on the process group (accessing state requires write access). Without state, only READ is needed.
  • Upload only, not replace. State is restored only for newly created components during upload. The replace endpoints (PUT and POST .../replace-requests) reject any flow definition containing component state with 400 Bad Request, because replacing an existing group produces a mix of new/updated/unchanged components where selective state restoration would be confusing.
  • Cluster LOCAL state handling. In a cluster, the export request is replicated to all nodes; each contributes its own LOCAL state keyed by ordinal position. The coordinator merges the responses. On import, each node picks up the state entry at its ordinal index. Ordinals are determined by sorting nodes by address:port.
  • Cluster topology validation on import. If the flow definition contains local state from S source nodes and the destination cluster has D connected nodes: S <= D is allowed (extra nodes get no state), S > D is rejected with 409 Conflict.
  • External services have state stripped. When includeReferencedServices=true, controller services from parent groups are included for wiring but their state is set to null — state export is scoped to the target group only.
  • Only @Stateful components get state. Non-stateful components always have null componentState, even when includeComponentState=true.

Note: UI changes would be as part of a follow-up effort.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
  • Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using ./mvnw clean install -P contrib-check
    • JDK 21
    • JDK 25

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant