NIFI-15683 Fix SFTP Move strategy failure when destination file alrea…#11281
NIFI-15683 Fix SFTP Move strategy failure when destination file alrea…#11281rakesh-rsky wants to merge 2 commits into
Conversation
exceptionfactory
left a comment
There was a problem hiding this comment.
Thanks for proposing this change @rakesh-rsky. A similar effort recently stalled, attempting to handle failures related to moving files. There are several problems with potential solutions, including existing expectations, and lack of clear failure paths for post-fetch operations. For these reasons, falling back to deleting a file may not be appropriate. In addition, catching the general IOException is too broad to be a safe approach. This implementation needs further consideration of potential options, before moving forward.
6a2d345 to
40ff482
Compare
|
Thank you for the review feedback @exceptionfactory. You're right on both points — the automatic delete-then-rename fallback was too aggressive, and catching the general I've reworked the approach:
This gives users explicit control over the conflict resolution behaviour while keeping the failure paths clean and predictable. |
…dy exists When FetchSFTP uses completion strategy 'Move', some SFTP servers (notably OpenSSH) return SSH_FX_FAILURE on rename if the destination file already exists, rather than atomically overwriting it. This causes a warning and leaves the source file in place. Add a delete-then-rename fallback: if the initial rename fails, attempt to delete the destination and retry the rename. If the destination does not exist (FileNotFoundException), the original rename exception is re-thrown to preserve the error signal for unrelated failures.
40ff482 to
2ac0233
Compare
exceptionfactory
left a comment
There was a problem hiding this comment.
Thanks for the work on this @rakesh-rsky. The revised approach looks like a good way forward, the new property and opt-in behavior provide a good path for optional handling.
From an implementation perspective, it would be helpful to define the new allowable values in an enum that implements DescribedValue, following the pattern elsewhere in the project.
Any description strings that go over one line can also be defined as multiline strings, instead of using concatenation.
The new test class looks good, and although there are divergent naming conventions, I recommend that new tests be named with Test as the suffix, so FetchSFTPTest in this case.
If you can address these items, that would help move this forward.
- Replace AllowableValue pair with MoveConflictResolution enum implementing DescribedValue, following the pattern used elsewhere in the codebase (e.g. DeleteSFTP.RemovalStrategy). - Use a text block for the Move Conflict Resolution property description instead of string concatenation. - Rename TestFetchSFTP to FetchSFTPTest to follow the Test suffix naming convention used for new test classes.
NIFI-15683 Fix SFTP Move strategy failure when destination file already exists
When FetchSFTP uses completion strategy 'Move', some SFTP servers (notably OpenSSH) return SSH_FX_FAILURE on rename if the destination file already exists, rather than atomically overwriting it. This causes a warning and leaves the source file in place.
Add a delete-then-rename fallback: if the initial rename fails, attempt to delete the destination and retry the rename. If the destination does not exist (FileNotFoundException), the original rename exception is re-thrown to preserve the error signal for unrelated failures.
Summary
NIFI-15683
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking
Pull Request Tracking
NIFI-00000NIFI-00000VerifiedstatusPull Request Formatting
mainbranchVerification
Please indicate the verification steps performed prior to pull request creation.
Build
./mvnw clean install -P contrib-checkLicensing
LICENSEandNOTICEfilesDocumentation