Skip directory entries in RecursiveCopyableDataset to fix IOException on empty source dirs

[agam-99]

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
  • https://issues.apache.org/jira/browse/GOBBLIN-XXXX

Description

• Here are some details about my PR, including screenshots (if applicable):

RecursiveCopyableDataset.getCopyableFilesImpl iterates over filesInSource entries and calls
CopyableFile.resolveReplicatedOwnerAndPermissionsRecursively(fs, file.getPath().getParent(), replacedPrefix, ...)
for each. This method walks from file.getPath().getParent() up to replacedPrefix collecting
directory permissions to replicate at the destination. It requires replacedPrefix to be an
ancestor of file.getPath().getParent().

When source.path points to an empty directory, FileListUtils.listFilesToCopyAtPath
(with includeEmptyDirectories=true) returns the directory itself as the sole FileStatus entry.
For this entry file.getPath() == replacedPrefix, so file.getPath().getParent() is one level
above replacedPrefix. This inverts the ancestry check and throws:

IOException: toPath /jobs/.../edp_lts_opsreview must be an ancestor of fromPath hdfs://cluster/.../u_ltscso.db

Fix: Guard the fromPath argument passed to resolveReplicatedOwnerAndPermissionsRecursively
with an isAncestor check:

Path parentPath = file.getPath().getParent();
Path ancestorFromPath = PathUtils.isAncestor(replacedPrefix, parentPath) ? parentPath : file.getPath();

When parentPath is within replacedPrefix (normal files, nested empty subdirs) it is used
unchanged — no behavior change. When parentPath is above replacedPrefix (empty root dir),
file.getPath() is used instead, which equals replacedPrefix, causing the walk to produce an
empty ancestors list. The empty directory is still replicated at the destination.

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

Added testEmptySourceDirectoryProducesCopyEntityForDirectory to RecursiveCopyableDatasetTest:

• Simulates the exact condition that triggered the bug: filesInSource contains only the
  empty-directory FileStatus entry (as returned by FileListUtils when includeEmptyDirectories=true)
• Verifies that getCopyableFiles produces one copy entity mapping the empty directory to the
  target path, rather than throwing IOException

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits
  if they address the same issue. In addition, my commits follow the guidelines from
  "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.30%. Comparing base (26406a8) to head (2852359).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #4181      +/-   ##
============================================
+ Coverage     43.26%   43.30%   +0.03%     
- Complexity     2583     2587       +4     
============================================
  Files           516      516              
  Lines         22087    22101      +14     
  Branches       2505     2505              
============================================
+ Hits           9557     9570      +13     
- Misses        11566    11569       +3     
+ Partials        964      962       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.