[AMORO-4122][ams] Fix partition filter not working due to @JsonIgnore on PendingInput.partitions

[lintingbin]

Why are the changes needed?

Close #4122

The partitions field in AbstractOptimizingEvaluator.PendingInput is annotated with @JsonIgnore, which causes partition information to be silently dropped during JSON serialization/deserialization through TableRuntimeStore (stored as CLOB in the database).

When PendingInput is read back from the database, the partitions map is always empty. This causes the partition filter in IcebergTableUtil.createOptimizingPlanner() to fall back to Expressions.alwaysTrue(), meaning all partitions are scanned instead of only the ones that need optimizing.

Fix

Add a serializable partitionPaths field (Set<String>) that stores partition info as specId:partitionPath strings. On serialization, build paths from the in-memory partitions map via PartitionSpec.partitionToPath(); on deserialization, rebuild the partitions map from paths using DataFiles.data().

How was this patch tested?

• TestIcebergTableUtil: 4 tests passed
• TestOptimizingEvaluator: 20 tests passed
• TestOptimizingPlanner: 20 tests passed

[codecov-commenter]

Codecov Report

❌ Patch coverage is 0% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 22.63%. Comparing base (67feed8) to head (ea2502a).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...o/optimizing/plan/AbstractOptimizingEvaluator.java	0.00%	39 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (67feed8) and HEAD (ea2502a). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (67feed8)	HEAD (ea2502a)
core	1	0

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #4178      +/-   ##
============================================
- Coverage     29.72%   22.63%   -7.10%     
+ Complexity     4252     2612    -1640     
============================================
  Files           677      461     -216     
  Lines         54722    42556   -12166     
  Branches       6964     6003     -961     
============================================
- Hits          16268     9632    -6636     
+ Misses        37245    32087    -5158     
+ Partials       1209      837     -372     

Flag	Coverage Δ
core	?
trino	22.63% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[czy006]

Using org.apache.iceberg.DataFiles.data() here will break partition rebuild for Mixed-format tables. Iceberg's native DataFiles.data() delegates to Conversions.fromPartitionString(), which only handles standard Iceberg type conversions. It does not support Amoro's custom partition transforms:

• month transform: Mixed-format tables serialize month partitions as yyyy-MM (e.g., month=2024-01). Iceberg's parser expects an integer month offset, not a date string, and will throw NumberFormatException.
• hour transform: Similarly serialized as yyyy-MM-dd-HH, which Iceberg cannot parse.
• Hive null sentinel: __HIVE_DEFAULT_PARTITION__ is not handled by Iceberg's parser.

Amoro already provides MixedDataFiles.data(PartitionSpec, String) in org.apache.amoro.utils.MixedDataFiles (line 110) that correctly handles all these cases via its own fromPartitionString().

The catch(Exception) block below will silently swallow these parse failures, logging only a warning while leaving partitions empty — producing the same Expressions.alwaysTrue() fallback as the original bug. This effectively makes the fix a no-op for Mixed-format tables with custom transforms.

Suggested fix:

import org.apache.amoro.utils.MixedDataFiles;
// ...
struct = MixedDataFiles.data(spec, partitionPath);

Also, please consider adding test cases for:

1. Mixed-format tables with month/hour partition transforms (round-trip serialization)
2. Unpartitioned tables (ensure EMPTY_STRUCT path works correctly)

[lintingbin]

Modified

[czy006]

cc @zhoujinsong