[AMORO-4163][ams] Fix CommitFailedException when loading legacy mixed-iceberg tables on Iceberg 1.7.2

[lintingbin]

Search before asking

• I have searched in the issues and found no similar issues.

What type of PR is this?

• Improvement
• Bug Fix
• Feature
• Refactoring

What does this PR do?

Found while investigating the CI failure in #4179.

Root Cause

Iceberg 1.7.x introduced a breaking change in HadoopTableOperations.commit(): it now uses reference equality (==) to compare the base argument against the internally cached currentMetadata. If they differ, it throws CommitFailedException("Cannot commit changes based on stale table metadata").

In InternalMixedIcebergHandler.newTableOperations(), when loading a legacy mixed-iceberg table (created before v0.7.0), the code called ops.current() to obtain the current metadata, then passed it as base to ops.commit(base, legacyCurrent). However, commit() internally calls versionAndMetadata() which may refresh the internal state and return a different object instance than what current() returned — even when the on-disk metadata has not changed. This causes the reference-equality check to fail, throwing CommitFailedException with no cause.

The exception propagates through MixedHadoopTableOperations (which only wraps CommitFailedException with a cause) and the Thrift layer, causing BasicMixedCatalog.createTableMeta() to throw IllegalStateException("update table meta failed"), which makes table creation fail.

This is the root cause of the flaky TestInternalMixedCatalogService.CompatibilityCatalogTests#testNewCatalogLoadHistorical regression observed after the Iceberg 1.7.2 upgrade (#4163).

Fix

Replace ops.current() with ops.refresh(). refresh() reads from disk and stores the result as the internal cached reference, returning that same object. When commit() then calls versionAndMetadata(), it finds the version unchanged on disk and returns the same cached reference — satisfying the reference-equality check without any additional disk I/O beyond the single refresh() call.

Checklist

• I have added corresponding tests for my changes.
• This PR only changes one thing, and it is clear from the title.
• I have documented my changes in the relevant documentation.