HIVE-25948: Iceberg: Enable cost-based selection between Fanout and Clustered writers using column stats NDV

[deniskuzZ]

What changes were proposed in this pull request?

Cost-based selection between Fanout and Clustered writers

Why are the changes needed?

Perf optimization

Does this PR introduce any user-facing change?

No

How was this patch tested?

mvn test -Dtest=TestIcebergCliDriver -Dqfile=dynamic_partition_writes.q

┌───────────────────────┬───────────────────────────┐
│       Scenario        │         Expected          │
├───────────────────────┼───────────────────────────┤
│ threshold=0 (default) │ no sort (NDV<MAX_WRITERS) │
├───────────────────────┼───────────────────────────┤
│ threshold=-1          │ no sort                   │
├───────────────────────┼───────────────────────────┤
│ threshold=1           │ sort                      │
├───────────────────────┼───────────────────────────┤
│ threshold=2           │ sort (NDV>2)              │
├───────────────────────┼───────────────────────────┤
│ threshold=100         │ no sort (NDV<=100)        │
├───────────────────────┼───────────────────────────┤
│ fanout=false          │ sort                      │
└───────────────────────┴───────────────────────────┘

[okumin]

Only minor suggestions

[okumin]

I would simply use numBuckets as return new BucketStatEstimator(). But the current approach also works correctly. I would not mind if you would like to keep the current one.

[okumin]

I guess we shouldn't inherit all the stats from the first argument. We can create an empty ColStatics and set countDistinct, numNulls, range, and isEstimated.

[deniskuzZ]

other estimators also use clone.

[okumin]

The original one is aligned with our checkstyle: https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6389&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true

Suggested change

	case -1:
	return false;
	case 0:
	break;
	case 1:
	return true;
	default:
	MAX_WRITERS = threshold;
	break;
	case -1:
	return false;
	case 0:
	break;
	case 1:
	return true;
	default:
	MAX_WRITERS = threshold;
	break;

[deniskuzZ]

switch and case had the same indentation, so i've added spaces

[deniskuzZ]

switch and case had the same indentation, so i've added spaces

[sonarqubecloud]

Quality Gate passed

Issues
10 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud