PHOENIX-7876 Improve EXPLAIN

[apurtell]

Phoenix's EXPLAIN [WITH REGIONS] output is today incomplete and easy to misread for both query analysis and performance investigation. Non-trivial joins render with almost no detail. The cost vector advertises three dimensions but only compares IO, and estimates/statistics are unreliable. Some optimizations are incorrectly categorized, for example some shown as row-eliminating predicates are actually column-projection optimizations. The operator trees of complex queries are flattened to a single ident level. Many planner facts are not surfaced at all, such as which rule chooses an index, which indexes the optimizer considered but rejected, what query rewrites took place (e.g. subquery decorrelation, star-join detection, right-to-left normalization, HAVING lift, RVC-offset translation, reverse-scan substitution, UNION ORDER BY, index expression substitution, and more), the specific hash-join strategy chosen, salt bucket counts, local vs. global vs. uncovered-global index distinctions, the particular flavor of atomic upsert chosen and server-side atomic update expressions, multi tenant context, CDC scope, transaction provider, projection lists, predicate to filter attribution, hints honored vs ignored, and the structure of the JSON/BSON/array path expressions evaluated server-side.

This proposal closes all of those gaps. Details provided in the design document

This work was implemented on the dev branch PHOENIX-7876-feature. All of the individual commits made on the dev branch are preserved as independent commits for this PR.

The bulk of the changes are to unit tests and integration tests where they formerly dumped raw EXPLAIN text and did exact string matching against the result. Now every test uses a fluent assertion API against ExplainPlanAttributes. The coverage of the tests has not changed. Their maintainability has improved 1000% (imho). The EXPLAIN features themselves have significantly more test coverage now. Everything I touched I added tests for. Most of them are connectionless tests that don't require a minicluster.

PHOENIX_EXPLAIN_MIGRATION_GUIDE.md is a migration guide for anyone scraping today's EXPLAIN PLAN output.

PHOENIX_EXPLAIN_EXAMPLES.md is a companion document for the reviewer's guide that illustrates the new capabilities with ~60 examples that exercise the new and modified EXPLAIN PLAIN features.

The following tests are known to have pre-existing failure or flake conditions and flake or fail on current master:

org.apache.phoenix.end2end.BackwardCompatibilityIT
org.apache.phoenix.end2end.GlobalConnectionTenantTableIT
org.apache.phoenix.end2end.index.IndexTwoPhaseCreateIT
org.apache.phoenix.end2end.index.LocalIndexIT
org.apache.phoenix.end2end.IndexScrutinyWithMaxLookbackIT
org.apache.phoenix.end2end.transform.TransformIT
org.apache.phoenix.end2end.transform.TransformMonitorIT
org.apache.phoenix.end2end.transform.TransformToolIT
org.apache.phoenix.jdbc.SecureUserConnectionsIT
org.apache.phoenix.query.MaxConcurrentConnectionsIT
org.apache.phoenix.rpc.UpdateCacheIT

I have root caused these failures and have either already filed a JIRA for them or will will shortly.

Co-Authored-By: Claude Opus 4.8[1m] noreply@anthropic.com

[virajjasani]

https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-2556/