[KYUUBI #7230][AUTHZ] Support skipping privilege check for DataSourceV2Relation without catalog and identifier

[j1wonpark]

Why are the changes needed?

DSv2 TableProviders that do not implement SupportsCatalogOptions (the MongoDB connector, the ClickHouse native connector, etc.), when used directly via format(...).load() / .save(), produce a DataSourceV2Relation with neither catalog nor identifier — Spark's non-catalog fallback in DataSourceV2Utils.loadV2Source / DataFrameWriter passes (None, None), and the read-side fallback carries Spark's own TODO note that "Non-catalog paths for DSV2 are currently not well defined".

For such relations DataSourceV2RelationTableExtractor falls back to table.name(), which is connector-defined and usually synthetic (e.g. MongoTable()). The resulting Ranger resource has no database element, and a resource missing a parent level of the database → table → column hierarchy matches no policy at all — not even database=* / table=* (RangerDefaultPolicyResourceMatcher.isHierarchyValidForResources, Ranger 2.6.0). So the current behavior is an unconditional deny that operators cannot lift with any policy, reported in #7230. Meanwhile the v1 path is already skipped: a LogicalRelation without catalogTable produces no privilege object.

This PR adds an operator-level opt-out, default false so default behavior is unchanged:

• spark.kyuubi.authz.skip.catalogless.v2.relation.enabled (default false): when enabled, the extractor returns None for a DataSourceV2Relation whose catalog and identifier are both empty, restoring parity with the v1 path.
• The key is added to AuthzConfigurationChecker's built-in restricted list (like spark.sql.runSQLOnFiles), so end users cannot flip it with SET. Operators should also consider listing it in kyuubi.session.conf.restrict.list.

Note: whether skipping is safe is deployment-specific — fine when credentials for the external system are supplied per query by the end user, risky when the shared Spark principal holds ambient credentials. Hence opt-in rather than unconditional. Skipped relations produce no Ranger audit events.

How was this patch tested?

• New DataSourceV2RelationTableExtractorSuite:
  • default behavior preserved: the synthetic table.name() is still extracted verbatim when the conf is off;
  • the relation is skipped when the conf is enabled;
  • relations carrying an identifier are unaffected even with the conf enabled (the database fallback from [KYUUBI #6541] [AUTHZ] Fix DataSourceV2RelationTableExtractor can't get the 'database' attribute if it's a Paimon plan. #6544 still applies).
• Extended AuthzConfigurationCheckerSuite: SET on the new key throws AccessControlException.

Was this patch authored or co-authored using generative AI tooling?

Assisted-by: Claude Opus 4.8