feat(kernel): close deferred feature rows — OAuth M2M/U2M, metric-view, initial namespace#396
Open
mani-mathur-arch wants to merge 2 commits into
Open
Conversation
mani-mathur-arch
changed the title
mani-mathur-arch
force-pushed
the
mani/sea-kernel-build-recipe
branch
from
38c283f to
77e0231
Compare
mani-mathur-arch
force-pushed
the
mani/sea-kernel-dais-gaps
branch
from
2207471 to
1ebcaa7
Compare
…initial namespace Close the four DAIS-scope rows the SEA-via-kernel backend previously rejected at connect time. All are Go-side only (no new kernel dependency): the OAuth setters are on merged kernel #162, metric-view is an existing session conf, and the namespace uses plain SQL. - Metric view: config.EffectiveSessionParams() derives the server conf (spark.sql.thriftserver.metadata.metricview.enabled) once, backend-neutrally, so both backends send the identical conf. The Thrift OpenSession special-case is removed (behaviour-preserving); the kernel forwards it via SessionConf. Reject dropped; reclassified forwarded. - Initial namespace: applied post-connect via USE CATALOG / USE SCHEMA (the OSS ODBC workaround) since the kernel C ABI has no catalog/schema setter. quoteIdent (untagged) backtick-quotes identifiers; a USE failure fails connect and closes the session. Reject dropped; reclassified forwarded. - OAuth M2M/U2M: the kernel drives its own OAuth flow from raw credentials (mirroring pyo3/napi and the Node/Python kernel bindings), read off cfg.Authenticator — the single source of truth (last-writer-wins, matching Thrift). The m2m/u2m authenticators expose auth.M2MCredentialsProvider / auth.U2MCredentialsProvider; resolveKernelAuth type-switches them and returns a *kernelAuth descriptor. KernelBackend.setAuth branches to set_auth_pat / set_auth_m2m / set_auth_u2m; U2M uses Go's cloud-inferred client id (kernel defaults for scopes/port). No new config fields. Verified: default CGO_ENABLED=0 suite + golangci-lint v2.12.2 clean; tagged databricks_kernel unit tests (auth-mode -> setter mapping, quoteIdent); live staging e2e for initial namespace (current_catalog/current_schema) and metric-view (session opens + queries; the conf is not SET-introspectable on either backend). M2M/U2M covered by unit tests (no staging service principal; U2M is interactive). Co-authored-by: Isaac
… single-source auth, coverage + docs Remediation of the code-review pass on the DAIS gap-closure work. Verified against source; full default + tagged suites, golangci-lint v2.12.2, and live staging e2e all green. - Move the OAuth credential-provider interfaces (M2MCredentialsProvider / U2MCredentialsProvider) out of the public auth package into internal/backend/kernel, so the secret-reading capability is not part of the driver's public API. The unexported m2m/u2m authenticators satisfy them structurally. - Collapse the duplicate auth descriptor: validateKernelConfig/resolveKernelAuth now return kernel.Auth directly (its type is in an untagged file, so the default build builds it cgo-free); dropped dbsql.kernelAuth, kernelAuthMode, and toKernelAuth (which also removed a stale build-tag comment). - Route the initial-namespace failure-path session close through call() so a failed close is logged (via lastError's Warn), mirroring CloseSession. - Add an env-guarded live M2M e2e (TestKernelE2EM2M, skips without DATABRICKS_CLIENT_ID/_SECRET) and a last-writer-wins auth regression test; the resolveKernelAuth -> kernel.Auth path is table-tested for M2M/U2M. - Document: U2M is interactive (browser on cache-miss, blocks up to the kernel's ~120s callback timeout, connect ctx deadline not honored during that window, no C-ABI override — use PAT/M2M for headless); the U2M Scopes/RedirectPort fields are dormant-but-wired (no Go option feeds them yet); the metric-view e2e is a deliberate connect-smoke (routing asserted in TestEffectiveSessionParams). Custom M2M scopes remain unforwardable over the C ABI (no scopes arg on set_auth_m2m) — a kernel gap shared with ODBC, no authz impact (all-apis always requested); tracked in the kernel-gaps notes rather than worked around. Co-authored-by: Isaac
mani-mathur-arch
force-pushed
the
mani/sea-kernel-dais-gaps
branch
from
89ce8fc to
024053d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes the four deferred feature rows the SEA-via-kernel backend previously rejected at connect time. All are Go-side only — no new kernel dependency: the OAuth setters are on merged kernel #162, metric-view is an existing server session conf, and the initial namespace uses plain SQL.
config.EffectiveSessionParams()derives the server conf (spark.sql.thriftserver.metadata.metricview.enabled) once, backend-neutrally, so both backends send the identical conf. The ThriftOpenSessionspecial-case is removed (behaviour-preserving); the kernel forwards it viaSessionConf. Reject dropped, reclassified forwarded.USE CATALOG/USE SCHEMA(the OSS ODBC workaround) since the kernel C ABI has no catalog/schema setter.quoteIdent(untagged) backtick-quotes identifiers; aUSEfailure fails connect and closes the session (routed throughcall()so the close is logged). Reject dropped, reclassified forwarded.cfg.Authenticator— the single source of truth (last-writer-wins, matching Thrift). The m2m/u2m authenticators expose credential-provider interfaces defined ininternal/backend/kernel(kept off the public API so a secret-read capability is never exported);resolveKernelAuthtype-switches them and returnskernel.Authdirectly.KernelBackend.setAuthbranches toset_auth_pat/set_auth_m2m/set_auth_u2m. U2M uses Go's cloud-inferred client id (scopes/redirect-port default to the kernel's; no Go option feeds them yet). No newconfig.UserConfigfields.Stacked on #395 (
mani/sea-kernel-build-recipe); additive only, so #393 and #395 are untouched.Commits
internal/(off the public API); collapsed the duplicate auth descriptor sovalidateKernelConfig/resolveKernelAuthreturnkernel.Authdirectly; routed the initial-namespace failure-path close throughcall()so it logs; added a live M2M e2e + a last-writer-wins auth regression test; documented U2M interactivity, the dormant U2M scopes/port wiring, and the metric-view smoke test.Known limitations (documented, not blocking)
set_auth_m2mtakes no scopes arg) — a kernel gap shared with ODBC; no authorization impact (all-apisis always requested).Testing
CGO_ENABLED=0suite +golangci-lintv2.12.2 — clean repo-wide.databricks_kernelunit tests — auth-mode → C-setter mapping (PAT/M2M/U2M),quoteIdent, config assembly/classification, error classifier.current_catalog()/current_schema()match config), metric-view (connect+query smoke — the conf isn'tSET-introspectable on either backend), Thrift parity.This pull request and its description were written by Isaac.