Skip to content

Hash-grouped query readers blend INSERT...EXEC callers and label them with one arbitrary text #2012

Description

@erikdarlingdata

The mechanism (reproduced, not theoretical)

INSERT INTO #t EXEC <proc> statements produce the same query_hash across different callee procedures — the hash normalizes the callee away. Reproduced on SQL Server 2022:

query_hash statement host object
0x9C01BDF7F0F05FC4 INSERT INTO #items EXEC dbo.inner_v3 wrapper_v3
0x9C01BDF7F0F05FC4 INSERT INTO #items EXEC dbo.inner_v4 wrapper_v4

This bit during live production triage on 2026-08-01: hash-keyed text served a V3-core's text for stats that belonged to a V4-core call on two of three per-tenant attribution datapoints — the wrong-callee label survived until disambiguated by hand via OBJECT_DEFINITION.

Where it lands in the product

Per-row storage is correct: the collector captures each row's own statement text (offset SUBSTRING) and #1767's payload dims key by content digest, which explicitly rejected query_hash for this class of reason. The corruption is read-side, in every reader that groups by query_hash and picks one representative text:

  • DarlingDataReader.TopQueriesSql (get_top_queries_by_cpu): groups (database_name, query_hash), takes MAX(sql_handle) + a latest-text LATERAL — colliding callers' stats are summed together and the line is labeled with one arbitrary caller's text.
  • ViewerDataService.QueryStats.cs (four hash-grouped sites) and ViewerDataService.FinOps.Workload.cs — same shape.
  • Lite's twins of the above.
  • The plan-analysis MCP tools resolve per-row and are unaffected; query_store_stats composers group by (module_name, query_hash), which already carries the host-object dimension and is much less exposed.

Staged fix

Stage 1 (no schema, this issue): make the merged groups honest. The #1767 digest column is on every row, so the hash-grouped readers can carry COUNT(DISTINCT query_text_digest) AS distinct_texts almost for free. When a group has more than one distinct text, the MCP payload says so (distinct_texts + a caveat that the shown text is one representative — literal variants are the common benign cause, INSERT...EXEC callers the misattributing one) and the Viewer/Lite grids get the count as a column so a "1 of 4 texts" group stops masquerading as a single statement.

Stage 2 (follow-up, schema): capture the host-object identity at collection — sys.dm_exec_sql_text.objectid is already in the collector's OUTER APPLY, just not selected. With object_name on the row, readers group by (query_hash, host object) where an object exists: separates proc-hosted callees exactly, preserves the ad-hoc literal-collapse behavior (objectid NULL) unchanged. New column + migration + reader/CAGG updates, so it gets its own PR.

Validation path: the container repro above feeds a live end-to-end (collect from the repro database, read back through the deduped reader, assert distinct_texts = 2 for the colliding hash); the production monitor then confirms real fleet collisions surface the flag.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions