diff --git a/.github/skills/oncall-weekly-telemetry-report/SKILL.md b/.github/skills/oncall-weekly-telemetry-report/SKILL.md index 70f57e54..57825794 100644 --- a/.github/skills/oncall-weekly-telemetry-report/SKILL.md +++ b/.github/skills/oncall-weekly-telemetry-report/SKILL.md @@ -5,9 +5,9 @@ description: Generate the weekly Android Broker on-call (OCE) WoW + 60-day trend # OCE Weekly Report -Produce the weekly Android Broker on-call (OCE) telemetry report as a self-contained HTML file at `$env:USERPROFILE\android-oce-reports\oncall-wow-report-YYYY-MM-DD.html` (where `YYYY-MM-DD` is the reporting-week Sunday — see "Inputs to confirm" §4). Writes to the user's home folder, **outside the workspace**, so reports never accidentally get committed. +Produce the weekly Android Broker on-call (OCE) telemetry report as a self-contained HTML file at `$env:USERPROFILE\android-oce-reports\oncall-wow-report-YYYY-MM-DD.html` (where `YYYY-MM-DD` is the **end-date of the rolling 7-day window** — see "Inputs to confirm" §1). Writes to the user's home folder, **outside the workspace**, so reports never accidentally get committed. -The output mirrors the structure of the canonical template at [`assets/templates/report-template.html`](assets/templates/report-template.html). The Step 1 bootstrap script copies the template into `~/android-oce-reports/oncall-wow-report-.html` and you edit it in place from there. Do **not** redesign the layout each week. +The output mirrors the structure of the canonical template at [`assets/templates/report-template.html`](assets/templates/report-template.html). The Step 1 bootstrap script copies the template into `~/android-oce-reports/oncall-wow-report-.html`, **stamps the resolved rolling-7-day window into the title / meta line / Generated banner**, and you edit it in place from there. Do **not** redesign the layout each run. **Before writing any KQL, read [`assets/docs/kusto-cheatsheet.md`](assets/docs/kusto-cheatsheet.md).** It captures the canonical view names, helper functions, the HLL device-count gotcha, week-alignment rules, and ready-to-paste query templates — distilled from the production Android Broker Dashboard. @@ -21,33 +21,46 @@ Reusable helpers in [`assets/`](assets/): | [`code-attribution-template.md`](assets/docs/code-attribution-template.md) | Per-card checklist for the deep code-attribution block (Originator / Top throw site / Wrapper / Caller hot-spots / Underlying cause / Top error_messages / Likely PRs / Next step) | | [`queries/`](assets/queries/) | Canonical KQL templates, one file per query — see [`queries/README.md`](assets/queries/README.md). Highlights: [`attr-union-by-dim.kql`](assets/queries/attr-union-by-dim.kql) (NEW — all 7 dims in one round-trip), [`error-message-and-location.kql`](assets/queries/error-message-and-location.kql) (now accepts BOTH `` and `` in one call) | | [`templates/`](assets/templates/) | Copy-paste HTML snippets (`spike-card.html`, `traffic-attr-card.html`, `sparkline-footer.html`) | -| [`bucket-trends.js`](assets/scripts/bucket-trends.js) | Bucket all error codes into 60-day regression / spike / improvement / flat. Run with `--metric=devs` AND `--metric=reqs`. Pass `--end=YYYY-MM-DD` (Sunday after the reporting week, exclusive) to drop the partial in-progress bucket. **`--summary` suppresses the verbose header; `--json=` emits a structured sidecar for programmatic consumption.** | +| [`bucket-trends.js`](assets/scripts/bucket-trends.js) | Bucket all error codes into 60-day regression / spike / improvement / flat. Run with `--metric=devs` AND `--metric=reqs`. Pass `--end=YYYY-MM-DD` (the Sunday that OPENS the current in-progress week, exclusive — i.e. `startofweek(today)`, printed by bootstrap as "Trend delta cutoff") to exclude the partial week from the delta math, plus **`--include-partial-end`** to still chart it as the final bar. **`--summary` suppresses the verbose header; `--json=` emits a structured sidecar for programmatic consumption.** | | [`agg.js`](assets/scripts/agg.js) | Per-error per-dim top-N rollup with WoW deltas. Workhorse for filling spike-attribution dim blocks. | | [`summarize-attribution.js`](assets/scripts/summarize-attribution.js) | Roll up 7-dim attribution slices for spike-attribution cards. Supports BOTH `--union ` (preferred for 2-week WoW; pairs with `attr-union-by-dim.kql`) AND legacy `--label= file.json` per-dim mode. **Auto-detects the array-form schema produced by `assets/scripts/run-kql.ps1` — no schema-transformer step needed.** | | [`find-suspect-prs.ps1`](assets/scripts/find-suspect-prs.ps1) | Parallel `git log -S` + `--grep` across broker/ + common/ for a class/method symbol, with PR numbers + URLs. Run *only after* the Originator pre-check has identified a specific throw-site class — the unscoped 4-week PR window is small enough (<30 PRs) to scan with plain `git log` first. | | [`validate-report.ps1`](assets/scripts/validate-report.ps1) | Pre-publish validator. Catches stale tokens, devs/reqs leaks, mojibake (U+FFFD), unbalanced `
` depth in Section 2 (the nested-callout bug), KPI/trend sparkline coverage, code-attribution depth, layout-guard CSS presence, and suspicious low-peak fabricated `data-trend` arrays. Run as part of Step 7. | | [`scripts/run-kql.ps1`](assets/scripts/run-kql.ps1) | **Direct-REST Kusto helper — drop-in fallback for the Azure Kusto MCP server when the MCP times out** (frequent on per-error-code queries). Acquires a token via `az`, POSTs to `/v2/rest/query`, writes a JSON file the JS helpers can consume directly. | -| [`scripts/bootstrap-report.ps1`](assets/scripts/bootstrap-report.ps1) | Bootstrap a new week's report from the canonical template. Auto-computes the reporting Sunday, creates `_data//`, prunes `_data` folders older than 60 days, and detects "unfilled template stub" vs "real prior report" collisions using a multi-marker fingerprint (title + meta date + first KPI value + size ratio). | +| [`scripts/bootstrap-report.ps1`](assets/scripts/bootstrap-report.ps1) | Bootstrap a new report from the canonical template. Auto-computes the rolling 7-day window (curEnd = today UTC; override with `-EndDate YYYY-MM-DD`), creates `_data//`, prunes `_data` folders older than 60 days, **stamps the resolved window into the `` / `<div class="meta">` block / Generated banner**, and detects "unfilled template stub" vs "real prior report" collisions using a template-only sentinel token (see § collision detection). | | [`scripts/visual-smoke.ps1`](assets/scripts/visual-smoke.ps1) | Optional Playwright-based layout smoke test. Renders the report at 1400 px viewport, captures a full-page screenshot under `~/android-oce-reports/_visual/`, and runs DOM-based overflow + adjacent-card-gap detection. Catches the rendered-layout bugs (text bleed, cards touching) that pure HTML/CSS validation can't see. | --- ## Inputs to confirm with the user -1. **Reporting week** — **first compute the most recent complete Sun→Sat week** (Sunday bucket = the most recent Sunday strictly before today, or today itself if today is a Sunday and the week's data is at least 6h old). Default to that and proceed without asking *unless*: - - today is itself a Sat or Sun **and** the user phrasing suggests they want "this week" (e.g. "current report", "latest data"). Then ASK explicitly between the in-progress and most-recent-complete options. - - today is a Mon–Fri — just default to the most recent complete week and proceed; do not ask. +> **⚠️ Do NOT ask the user for a reporting date by default.** The skill uses a **rolling 7-day window** ending at start-of-day UTC on the invocation day so the most recent complete days are always captured. `assets/scripts/bootstrap-report.ps1` computes and stamps the window automatically. The previous "confirm the Sunday bucket with the user" flow silently produced stale windows (e.g. a Thursday run emitted a 4-day partial window and dropped the last complete week) — that failure mode is what this section explicitly guards against. - If the user picks the in-progress week: - - Add the badge text *"Live data — current bucket may still be filling"* to the report header. - - The `bucket-trends.js` `--end` flag + the `| where week < datetime(<END>)` source filter both still apply (use the Sunday AFTER the reporting week as `<END>`); they will drop the partial-end-bucket warning. +1. **Reporting window** — do NOT ask. Run `bootstrap-report.ps1` with no arguments; it resolves the window silently and prints: + ``` + Resolved reporting window (UTC): # example values for a run on 2026-07-15 + Last 7 days: 2026-07-08 -> 2026-07-15 (exclusive upper bound) + Baseline: 2026-07-01 -> 2026-07-08 + 60-day trend: 2026-05-16 -> 2026-07-15 (literal 60d ending today; chart includes current partial week) + Trend delta cutoff: weeks < 2026-07-12 (startofweek(curEnd); pass as bucket-trends.js --end) + ``` + These dates are stamped into the report's `<title>`, `<div class="meta">`, and Generated banner during bootstrap — you do not hand-edit them (see `assets/templates/template-readme.md` § Date fields). - Note that Kusto's `startofweek()` is **Sunday-aligned**, so a user-spoken "week of May 3 → May 9" maps to the bucket `startofweek == 2026-05-03`. Off-by-one-week is the #1 silent error — verify by printing the distinct `startofweek` buckets from your first query and confirming the label matches the user's intent. -2. **Comparison baseline** — defaults to the prior complete week. -3. **60-day window** — last 8 complete weeks (drop the partial start week when computing trend deltas). -4. **Output filename** — `$env:USERPROFILE\android-oce-reports\oncall-wow-report-YYYY-MM-DD.html`, where `YYYY-MM-DD` is the **Sunday `startofweek` bucket** of the reporting week (e.g. the report for the week of May 3 → May 9, 2026 is `oncall-wow-report-2026-05-03.html`). User-scoped, outside the workspace; the date matches the Kusto bucket label used throughout the report. + **Override only when the user explicitly requests a non-default window.** Signals: "the week of X", "as of last Friday", "the report from three weeks ago", "in-progress data", "just today's numbers". Then use: + ```pwsh + .\bootstrap-report.ps1 -EndDate 2026-07-02 # e.g. reproduce the report as of Jul 2 + ``` + `-EndDate` is the exclusive upper bound of the current window (`curEnd`); the script derives `curStart = curEnd - 7d` and `prevStart = curEnd - 14d` deterministically. `-EndDate` must be today or earlier — future dates are refused. -If any of these are unstated, ask once, then proceed. + **Supported override = `-EndDate` only (shifts the window *end*).** The 7-day primary span, the 7-day baseline, and the 60-day trend form a fixed frame that moves *rigidly* with `-EndDate`. There is **no** custom start-date and **no** arbitrary-span flag — a request like "last 30 days" or "from Jun 1 to Jun 20" is **not** expressible via a single parameter. To produce a longer or custom span you must edit the KQL window placeholders (`<CUR_START>` / `<CUR_END>` / `<PREV_START>`; the baseline end is always `<CUR_START>`, so there is no separate `<PREV_END>` token) by hand. If the user asks for a non-7-day span, say so up front rather than silently emitting a 7-day report. + +2. **Comparison baseline** — auto-computed as the immediately-prior 7 days (`[curEnd - 14d, curEnd - 7d)`). No user input. + +3. **60-day trend window** — auto-computed as the **literal last 60 days ending today** (`[curEnd - 60d, curEnd)`), so both bounds move with `-EndDate`. The section is still Sun-Sat weekly-bucketed (Kusto `startofweek()` is Sunday-aligned) because the trend needs stable weekly denominators, but the final bar is the **current in-progress (partial) week** — the chart ends today. Regression/improvement **delta classification is still computed on complete weeks only** (`bucket-trends.js --end=startofweek(curEnd) --include-partial-end`); a partial week as "last" would read as a fake −99% improvement, so it is charted but excluded from the delta math. `bootstrap-report.ps1` prints both the 60d data window and the "Trend delta cutoff" (= `startofweek(curEnd)`). + +4. **Output filename** — `$env:USERPROFILE\android-oce-reports\oncall-wow-report-YYYY-MM-DD.html` where `YYYY-MM-DD` is the resolved `curEnd`. Example: run on 2026-07-09 (default) → `oncall-wow-report-2026-07-09.html`. User-scoped, outside the workspace. The filename date always matches the meta-line "Last 7 days" end-date; `validate-report.ps1` check #11 asserts this. + +**Kusto note (60-day trend section only):** `startofweek()` is Sunday-aligned, so `startofweek('2026-05-09') == 2026-05-03T00:00:00Z`. When authoring weekly-bucketed queries (60-day trend, `wow-table-sparkline-series.kql`), verify by printing the distinct `startofweek(EventInfo_Time)` values from your first query. Off-by-one-week is the #1 silent error in weekly-bucket queries. --- @@ -60,7 +73,7 @@ If any of these are unstated, ask once, then proceed. - **Slow-burn 60-day regressions** — codes/types climbing on the 60d window that are flat WoW. Anything that *also* moved WoW belongs in the red callout above (with `60d↑`), not here. Link to the 60-Day Trend section. - **Real wins this week**, with PR links. - **Traffic shape** — flat / surge / collapse summary. -3. **📈 60-Day Trend Analysis** — built from the `ErrorStatsMetrics` materialized view over the last 8 complete weeks. **Run the bucketing pipeline FOUR times — the cross-product of `{error_code, error_type} × {devices, requests}`** — and union the regression sets. An entry (code OR type) is flagged if it regresses on either metric. +3. **📈 60-Day Trend Analysis** — built from the `ErrorStatsMetrics` materialized view over the **literal last 60 days ending today** (final bar = current in-progress week). **Run the bucketing pipeline FOUR times — the cross-product of `{error_code, error_type} × {devices, requests}`** — and union the regression sets. An entry (code OR type) is flagged if it regresses on either metric. Deltas are computed on complete weeks only; the partial current week is charted but excluded from classification. - **% of devices** affected (`devicesHit / authActiveDevices`) — catches errors hitting more users. - **% of requests** affected (`errRequests / authTotalRequests`) — catches per-device retry storms (fewer users, more traffic per user). The previous report would have missed `kdfv2_key_derivation_error` (262 → 5,374 requests on ~57 devices) without this dim. @@ -83,20 +96,21 @@ If any of these are unstated, ask once, then proceed. ### Step 1 — Bootstrap the new report file from the template -This skill ships with a canonical template at [`assets/templates/report-template.html`](assets/templates/report-template.html) (a real prior week's report kept as the reference layout). **Use [`assets/scripts/bootstrap-report.ps1`](assets/scripts/bootstrap-report.ps1)** to handle all the boilerplate (Sunday-date computation, `_data/<sunday>/` directory, retention-pruning, collision detection): +This skill ships with a canonical template at [`assets/templates/report-template.html`](assets/templates/report-template.html) (a real prior report kept as the reference layout). **Use [`assets/scripts/bootstrap-report.ps1`](assets/scripts/bootstrap-report.ps1)** to handle all the boilerplate (rolling-window computation, `_data/<end-date>/` directory, header stamping, retention-pruning, collision detection): ```pwsh .\.github\skills\oncall-weekly-telemetry-report\assets\scripts\bootstrap-report.ps1 -# Optional: explicit reporting Sunday + force overwrite -# .\bootstrap-report.ps1 -ReportingSunday 2026-05-31 -Force +# Optional: explicit end-date (curEnd, exclusive upper bound) + force overwrite +# .\bootstrap-report.ps1 -EndDate 2026-07-02 -Force ``` What it does: -* Computes the reporting-Sunday from the system clock (most recent complete Sun-Sat week). -* Creates `~/android-oce-reports/oncall-wow-report-<sunday>.html` from the canonical template. -* Creates `~/android-oce-reports/_data/<sunday>/` for raw KQL JSON payloads. -* Prunes `_data/<old-sunday>/` folders older than 60 days so the cache doesn't accumulate. -* **Collision detection (the v8-hardened version):** uses a multi-marker fingerprint (title + meta-line dates + first-KPI value + size ratio) to distinguish an "unfilled template stub" (silently re-bootstrap) from a "real populated report" (HARD HALT, exit 2, require `-Force` to overwrite). The earlier single-marker (title only) version misclassified populated reports as stubs and overwrote real work. +* Resolves the rolling 7-day window from the system clock in UTC (`curEnd = today`, `curStart = curEnd - 7d`, `prevStart = curEnd - 14d`) — or from `-EndDate` if passed. +* Creates `~/android-oce-reports/oncall-wow-report-<curEnd>.html` from the canonical template. +* Creates `~/android-oce-reports/_data/<curEnd>/` for raw KQL JSON payloads. +* **Stamps the resolved window into the report's `<title>`, `<div class="meta">` block, and Generated banner** — you never hand-edit header dates. The resolved window is echoed in the report header for transparency. +* Prunes `_data/<old-end-date>/` folders older than 60 days so the cache doesn't accumulate. +* **Collision detection (fail-safe):** an existing same-day report is silently re-bootstrapped only when it is *positively* identified as an unpopulated stub — it still carries the `OCE-UNPOPULATED-STUB` sentinel that bootstrap injects **and** its first KPI still equals the template's value. Anything else (sentinel removed, or KPIs edited) is treated as real work: **HARD HALT, exit 2**, requiring `-Force` to overwrite. `validate-report.ps1` refuses to pass a report that still carries the sentinel, so a published report can never be misclassified as a stub. Edit the bootstrapped file in place — the template ships as a real prior-week report (not a tokenized skeleton). **Walk top-to-bottom and replace every prior-week date / KPI value / table row / verdict / PR citation with current-week data.** The CSS, sparkline JS, section ordering, and attribution-card markup are canonical — do not redesign them. See [`assets/templates/template-readme.md`](assets/templates/template-readme.md) for the full guide on what to change vs leave alone, the sparkline color palette, the CSS class reference, and the two v8 layout traps. @@ -154,32 +168,30 @@ Don't pre-filter to a hand-picked top-N list — small-but-rising errors (e.g. ` #### 3a. Per-error-code trend -Use [`assets/queries/60d-trend-codes.kql`](assets/queries/60d-trend-codes.kql) (template; replace `<START>` and `<END>` tokens — `<END>` is **exclusive** and equals the Sunday AFTER the reporting week, e.g. for a 2026-05-03 report use `2026-05-10`): +Use [`assets/queries/60d-trend-codes.kql`](assets/queries/60d-trend-codes.kql) (template; replace `<TREND_START>` and `<TREND_END>` tokens. **`<TREND_START>` = `curEnd − 60d`** and **`<TREND_END>` = `curEnd` (today), exclusive** — the literal last 60 days. `bootstrap-report.ps1` prints the resolved values): ```kql materialized_view('ErrorStatsMetrics') -| where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) +| where EventInfo_Time >= datetime(<TREND_START>) and EventInfo_Time < datetime(<TREND_END>) | where isnotempty(error_code) and error_code != 'success' | summarize errs = sum(countOverall), devs = dcount_hll(hll_merge(countDevicesHll)) by week = startofweek(EventInfo_Time), error_code -| where week < datetime(<END>) // drop partial in-progress week at the source | order by error_code asc, week asc ``` -**The `| where week < datetime(<END>)` line is mandatory.** Without it, if Kusto has crossed midnight UTC into the next Sunday, a tiny partial bucket lands as `last` and turns every code into a fake −99% improvement. `bucket-trends.js` will also auto-detect and warn about this, but filtering at the source is preferred. +**Do NOT filter the partial in-progress week here.** The chart wants it as the final bar (the window ends today). The partial week is excluded from the regression/improvement **delta math** by `bucket-trends.js` via `--end=<TREND_CLASS_END> --include-partial-end` (see 3c), not at the source — a partial week driving the delta would read as a fake −99% improvement, which is exactly why classification and display are split in the JS. #### 3b. Per-error-type trend (same rigor) ```kql materialized_view('ErrorStatsMetrics') | extend unified_error_type = MergeUiRequiredExceptions(error_type) -| where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) +| where EventInfo_Time >= datetime(<TREND_START>) and EventInfo_Time < datetime(<TREND_END>) | where isnotempty(unified_error_type) | summarize errs = sum(countOverall), devs = dcount_hll(hll_merge(countDevicesHll)) by week = startofweek(EventInfo_Time), unified_error_type -| where week < datetime(<END>) | order by unified_error_type asc, week asc ``` @@ -190,16 +202,19 @@ materialized_view('ErrorStatsMetrics') `bucket-trends.js` defaults to grouping by `error_code`. For the type runs you MUST pass `--key=unified_error_type` so it picks up the right column from the type-trend JSON. ```pwsh -# Error codes — by devices, then by requests -node .github\skills\oncall-weekly-telemetry-report\assets\scripts\bucket-trends.js <codes.json> --start=2026-03-08 --end=2026-05-10 -node .github\skills\oncall-weekly-telemetry-report\assets\scripts\bucket-trends.js <codes.json> --start=2026-03-08 --end=2026-05-10 --metric=reqs +# Error codes — by devices, then by requests. +# TREND_START = curEnd - 60d (literal 60d start) +# TREND_CLASS_END = startofweek(today) ("Trend delta cutoff" printed by bootstrap) +# --include-partial-end charts the current partial week while excluding it from deltas. +node .github\skills\oncall-weekly-telemetry-report\assets\scripts\bucket-trends.js <codes.json> --start=<TREND_START> --end=<TREND_CLASS_END> --include-partial-end +node .github\skills\oncall-weekly-telemetry-report\assets\scripts\bucket-trends.js <codes.json> --start=<TREND_START> --end=<TREND_CLASS_END> --include-partial-end --metric=reqs # Error types — by devices, then by requests (note --key) -node .github\skills\oncall-weekly-telemetry-report\assets\scripts\bucket-trends.js <types.json> --start=2026-03-08 --end=2026-05-10 --key=unified_error_type -node .github\skills\oncall-weekly-telemetry-report\assets\scripts\bucket-trends.js <types.json> --start=2026-03-08 --end=2026-05-10 --key=unified_error_type --metric=reqs +node .github\skills\oncall-weekly-telemetry-report\assets\scripts\bucket-trends.js <types.json> --start=<TREND_START> --end=<TREND_CLASS_END> --include-partial-end --key=unified_error_type +node .github\skills\oncall-weekly-telemetry-report\assets\scripts\bucket-trends.js <types.json> --start=<TREND_START> --end=<TREND_CLASS_END> --include-partial-end --key=unified_error_type --metric=reqs ``` -`--end` is the Sunday AFTER the reporting week (exclusive). The script also auto-detects partial end-buckets and warns, but passing `--end` explicitly is safer. +`--end` is `<TREND_CLASS_END>` = `startofweek(today)` (exclusive) — the Sunday that opens the current in-progress week. Weeks at or after it (the partial current week) are excluded from delta classification; `--include-partial-end` keeps that week in the emitted `series` so the chart ends today. The script also auto-detects partial end-buckets and warns if `--end` is omitted, but passing it explicitly is safer. Take the **union** of all four regression sets. Both `error_code` and `error_type` regressions get a spike-attribution card in Step 5. @@ -222,15 +237,15 @@ It will print regression / spike / improvement / flat buckets, sorted by peak. T The 60d bucketer's `--peak-floor=10000` exists for good reason (otherwise the 60d regression list would be 200+ tiny noise codes), but it **silently drops every code whose absolute weekly volume stays under 10K** — even if that code is brand-new or just spiked 5× WoW. Real examples this skill has missed in the past: - `Failed to parse JWT` — went `7 → 32 → 54 → 46 → 55 → 892 → 3,461` over 7 weeks (2-week-old NEW spike, real broker code in `IDToken.parseJWT:38`). Never crossed the 10K floor. -- `Code:-11` — sat at ~1,030 devs/wk for 7 weeks then jumped to 2,433 (+165% WoW). Sub-floor. +- `Code:-11` — sat at ~1,030 devs/week for 7 weeks then jumped to 2,433 (+165% WoW). Sub-floor. - `SSLHandshakeException` — devices flat at 260 but requests +186% WoW (per-device retry storm). The bucketer's reqs-axis floor (100K) just barely captures it but the device floor doesn't. To catch these, **always** run [`assets/queries/wow-movers.kql`](assets/queries/wow-movers.kql) **as a separate pass after the 60d bucketing**: ```kql -// inputs: <CURR_START> = reporting-week Sunday, <CURR_END> = next Sunday (excl), -// <PRIOR_START> = baseline-week Sunday -// floor: cDev>=500 OR cReq>=5000 move: |Δd|>=25% OR |Δr|>=50% OR new-this-week +// inputs: <CUR_END> = curEnd (exclusive), <CUR_START> = curEnd - 7d, +// <PREV_START> = curEnd - 14d. Printed by bootstrap-report.ps1. +// floor: cDev>=500 OR cReq>=5000 move: |Δd|>=25% OR |Δr|>=50% OR new-this-window ``` Run it **twice — once for `error_code`, once for `error_type`**. **Merge its output rows into the same 🔴 WoW regressions callout as the standard WoW table** (sorted by current-week device count descending). Tag rows that came in via this pass with `NEW` if they were absent or near-zero in the prior week. Do *not* render this as a separate "emerging" callout — the size split is implementation detail; readers prioritize naturally by absolute device count + originator chip. @@ -241,7 +256,7 @@ For each WoW mover (regardless of size), you still owe the full Code Attribution > ⚠️ **HARD RULE — Originator pre-check.** Before claiming `Originator: Broker` on any card, you MUST run [`assets/queries/error-message-and-location.kql`](assets/queries/error-message-and-location.kql) for that error code (or type) and read **(a) the throw-site stack and (b) the top 3 `error_message` strings**. Most broker error codes flow through `common/ExceptionAdapter.{getExceptionFromTokenErrorResponse, exceptionFromAuthorizationResult, clientExceptionFromException}` — which intentionally bridge eSTS responses into broker exceptions. **If the throw site is in any of those three methods AND the error_message starts with `AADSTS`, the originator is eSTS, not broker.** See the AADSTS reference table in [`assets/docs/kusto-cheatsheet.md`](assets/docs/kusto-cheatsheet.md). Cards that skip this step must be marked low-confidence, not high. > -> **Window:** use the FULL 7-day reporting window (`<CURR_START>` → `<CURR_END>`) on `PipelineInfo_IngestionTime`, NOT a narrower 3–5 day slice — low-volume types (e.g. `SSLHandshakeException`, `IntuneAppProtectionPolicyRequiredException`) routinely return zero rows in a sub-week window. If a code/type still returns nothing, fall back to the prior 14 days before declaring "no data". +> **Window:** use the FULL 7-day rolling window (`<CUR_START>` → `<CUR_END>`) on `PipelineInfo_IngestionTime`, NOT a narrower 3–5 day slice — low-volume types (e.g. `SSLHandshakeException`, `IntuneAppProtectionPolicyRequiredException`) routinely return zero rows in a sub-window slice. If a code/type still returns nothing, fall back to the prior 14 days (`<PREV_START>` → `<CUR_END>`) before declaring "no data". For every regression card, the Code Attribution block **must** populate the following fields. Shallow PR-citation only is not acceptable. Use [`assets/docs/code-attribution-template.md`](assets/docs/code-attribution-template.md) as the per-card checklist. @@ -309,7 +324,7 @@ Slice on **all 7 dimensions** for each spike. **Preferred for 2-week WoW attribu For `error_type` cards, swap `error_code in (codes)` for `unified_error_type in (types)` and aggregate by the `MergeUiRequiredExceptions(error_type)` extension — otherwise everything else is identical. -> **Low-volume fallback (extends Step 4's pre-check fallback to the 7-dim union):** when a code/type returns sparse dim rows in the 7-day reporting window — typical for sub-1k-device entries like `TimeoutCancellationException`, `JsonSyntaxException`, `kdfv2_key_derivation_error` — widen the union query to **14 days** (`<START>` = baseline-week Sunday − 7d) before declaring "broad — needs targeted slice". The added week of context usually surfaces enough rows to compute concentration percentages. If a code STILL has no concentration after 14 days, mark every dim cell as "not sliced — sub-week volume; file the bug first, slice on persistence" — do NOT fabricate "Broad" verdicts. +> **Low-volume fallback (extends Step 4's pre-check fallback to the 7-dim union):** when a code/type returns sparse dim rows in the 7-day rolling window — typical for sub-1k-device entries like `TimeoutCancellationException`, `JsonSyntaxException`, `kdfv2_key_derivation_error` — widen the union query to **14 days** (use `<PREV_START>` as the lower bound so the window becomes `[curEnd − 14d, curEnd)`) before declaring "broad — needs targeted slice". The added week of context usually surfaces enough rows to compute concentration percentages. If a code STILL has no concentration after 14 days, mark every dim cell as "not sliced — sub-window volume; file the bug first, slice on persistence" — do NOT fabricate "Broad" verdicts. | # | Dimension | Source | Cross-check | |---|-----------|--------|-------------| @@ -326,16 +341,19 @@ For `error_type` cards, swap `error_code in (codes)` for `unified_error_type in Because `error_type` is an umbrella over many `error_code` values, every `error_type` regression card MUST also include an **8th dimension: sub-code breakdown** showing the top 3–5 `error_code`s rolled up under that type, with their device counts and Δ vs prior week. This lets the reader see whether the type-level move is driven by one sub-code or many — and routes the deep Code Attribution work to the right sub-code. ```kql +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let target_types = dynamic(['ClientException', 'ServiceException']); materialized_view('ErrorStatsMetrics') | extend unified_error_type = MergeUiRequiredExceptions(error_type) -| where EventInfo_Time > ago(14d) +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd | where unified_error_type in (target_types) -| extend wk = startofweek(EventInfo_Time) +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by wk, unified_error_type, error_code -| order by unified_error_type asc, wk asc, devs desc + by week, unified_error_type, error_code +| order by unified_error_type asc, week asc, devs desc ``` Cite the dominant sub-codes inline in the type card's verdict (e.g. *"`ClientException` −10.2% drop is dominated by −8.5 pp `timed_out_execution` + −3.4 pp `unknown_authority`"*) and link to those sub-codes' own attribution cards. The deep Code Attribution block (Step 4) for the type card itself focuses on the **wrapper / catch-and-rethrow** path that defines the type (e.g. `BaseException.java`, `ServiceException.java` constructors), not on each sub-code. @@ -430,12 +448,16 @@ Cite the suspect PR(s) with the same confidence ratings used in Code Attribution **6c. Per-error traffic attribution (is the *error* spike traffic-driven?).** For every error code flagged in Step 5 as a regression, additionally check whether the spike is *traffic-driven* rather than *failure-rate-driven*: ```kql +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let target_code = "<error_code>"; materialized_view('ErrorStatsMetrics') -| where EventInfo_Time > ago(14d) and error_code == target_code +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd and error_code == target_code +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize errs = sum(countOverall), devs = dcount_hll(hll_merge(countDevicesHll)) - by week = startofweek(EventInfo_Time), calling_package_name + by week, calling_package_name | order by week asc, devs desc ``` @@ -464,7 +486,7 @@ The validator hard-fails on: 6. **Chartless KPI grid** — if more than half the `.kpi` tiles lack a `data-spark` element (catches the v7 regression where the body was rebuilt without sparklines). Also warns when total chart count (sparks + trends + inline svgs) is < 15. 7. **Code-attribution depth** — each `.attr-card`'s "Code attribution" block must contain an `Originator` row (proxy for the full 8-field structure: Originator / Top throw site / Wrapper / Caller hot-spots / Underlying cause / Top error_messages / Likely PRs / Next step). Catches the v7-third-pass regression where cards shipped with a `pr-list`-only stub. 8. **Attribution-card layout guards (v8)** — the CSS must define `.attr-card { margin-bottom: 16px }` AND `.dim-row` overflow rules (`text-overflow: ellipsis` + `min-width: 0`). Catches the "cards touching" and "text bleeding out of dim boxes" regressions from a stale `<head>` block. -9. **Fabricated-sparkline heuristic (v8)** — warns when a `data-trend` array's peak value is < 100 (almost certainly hand-rolled rather than sourced from real data). See [`assets/queries/wow-table-sparkline-series.kql`](assets/queries/wow-table-sparkline-series.kql) for the canonical KQL that pulls real 8-week series for every code in the WoW tables. +9. **Fabricated-sparkline heuristic (v8)** — warns when a `data-trend` array's peak value is < 100 (almost certainly hand-rolled rather than sourced from real data). See [`assets/queries/wow-table-sparkline-series.kql`](assets/queries/wow-table-sparkline-series.kql) for the canonical KQL that pulls real 8-week series for every code in the WoW tables. Its `<SPARK_START>` / `<SPARK_END>` tokens are the last **8 complete** Sun-Sat weeks (`<SPARK_END>` = `startofweek(today)`, exclusive) — deliberately distinct from the trend-chart's `<TREND_START>` / `<TREND_END>` (literal last 60 days ending today). Per-row sparklines stay on complete weeks so a partial final point doesn't create a misleading dip in every WoW row. Then: - **Run the visual smoke test (recommended)** — catches rendered-layout bugs that pure HTML/CSS validation can't see: @@ -491,9 +513,9 @@ Then: - **Never sum percentiles.** Latency is a TDigest sketch — `percentile_tdigest(tdigest_merge(responseTimeTDigest), N, typeof(long))` only. - **Always apply `MergeAccountType` / `MergeIsSharedDevice` / `MergeUiRequiredExceptions`** so this report agrees with the dashboard. - **Confirm the week bucket label matches the user's intent** before writing the rest of the queries (Sunday-aligned). -- **Always filter the partial in-progress week at the source** with `| where week < datetime(<END>)` where `<END>` is the Sunday immediately after the reporting week. Otherwise `bucket-trends.js` will show every error as a fake −99% improvement once UTC has crossed midnight Sunday. -- **Never carry a numeric telemetry value forward between runs.** Every KPI, table cell, delta %, device/request count, sparkline point, and verdict number must be re-pulled from Kusto for *this* week — never copied from last week's report, from a checkpoint/summary, from notes, or from memory. Telemetry shifts week to week and stale numbers read as fabricated. Near-miss precedent: a `no_tokens_found` count was about to be carried as ~23.7M when the actual current-week value was ~4.86M — a ~5× error that only the re-pull caught. If a number isn't backed by a query result file in this run's `_data/<sunday>/`, it does not go in the report. -- **Never hardcode the "Generated" date.** It is the *run* date (system clock, local), auto-stamped by `bootstrap-report.ps1`. If you rebuild the body programmatically, derive it live with a **local-date** formatter (`new Date().toLocaleDateString('en-CA')` in Node, `(Get-Date).ToString('yyyy-MM-dd')` in PowerShell) — never paste a literal, and avoid `new Date().toISOString().slice(0,10)` (UTC; stamps tomorrow's date when run in the evening in a UTC-negative timezone). The v8 "Generated 2026-06-15 on a 2026-06-18 file" bug came from a hardcoded string in the assembler. (Reporting-week / baseline / 60d window dates are author-set and verified against the user's intended Sunday bucket — see template-readme "Date fields".) +- **Do NOT filter the partial in-progress week at the source in the 60-day trend queries** — the chart ends today and wants that partial week as its final bar. Exclude it from the regression/improvement **delta math** instead by running `bucket-trends.js --end=<startofweek(today)> --include-partial-end`: the `--end` cutoff drops the partial week from first/last/delta classification while `--include-partial-end` keeps it in the emitted `series`. Skipping `--end` (or the cutoff) would make `bucket-trends.js` show every error as a fake −99% improvement. The per-row `wow-table-sparkline-series.kql` is the exception — it keeps 8 complete weeks (`<SPARK_END>` = `startofweek(today)`, with the partial week filtered at the source) so no WoW row ends on a misleading partial dip. +- **Never carry a numeric telemetry value forward between runs.** Every KPI, table cell, delta %, device/request count, sparkline point, and verdict number must be re-pulled from Kusto for *this* run — never copied from a previous report, from a checkpoint/summary, from notes, or from memory. Telemetry shifts between runs and stale numbers read as fabricated. Near-miss precedent: a `no_tokens_found` count was about to be carried as ~23.7M when the actual current-window value was ~4.86M — a ~5× error that only the re-pull caught. If a number isn't backed by a query result file in this run's `_data/<end-date>/`, it does not go in the report. +- **Never hardcode the "Generated" date.** It is the *run* date in **UTC**, auto-stamped by `bootstrap-report.ps1` (which uses `(Get-Date).ToUniversalTime()`). If you rebuild the body programmatically, derive it live with a **UTC-date** formatter (`new Date().toISOString().slice(0,10)` in Node, `[datetime]::UtcNow.ToString('yyyy-MM-dd')` in PowerShell) — never paste a literal, and stay on UTC so the assembler can never stamp a different day than `bootstrap-report.ps1`. The v8 "Generated 2026-06-15 on a 2026-06-18 file" bug came from a hardcoded string in the assembler. (Reporting-week / baseline / 60d window dates are author-set and verified against the user's intended Sunday bucket — see template-readme "Date fields".) - **Originator pre-check is mandatory.** A card cannot claim `Originator: Broker` without first running [`assets/queries/error-message-and-location.kql`](assets/queries/error-message-and-location.kql) and reading the throw site + top 3 `error_message` strings. If the throw site is in `common/ExceptionAdapter.{getExceptionFromTokenErrorResponse, exceptionFromAuthorizationResult}` AND the message starts with `AADSTS`, the originator is **eSTS, not broker** — see the AADSTS reference in [`assets/docs/kusto-cheatsheet.md`](assets/docs/kusto-cheatsheet.md). - **WoW-movers pass is mandatory.** The 60d bucketer's `--peak-floor` silently drops sub-10K-device codes, so [`assets/queries/wow-movers.kql`](assets/queries/wow-movers.kql) MUST be run as a separate pass for both `error_code` and `error_type` (per Step 3d). Its output is **merged into the single 🔴 WoW regressions callout**, sorted by current-week device count descending, with rows tagged `NEW` / `60d↑` / originator chip. Do not render a separate "emerging" callout. Skipping the pass is how the Apr 26 `Failed to parse JWT` spike (7 → 3,461 devs over 7 weeks) hid for two reports running. - **Section 2 callouts are at-a-glance, Section 4 is the deep dive.** WoW / Slow-burn / Wins items in Section 2 use the `.item` flat-row pattern (no nested cards, no per-item left bars — the parent `.callout` border is the only severity affordance). Each row is a single line of metric chips + a one-line body + an `Attribution card →` link to the corresponding `.attr-card` in Section 4. Do NOT duplicate the dim slicing, PR analysis, or detailed verdict between the two sections — Section 4 is where that lives. See [`assets/templates/template-readme.md`](assets/templates/template-readme.md) for the CSS class reference and the example `.item` markup. @@ -512,9 +534,9 @@ Then: ## Output checklist -- [ ] New `oncall-wow-report-YYYY-MM-DD.html` (where `YYYY-MM-DD` is the reporting-week Sunday) exists at `$env:USERPROFILE\android-oce-reports\` (NOT at repo root). If a file for this Sunday already existed, the chat session explicitly stated what changed before regenerating. +- [ ] New `oncall-wow-report-YYYY-MM-DD.html` (where `YYYY-MM-DD` is the resolved `curEnd` — the end-date of the rolling 7-day window) exists at `$env:USERPROFILE\android-oce-reports\` (NOT at repo root). If a file for this end-date already existed, the chat session explicitly stated what changed before regenerating. - [ ] All sections present and populated (incl. 🚚 Traffic Attribution — even if “None this week”) -- [ ] **60-day trend bucketing run on the full cross-product** — `{error_code, error_type} × {devices, requests}` = 4 runs — union of regressions reported. Per-request retry storms (e.g. small device pool, exploding request count) are flagged on both axes. Source KQL filtered the partial in-progress week with `| where week < datetime(<END>)`. +- [ ] **60-day trend bucketing run on the full cross-product** — `{error_code, error_type} × {devices, requests}` = 4 runs — union of regressions reported. Per-request retry storms (e.g. small device pool, exploding request count) are flagged on both axes. Source KQL spans the literal last 60 days ending today (no source-side partial-week filter); the partial current week is excluded from delta classification via `bucket-trends.js --end=<startofweek(today)> --include-partial-end` and charted as the final bar. - [ ] **WoW-movers pass run** ([`wow-movers.kql`](assets/queries/wow-movers.kql)) for BOTH `error_code` and `error_type`. Its output rows are **merged into the single 🔴 WoW regressions callout in Section 2** (sorted by curr-week devices descending), each row tagged `NEW` / `60d↑` / originator chip. No separate "emerging" callout. Every row carries throw-site, dominant message, originator, and a next step. If the WoW callout is empty (rare), render "None this week" rather than omit. - [ ] **Both error-codes AND error-types WoW tables have `Δ requests %` and `Δ devices %` columns**, the 60d sparkline, and a status pill. Any row crossing threshold on either metric is in the regression list. - [ ] Every WoW regression AND every 60d regression — **for both `error_code` and `error_type`** — has its own spike-attribution card with all 7 dimensions sliced. Cards are built from [`assets/templates/spike-card.html`](assets/templates/spike-card.html). @@ -527,7 +549,7 @@ Then: - [ ] Denominator caveat (if used) is backed by [`broker-version-share-wow.kql`](assets/queries/broker-version-share-wow.kql) or [`broker-version-share.kql`](assets/queries/broker-version-share.kql) evidence naming the responsible version cohort. No hand-waving. - [ ] Auth-only denominator used for all reliability %s, denominator caveat called out at top. - [ ] No `\bdevs\b` or `\breqs\b` in user-facing text. (`Select-String -Pattern '\bdevs\b|\breqs\b' -CaseSensitive:$false` returns 0.) -- [ ] **Sparklines rendered.** Every `.kpi` tile in the Top-line health section has a `data-spark` array with 8–9 weekly values. Every row in the 60-day trend tables and both WoW tables (codes + types) has a `data-trend` mini-spark. The validator's chart-coverage check passes (KPI coverage ≥1/2 of tiles, total elements ≥15). Past failure mode: the v7 body rebuild dropped all sparklines silently — see `template-readme.md` § "Sparklines are MANDATORY". +- [ ] **Sparklines rendered.** Every `.kpi` tile in the Top-line health section has a `data-spark` array with 8–9 weekly values. Every row in the 60-day trend tables and both WoW tables (codes + types) has a `data-trend` mini-spark. Note the 60-day trend `data-trend` arrays now end on the current partial week (≈9 points incl. the in-progress bar), while the WoW-table sparklines keep 8 complete weeks. The validator's chart-coverage check passes (KPI coverage ≥1/2 of tiles, total elements ≥15). Past failure mode: the v7 body rebuild dropped all sparklines silently — see `template-readme.md` § "Sparklines are MANDATORY". - [ ] **Code-attribution depth.** Every `.attr-card`'s Code attribution block uses the full 8-field `<div class="origin-row">` structure (Originator / Top throw site / Wrapper / Caller hot-spots / Underlying cause / Top error_messages / Likely PRs / Next step) per [`assets/docs/code-attribution-template.md`](assets/docs/code-attribution-template.md). A `pr-list`-only stub is **not acceptable** — the validator hard-fails this. Past failure mode (v7 third pass): all 10 cards shipped with PR-only stubs and lost the throw-site / wrapper / underlying-cause analysis. - [ ] No stale text from previous weeks. (`Select-String -Pattern 'EXAMPLE CONTENT BELOW'` returns 0 — that's the unfinished-section sentinel. The template no longer ships `{{TOKEN}}` placeholders since v2; if the file still contains any `{{`, that's also a leftover.) - [ ] `get_errors` clean on the HTML file. diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/docs/kusto-cheatsheet.md b/.github/skills/oncall-weekly-telemetry-report/assets/docs/kusto-cheatsheet.md index ebc4c9de..c09c2a6c 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/docs/kusto-cheatsheet.md +++ b/.github/skills/oncall-weekly-telemetry-report/assets/docs/kusto-cheatsheet.md @@ -38,6 +38,45 @@ Time filter on materialized views is always **`EventInfo_Time`**. Use `PipelineI --- +## 3. Rolling 7-day WoW window (PRIMARY / attribution / latency) + +**The report's primary window is a **rolling 7-day window** ending at start-of-day UTC on `-EndDate` (default: today), NOT a Sun→Sat calendar week.** Only the 60-day trend section (§ 7 below) still uses `startofweek()` bucketing. + +Canonical template — two rows per key (`week` in `{prevStart, curStart}`): + +```kql +let curEnd = datetime(<CUR_END>); // exclusive upper bound (e.g. 2026-07-09) +let curStart = datetime(<CUR_START>); // curEnd - 7d +let prevStart = datetime(<PREV_START>); // curEnd - 14d +materialized_view('<view>') +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) +| summarize <aggregations> + by week, <keys> +| order by <keys> asc, week asc +``` + +Notes: +- Use `>= ... and < ...` (half-open) NOT `between (.. .. ..)`. `between` is inclusive-inclusive in Kusto; the half-open form correctly assigns the exact `curEnd` boundary to no bucket (the query drops it because `EventInfo_Time < curEnd`). +- The window boundaries are always aligned to `00:00 UTC` — this matches Kusto's default datetime semantics and the bootstrap script's `-EndDate` interpretation. +- `week` values are datetimes; they sort lexicographically (ISO 8601) so downstream JS helpers (`agg.js`, `summarize-attribution.js`) that pick smallest = prev, largest = cur continue to work unchanged. +- All primary/WoW `.kql` templates in `../queries/` follow this pattern. See `../queries/reliability-auth-only.kql` and `../queries/attr-union-by-dim.kql` for full examples. + +Compute the placeholder values via `bootstrap-report.ps1` (which prints them to stdout as `Resolved reporting window (UTC):`) or manually: + +| Placeholder | Formula | Example (`-EndDate 2026-07-09`) | +|---|---|---| +| `<CUR_END>` | `-EndDate` | `2026-07-09` | +| `<CUR_START>` | `<CUR_END> - 7d` | `2026-07-02` | +| `<PREV_START>` | `<CUR_END> - 14d` | `2026-06-25` | +| `<TREND_END>` | `<CUR_END>` (today) — literal 60d trend upper bound, exclusive | `2026-07-09` | +| `<TREND_START>` | `<CUR_END> - 60d` | `2026-05-10` | +| `<TREND_CLASS_END>` | `startofweek(<CUR_END>)` — delta cutoff, `bucket-trends.js --end` | `2026-07-05` | +| `<SPARK_END>` | `startofweek(<CUR_END>)` — WoW-sparkline upper bound (8 complete weeks) | `2026-07-05` | +| `<SPARK_START>` | `<SPARK_END> - 56d` | `2026-05-10` | + +--- + ## 3. THE distinct-device-count gotcha (most important rule) `countDevices` on `ErrorStats*` is a **per-row distinct count, not additive**. If you sum it across multiple rows you will double-count any device that appeared in more than one slice. **The dashboard never does this.** Every dashboard query computes devices via: @@ -101,10 +140,14 @@ materialized_view('PerfStatsUpdated') ## 7. Week alignment — Kusto `startofweek()` is **Sunday-aligned** -If a user says "the week of May 2 → May 9", Kusto buckets it as `startofweek('2026-05-09') == 2026-05-03T00:00:00Z`. **Always confirm**: print the distinct `startofweek(EventInfo_Time)` values from your first query and verify the bucket label matches the user's intent. Off-by-one-week is the #1 silent error. +> **Scope:** only the 60-day trend section (§ 3 of the report / `bucket-trends.js` pipeline) still uses `startofweek()` weekly buckets. The primary/WoW section uses a rolling 7-day window — see § 3 of this cheatsheet. -For an 8-complete-week 60-day window ending Sat May 9, the buckets are: -`2026-03-08, 03-15, 03-22, 03-29, 04-05, 04-12, 04-19, 04-26, 05-03` — that's 9 buckets, one of which (the first) was a partial start. Drop the first; keep 8 complete weeks. +If a user says "the week of May 2 → May 9", Kusto buckets it as `startofweek('2026-05-09') == 2026-05-03T00:00:00Z`. When writing weekly-bucketed queries (60-day trend, `wow-table-sparkline-series.kql`), **always confirm**: print the distinct `startofweek(EventInfo_Time)` values from your first query and verify the bucket labels match your intended range. Off-by-one-week is the #1 silent error in weekly-bucket queries. + +For the literal-60-day trend window ending today (say `<CUR_END> = 2026-07-09`, so `<TREND_START> = 2026-05-10`), `startofweek()` produces buckets: +`2026-05-03, 05-10, 05-17, 05-24, 05-31, 06-07, 06-14, 06-21, 06-28, 07-05` — the first (`05-03`) is a partial start (window opens mid-week `05-10`) and is dropped by `bucket-trends.js --start`; the last (`07-05`) is the **partial current week** — it is charted as the final bar but excluded from delta classification via `--end=<TREND_CLASS_END>` (= `startofweek(today)` = `2026-07-05`) `--include-partial-end`. + +The `wow-table-sparkline-series.kql` per-row sparklines are different: they stay on the last **8 complete** weeks (`<SPARK_END> = startofweek(today)`, exclusive, partial week filtered at the source) so no WoW row ends on a misleading partial dip. --- @@ -148,9 +191,11 @@ union s, i ### 8b. 60-day error trend (feeds `bucket-trends.js`) +Literal last 60 days ending today (`<TREND_START> = CUR_END - 60d`, `<TREND_END> = CUR_END`). Do NOT drop the partial current week here — it's the chart's final bar. Exclude it from deltas in the bucketer instead: `bucket-trends.js --end=<TREND_CLASS_END> --include-partial-end` where `<TREND_CLASS_END> = startofweek(today)`. + ```kql materialized_view('ErrorStatsMetrics') -| where EventInfo_Time > ago(70d) +| where EventInfo_Time >= datetime(<TREND_START>) and EventInfo_Time < datetime(<TREND_END>) | where isnotempty(error_code) and error_code != 'success' | summarize errs = sum(countOverall), devs = dcount_hll(hll_merge(countDevicesHll)) @@ -158,22 +203,25 @@ materialized_view('ErrorStatsMetrics') | order by error_code asc, week asc ``` -### 8c. Spike attribution — one slicing dim at a time +### 8c. Spike attribution — rolling 7-day WoW form (one slicing dim at a time) -The MCP tool can return ~50–700 KB of JSON; multi-dim cartesians blow this out. **Slice one dimension per query**, then post-process with `summarize-attribution.js`: +The MCP tool can return ~50–700 KB of JSON; multi-dim cartesians blow this out. **Slice one dimension per query**, then post-process with `summarize-attribution.js`. The rolling-window form (see § 3): ```kql +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let codes = dynamic(['no_tokens_found','unauthorized_client','Code:-6', 'unknown_crypto_error','null_pointer_error','timed_out_execution']); materialized_view('ErrorStatsMetrics') | extend unified_account_type = MergeAccountType(account_type) | extend unified_is_shared_device = MergeIsSharedDevice(is_shared_device) -| where EventInfo_Time > ago(14d) +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd | where error_code in (codes) -| extend wk = startofweek(EventInfo_Time) +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize devs = dcount_hll(hll_merge(countDevicesHll)) - by wk, error_code, span_name // <-- swap this dim per query -| order by error_code asc, wk asc, devs desc + by week, error_code, span_name // <-- swap this dim per query +| order by error_code asc, week asc, devs desc ``` Run once each with the trailing dim set to: `span_name`, `calling_package_name`, `active_broker_package_name`, `broker_version`, `unified_account_type`, `unified_is_shared_device`, `client_sku`. That's the full 7. @@ -201,12 +249,12 @@ materialized_view('BrokerAdoptionStatsUpdated') | Script | Purpose | |---|---| -| [`bucket-trends.js`](bucket-trends.js) | Bucket every error code into regression / spike / improvement / flat across an N-week window. Pass `--end=YYYY-MM-DD` (Sunday after the reporting week, exclusive) to drop partial in-progress buckets. | +| [`bucket-trends.js`](bucket-trends.js) | Bucket every error code into regression / spike / improvement / flat across an N-week window. Pass `--end=YYYY-MM-DD` (= `startofweek(today)`, exclusive) to exclude the partial in-progress week from delta classification, plus `--include-partial-end` to still chart it as the final bar. | | [`agg.js`](agg.js) | Per-error per-dim top-N rollup with WoW deltas. Feeds spike-attribution dim blocks. | | [`summarize-attribution.js`](summarize-attribution.js) | Roll up 7-dim attribution slices per (error_code, week) — feeds the spike-attribution cards | | [`queries/`](queries/) | Canonical KQL templates, one per query — see [`queries/README.md`](queries/README.md) | | [`templates/`](templates/) | Copy-paste HTML snippets for cards / footer JS | -| [`report-template.html`](../templates/report-template.html) | Canonical layout. Copy to `~/android-oce-reports/oncall-wow-report-<sunday>.html` and replace `{{TOKENS}}` only — never restructure CSS | +| [`report-template.html`](../templates/report-template.html) | Canonical layout. Bootstrap copies it to `~/android-oce-reports/oncall-wow-report-<end-date>.html` and stamps the resolved window into the header. Replace prose/values only — never restructure CSS. | --- diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-codes.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-codes.kql index a9e03506..1e2c6270 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-codes.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-codes.kql @@ -1,15 +1,32 @@ -// 60-day per-error-code trend. -// Inputs (replace before pasting): -// <START> = first Sunday of the 60d window (e.g. 2026-03-08) -// <END> = end of the reporting week, EXCLUSIVE = next Sunday after the -// reporting week's Sunday (e.g. for a 2026-05-03 report, use 2026-05-10) -// Output: feed to assets/scripts/bucket-trends.js with --start=<START> (no --end needed -// because we filter the partial bucket out at the source — preferred). +// 60-day per-error-code trend (LITERAL last 60 days ending today). +// +// The 60-day trend spans the literal last 60 days ending at +// curEnd (today), so BOTH bounds move with the report date. The trend CHART +// includes the current in-progress week as its final (partial) bar. Per-week +// bucketing is still Sun-Sat aligned (Kusto startofweek() is Sunday-based), so the +// first and last buckets are partial by construction. +// +// IMPORTANT: do NOT drop the current in-progress week here -- the chart wants it. +// bucket-trends.js excludes the partial week from the regression/improvement DELTA +// math via `--end=<TREND_CLASS_END> --include-partial-end` (TREND_CLASS_END = +// startofweek(today)) while still charting it. A partial week driving the delta +// would read as a fake -99% improvement -- that's why classification and display +// are split in the JS, not filtered out here. +// +// Inputs (replace before pasting; bootstrap-report.ps1 prints all values): +// <TREND_START> = curEnd - 60d (first calendar day of the 60-day window) +// <TREND_END> = curEnd (today), EXCLUSIVE. Data is pulled up to but not +// including 00:00 UTC today, so the final Sun-Sat bucket holds +// Sunday..yesterday of the current week (the partial bar). +// +// Output: feed to assets/scripts/bucket-trends.js with +// --start=<TREND_START> --end=<TREND_CLASS_END> --include-partial-end +// where TREND_CLASS_END = startofweek(today) (bootstrap prints it as the +// "Trend delta cutoff" value). materialized_view('ErrorStatsMetrics') -| where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) +| where EventInfo_Time >= datetime(<TREND_START>) and EventInfo_Time < datetime(<TREND_END>) | where isnotempty(error_code) and error_code != 'success' | summarize errs = sum(countOverall), devs = dcount_hll(hll_merge(countDevicesHll)) by week = startofweek(EventInfo_Time), error_code -| where week < datetime(<END>) // drop partial end-week | order by error_code asc, week asc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-types.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-types.kql index 951e840f..7f37fd56 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-types.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-types.kql @@ -1,10 +1,18 @@ // 60-day per-error-type trend (with MergeUiRequiredExceptions to collapse variants). +// +// LITERAL last 60 days ending today; Sun-Sat weekly bucketing kept (Kusto +// startofweek() is Sunday-aligned). The current in-progress week is charted as the +// final partial bar and is excluded from delta classification by bucket-trends.js, +// not filtered here. See 60d-trend-codes.kql for full semantics. +// +// Inputs (bootstrap-report.ps1 prints all values): +// <TREND_START> = curEnd - 60d +// <TREND_END> = curEnd (today), EXCLUSIVE materialized_view('ErrorStatsMetrics') | extend unified_error_type = MergeUiRequiredExceptions(error_type) -| where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) +| where EventInfo_Time >= datetime(<TREND_START>) and EventInfo_Time < datetime(<TREND_END>) | where isnotempty(unified_error_type) | summarize errs = sum(countOverall), devs = dcount_hll(hll_merge(countDevicesHll)) by week = startofweek(EventInfo_Time), unified_error_type -| where week < datetime(<END>) | order by unified_error_type asc, week asc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/README.md b/.github/skills/oncall-weekly-telemetry-report/assets/queries/README.md index 12351dae..a48c5276 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/README.md +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/README.md @@ -1,34 +1,48 @@ # `assets/queries/` — canonical KQL templates Each `.kql` here is a paste-and-replace template for one of the queries the OCE -weekly report needs. Token convention: +report needs. The reporting window is a **rolling 7-day window** ending at +start-of-day UTC on `-EndDate` (default: today). See +[`../scripts/bootstrap-report.ps1`](../scripts/bootstrap-report.ps1) for the +canonical window computation, and SKILL.md § "Inputs to confirm" for the +rationale. + +## Placeholder convention | Token | Meaning | |---|---| -| `<START>` | Sunday of the earliest week in the window, ISO date e.g. `2026-03-08` | -| `<END>` | Sunday immediately AFTER the reporting week (EXCLUSIVE upper bound). For a 2026-05-03 report use `2026-05-10`. | -| `<PREV_WEEK>` | Sunday of the prior week (the WoW baseline). | +| `<CUR_END>` | Exclusive upper bound of the current 7-day window (e.g. `2026-07-09` on a run at any local time on 2026-07-09 UTC). | +| `<CUR_START>` | `<CUR_END> - 7d` (e.g. `2026-07-02`). Inclusive lower bound of the current window. | +| `<PREV_START>` | `<CUR_END> - 14d` (e.g. `2026-06-25`). Inclusive lower bound of the prior 7-day baseline window. The baseline window is `[PREV_START, CUR_START)`. | +| `<TREND_START>` | First calendar day of the 60-day trend chart window: `CUR_END - 60d` (literal 60 days ending today). | +| `<TREND_END>` | Exclusive upper bound of the 60-day trend chart window: `CUR_END` (today). The final Sun-Sat bucket is the current in-progress (partial) week — charted, but excluded from delta classification. | +| `<TREND_CLASS_END>` | Delta-classification cutoff = `startofweek(CUR_END)` (the Sunday that opens the current in-progress week). Passed to `bucket-trends.js` as `--end` (with `--include-partial-end`). On a `2026-07-09` (Thu) run, that's `2026-07-05`. Bootstrap prints it as "Trend delta cutoff". | +| `<SPARK_START>` | First Sunday of the WoW-table sparkline series: `startofweek(CUR_END) - 56d`. **Sunday-aligned; 8 complete weeks.** | +| `<SPARK_END>` | Sunday that OPENS the current in-progress week, exclusive: `startofweek(CUR_END)`. The `| where week < datetime(<SPARK_END>)` filter keeps the WoW-row sparklines on 8 complete weeks. | | `<CODES_LIST>` | Comma-separated KQL string list, e.g. `'invalid_resource', 'null_pointer_error'` | | `<TYPES_LIST>` | Same shape but for `unified_error_type`. | -| `<DIM>` | A single column name, replace per dimension run. | +| `<DIM>` | A single column name, replaced per dimension run. | + +**The primary/WoW queries emit two rows per key (bucket = `prevStart` or `curStart`).** The JS helpers (`agg.js`, `summarize-attribution.js`) sort the bucket label lexicographically and treat the smaller value as "prev" and the larger as "cur" — so any pair of sortable datetimes works. -**The `<END>` filter is mandatory.** Always include `| where week < datetime(<END>)` after the `summarize` so the partial in-progress week is dropped at the source. Otherwise `bucket-trends.js` will see a fake −99% improvement on every code (the partial bucket will look like a fleet-wide collapse). +**The 60-day trend queries emit Sun-Sat weekly buckets over the literal last 60 days ending today** (`[CUR_END - 60d, CUR_END)`). `startofweek()` is Sunday-aligned in Kusto. Do **not** filter the partial in-progress week at the source — the chart wants it as the final bar. Exclude it from the delta math via `bucket-trends.js --end=<TREND_CLASS_END> --include-partial-end` (`<TREND_CLASS_END>` = `startofweek(CUR_END)`); otherwise a partial "last" week reads as a fake −99% improvement on every code. The `wow-table-sparkline-series.kql` file is the exception: it keeps 8 complete weeks via `| where week < datetime(<SPARK_END>)` so no WoW row ends on a partial dip. ## File index | File | Purpose | Section it feeds | |---|---|---| -| [`reliability-auth-only.kql`](reliability-auth-only.kql) | Per-week auth-only requests/devices | Top-line health, denominator caveat | -| [`broker-version-share.kql`](broker-version-share.kql) | Per-week per-version share — **evidence for denominator caveat** | Denominator caveat callout, broker adoption | +| [`reliability-auth-only.kql`](reliability-auth-only.kql) | Auth-only requests/devices for the current + prior 7-day windows | Top-line health, denominator caveat | +| [`broker-version-share.kql`](broker-version-share.kql) | Per-version share for the WoW window — **evidence for denominator caveat** | Denominator caveat callout, broker adoption | | [`broker-version-share-wow.kql`](broker-version-share-wow.kql) | Single WoW snapshot of version share — fastest evidence for cohort transitions | Denominator caveat callout | -| [`60d-trend-codes.kql`](60d-trend-codes.kql) | Feeds `bucket-trends.js` for codes | 60-day trend analysis | -| [`60d-trend-types.kql`](60d-trend-types.kql) | Feeds `bucket-trends.js` for types | 60-day trend analysis | -| [`wow-movers.kql`](wow-movers.kql) | **MANDATORY second pass** — catches small-base codes that spiked sharply this week (below the 60d bucketer's reporting threshold). Run for both `error_code` and `error_type`. **Merge its output rows into the single 🔴 WoW regressions callout** alongside the standard WoW table; tag rows that were absent or near-zero last week with `NEW`. Do not render a separate "emerging" callout. | 🔴 WoW regressions callout (Section 2) | -| [`attr-union-by-dim.kql`](attr-union-by-dim.kql) | **PREFERRED for 2-week WoW.** All 7 dims for N codes (or types) in ONE round-trip; pipe through `summarize-attribution.js --union`. | Spike attribution cards | -| [`attr-codes-by-dim.kql`](attr-codes-by-dim.kql) | Per-dim form (run 7 times). Fall back to this only when the union exceeds payload size or the time window is wider than 2 weeks. | Spike attribution cards | +| [`60d-trend-codes.kql`](60d-trend-codes.kql) | Feeds `bucket-trends.js` for codes (Sun-Sat weekly buckets over the literal last 60 days ending today; final bar = partial current week) | 60-day trend analysis | +| [`60d-trend-types.kql`](60d-trend-types.kql) | Feeds `bucket-trends.js` for types (Sun-Sat weekly buckets over the literal last 60 days ending today; final bar = partial current week) | 60-day trend analysis | +| [`wow-movers.kql`](wow-movers.kql) | **MANDATORY second pass** — catches small-base codes that spiked sharply in the current window (below the 60d bucketer's reporting threshold). Run for both `error_code` and `error_type`. **Merge its output rows into the single 🔴 WoW regressions callout** alongside the standard WoW table; tag rows that were absent or near-zero in the prior window with `NEW`. Do not render a separate "emerging" callout. | 🔴 WoW regressions callout (Section 2) | +| [`attr-union-by-dim.kql`](attr-union-by-dim.kql) | **PREFERRED for WoW.** All 7 dims for N codes (or types) in ONE round-trip; pipe through `summarize-attribution.js --union`. | Spike attribution cards | +| [`attr-codes-by-dim.kql`](attr-codes-by-dim.kql) | Per-dim form (run 7 times). Fall back to this only when the union exceeds payload size. | Spike attribution cards | | [`attr-types-by-dim.kql`](attr-types-by-dim.kql) | Per-dim form for type regressions | Spike attribution cards | | [`type-subcode-decomposition.kql`](type-subcode-decomposition.kql) | 8th dim for type cards | Type spike-attribution cards | -| [`error-message-and-location.kql`](error-message-and-location.kql) | **MANDATORY** for every broker-tagged regression. Now accepts BOTH `<CODES_LIST>` and `<TYPES_LIST>` so codes + types can be sliced in one round-trip. | Code attribution block | +| [`error-message-and-location.kql`](error-message-and-location.kql) | **MANDATORY** for every broker-tagged regression. Accepts BOTH `<CODES_LIST>` and `<TYPES_LIST>` so codes + types can be sliced in one round-trip. | Code attribution block | | [`os-version-slice.kql`](os-version-slice.kql) | OS / OEM concentration (raw `android_spans`). **On-demand only** per Step 5 — don't slice every card. | OS-version dim in attribution cards (when applicable) | -| [`latency.kql`](latency.kql) | p50/p95/p99 by hot span | Latency section | -| [`app-share.kql`](app-share.kql) | Top calling apps by week | Traffic analysis | +| [`latency.kql`](latency.kql) | p50/p95/p99 by hot span for the WoW window | Latency section | +| [`app-share.kql`](app-share.kql) | Top calling apps for the WoW window | Traffic analysis | +| [`wow-table-sparkline-series.kql`](wow-table-sparkline-series.kql) | 8-week per-code/per-type sparkline series (Sun-Sat weekly buckets, `<SPARK_START>`/`<SPARK_END>` tokens — 8 complete weeks) for the WoW table `data-trend` arrays. **MANDATORY** — sparkline arrays must come from real data, never fabricated. | Section 6/7 tables | diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/app-share.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/app-share.kql index 4e138833..c1ea6243 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/app-share.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/app-share.kql @@ -1,10 +1,16 @@ -// Top calling apps share for last N weeks (typically 3). +// Top calling apps share for the WoW window. +// +// Rolling 7-day window. See SKILL.md and bootstrap-report.ps1 for the canonical +// window computation. Two rows per app (week = prevStart or curStart). +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); materialized_view('AppStatsUpdated') -| where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize req = sum(countRequests), dev = dcount_hll(hll_merge(countDevicesHll)) - by week = startofweek(EventInfo_Time), calling_package_name -| where week < datetime(<END>) + by week, calling_package_name | order by week asc, req desc | summarize topApps = make_list(pack('app', calling_package_name, 'req', req, 'dev', dev), 25) by week diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-codes-by-dim.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-codes-by-dim.kql index fa31ff56..cbff0630 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-codes-by-dim.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-codes-by-dim.kql @@ -2,16 +2,21 @@ // Run 7 times with <DIM> set to each of: // span_name | calling_package_name | active_broker_package_name | // broker_version | unified_account_type | unified_is_shared_device | client_sku -// (Plus android_spans-based for OS version — see os-version-slice.kql.) +// (Plus android_spans-based for OS version -- see os-version-slice.kql.) +// +// Rolling 7-day WoW window. See SKILL.md and bootstrap-report.ps1 for the +// canonical window computation. Two week buckets per (code, dim value). +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let codes = dynamic([<CODES_LIST>]); materialized_view('ErrorStatsMetrics') | extend unified_account_type = MergeAccountType(account_type) | extend unified_is_shared_device = MergeIsSharedDevice(is_shared_device) -| where EventInfo_Time between (datetime(<PREV_WEEK>) .. datetime(<END>)) +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd | where error_code in (codes) -| extend wk = startofweek(EventInfo_Time) -| where wk < datetime(<END>) +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by wk, error_code, <DIM> -| order by error_code asc, wk asc, devs desc + by week, error_code, <DIM> +| order by error_code asc, week asc, devs desc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-types-by-dim.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-types-by-dim.kql index d9b06736..897cb32b 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-types-by-dim.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-types-by-dim.kql @@ -1,15 +1,19 @@ // Spike attribution: types by ONE dimension at a time. // Same usage as attr-codes-by-dim.kql but for error_type regressions. +// +// Rolling 7-day WoW window. See SKILL.md and bootstrap-report.ps1. +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let types = dynamic([<TYPES_LIST>]); materialized_view('ErrorStatsMetrics') | extend unified_error_type = MergeUiRequiredExceptions(error_type) | extend unified_account_type = MergeAccountType(account_type) | extend unified_is_shared_device = MergeIsSharedDevice(is_shared_device) -| where EventInfo_Time between (datetime(<PREV_WEEK>) .. datetime(<END>)) +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd | where unified_error_type in (types) -| extend wk = startofweek(EventInfo_Time) -| where wk < datetime(<END>) +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by wk, unified_error_type, <DIM> -| order by unified_error_type asc, wk asc, devs desc + by week, unified_error_type, <DIM> +| order by unified_error_type asc, week asc, devs desc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-union-by-dim.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-union-by-dim.kql index 00afd33f..02be1d66 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-union-by-dim.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-union-by-dim.kql @@ -1,64 +1,70 @@ -// Spike-attribution union — all 7 dims for N codes (or N types) in ONE query. +// Spike-attribution union -- all 7 dims for N codes (or N types) in ONE query. // -// Recommended for the standard 2-week WoW attribution pass (Step 5 of SKILL.md). -// 1 round-trip vs 7. ~800 KB payload for 8 codes; well under the MCP limit. +// Recommended for the standard WoW attribution pass (SKILL.md Step 5). 1 +// round-trip vs 7. ~800 KB payload for 8 codes; well under the MCP limit. // Falls back to per-dim files (assets/queries/attr-codes-by-dim.kql) if you // need a wider time window or you exceed payload size. // +// Rolling 7-day WoW window. See SKILL.md and bootstrap-report.ps1 for the +// canonical window computation. +// // Inputs: -// <CODES> e.g. dynamic(['no_tokens_found','timed_out_execution', ...]) -// <START> inclusive (e.g. datetime(2026-04-26)) -// <END> EXCLUSIVE Sunday after the reporting week (e.g. datetime(2026-05-10)) -// <KEY> either `error_code` or `unified_error_type` (the latter for type cards) +// <CODES> e.g. dynamic(['no_tokens_found','timed_out_execution', ...]) +// <CUR_END> e.g. 2026-07-09 (exclusive) +// <CUR_START> curEnd - 7d, e.g. 2026-07-02 +// <PREV_START> curEnd - 14d, e.g. 2026-06-25 +// <KEY> either `error_code` or `unified_error_type` (the latter for type cards) // // Output schema (consumed by `summarize-attribution.js --union`): // dim string short label per dimension -// wk datetime reporting week +// week datetime bucket start (prevStart or curStart) // <KEY> string error_code or unified_error_type // val_string string dim value (cast via tostring() in every union leg) // devs long dcount_hll merged device count // errs long sum of countOverall (request count) // -// For type cards, swap the first line and key: +// For type cards, swap the first `base = ...` line and the key column: // let base = materialized_view('ErrorStatsMetrics') // | extend unified_error_type = MergeUiRequiredExceptions(error_type) -// | where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) +// | where EventInfo_Time >= prevStart and EventInfo_Time < curEnd // | where unified_error_type in (<TYPES>) -// | extend wk = startofweek(EventInfo_Time); - -// IMPORTANT — column-aliasing gotcha: every union branch MUST emit `val_string` +// | extend week = iff(EventInfo_Time >= curStart, curStart, prevStart); +// +// IMPORTANT -- column-aliasing gotcha: every union branch MUST emit `val_string` // as a real `string` (never `bool(null)`), or Kusto will rename the columns // `val_string_string` and `val_string_bool` in the result schema, which then // breaks `summarize-attribution.js` (it now accepts both names as a fallback, // but emitting one consistent `string` column is cleaner). Use `tostring()` on // non-string dims (e.g. shared_dev) so every leg has a string-typed column. +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let codes = dynamic([<CODES>]); let base = materialized_view('ErrorStatsMetrics') - | where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) + | where EventInfo_Time >= prevStart and EventInfo_Time < curEnd | where error_code in (codes) - | extend wk = startofweek(EventInfo_Time); + | extend week = iff(EventInfo_Time >= curStart, curStart, prevStart); (base | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by dim='span', wk, error_code, val_string=tostring(span_name)) + by dim='span', week, error_code, val_string=tostring(span_name)) | union (base | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by dim='calling_app', wk, error_code, val_string=tostring(calling_package_name)) + by dim='calling_app', week, error_code, val_string=tostring(calling_package_name)) | union (base | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by dim='active_broker', wk, error_code, val_string=tostring(active_broker_package_name)) + by dim='active_broker', week, error_code, val_string=tostring(active_broker_package_name)) | union (base | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by dim='broker_ver', wk, error_code, val_string=tostring(broker_version)) + by dim='broker_ver', week, error_code, val_string=tostring(broker_version)) | union (base | extend t = MergeAccountType(account_type) | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by dim='acct_type', wk, error_code, val_string=tostring(t)) + by dim='acct_type', week, error_code, val_string=tostring(t)) | union (base | extend s = MergeIsSharedDevice(is_shared_device) | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by dim='shared_dev', wk, error_code, val_string=tostring(s)) + by dim='shared_dev', week, error_code, val_string=tostring(s)) | union (base | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by dim='client_sku', wk, error_code, val_string=tostring(client_sku)) -| where wk < datetime(<END>) -| order by error_code asc, dim asc, wk asc, devs desc + by dim='client_sku', week, error_code, val_string=tostring(client_sku)) +| order by error_code asc, dim asc, week asc, devs desc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share-wow.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share-wow.kql index 0fe05c58..46dd009e 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share-wow.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share-wow.kql @@ -1,24 +1,30 @@ -// WoW broker-version share comparison \u2014 the canonical evidence for the +// WoW broker-version share comparison -- the canonical evidence for the // "denominator caveat" callout when an entire version cohort retires. Use this // instead of the time-series `broker-version-share.kql` when you need a single // WoW snapshot showing which versions gained/lost share. Modeled on // `wow-movers.kql`. // +// Rolling 7-day window. See SKILL.md and bootstrap-report.ps1 for the canonical +// window computation. +// // Inputs: -// <CURR_START> Sunday of the reporting week (e.g. 2026-05-03) -// <CURR_END> Sunday after (exclusive, e.g. 2026-05-10) -// <PRIOR_START> Sunday of the baseline week (e.g. 2026-04-26) +// <CUR_END> end of the current window, EXCLUSIVE (e.g. 2026-07-09) +// <CUR_START> curEnd - 7d (e.g. 2026-07-02) +// <PREV_START> curEnd - 14d (e.g. 2026-06-25) // -// Floor: only versions with >100M reqs in either week (filters long-tail). -// Output sorted by current-week req count descending. +// Floor: only versions with >100M reqs in either window (filters long-tail). +// Output sorted by current-window req count descending. +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let curr = materialized_view('BrokerAdoptionStatsUpdated') - | where EventInfo_Time between (datetime(<CURR_START>) .. datetime(<CURR_END>)) + | where EventInfo_Time >= curStart and EventInfo_Time < curEnd | summarize cReq = sum(countRequests), cDev = dcount_hll(hll_merge(countDevicesHll)) by broker_version; let prior = materialized_view('BrokerAdoptionStatsUpdated') - | where EventInfo_Time between (datetime(<PRIOR_START>) .. datetime(<CURR_START>)) + | where EventInfo_Time >= prevStart and EventInfo_Time < curStart | summarize pReq = sum(countRequests), pDev = dcount_hll(hll_merge(countDevicesHll)) by broker_version; diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share.kql index bdfc5a41..7b8f425c 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share.kql @@ -1,10 +1,17 @@ -// Per-broker-version request and device share — the canonical evidence for -// the "denominator caveat" callout. If the all-spans device count moved >20% -// WoW, this query tells you WHICH version cohort drove it. +// Per-broker-version request and device share for the WoW window -- the +// canonical evidence for the "denominator caveat" callout. If the all-spans +// device count moved >20% WoW, this query tells you WHICH version cohort drove +// it. +// +// Rolling 7-day window. See SKILL.md and bootstrap-report.ps1 for the canonical +// window computation. Two rows per broker_version (week = prevStart or curStart). +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); materialized_view('BrokerAdoptionStatsUpdated') -| where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize req = sum(countRequests), dev = dcount_hll(hll_merge(countDevicesHll)) - by week = startofweek(EventInfo_Time), broker_version -| where week < datetime(<END>) + by week, broker_version | order by week asc, req desc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/error-message-and-location.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/error-message-and-location.kql index 0272daaa..bd1c0500 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/error-message-and-location.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/error-message-and-location.kql @@ -11,30 +11,34 @@ // THIS TEMPLATE COVERS BOTH error_code AND error_type IN ONE ROUND-TRIP. // Pass an empty list for the side you don't want to slice. // +// Rolling 7-day window. See SKILL.md and bootstrap-report.ps1. +// // Inputs: // <CODES_LIST> e.g. 'invalid_resource', 'null_pointer_error' (or empty) // <TYPES_LIST> e.g. 'IntuneAppProtectionPolicyRequiredException' (or empty) -// <START> datetime — should be the reporting-week Sunday (e.g. 2026-05-03). -// Use the FULL 7-day reporting window, NOT a narrower 3-5 day slice -// (low-volume types like SSLHandshakeException / Intune* may return -// zero rows in a sub-week window). -// <END> datetime of next Sunday (exclusive) +// <CUR_START> datetime -- start of the current 7-day window (e.g. 2026-07-02) +// Use the FULL 7-day window, NOT a narrower 3-5 day slice +// (low-volume types like SSLHandshakeException / Intune* may +// return zero rows in a sub-window slice). +// <CUR_END> datetime -- exclusive upper bound (e.g. 2026-07-09) // -// Tip: if the reporting window returns no rows for a low-volume code/type, fall -// back to the prior 14-day window (`<START> - 7d .. <END>`) before giving up. +// Tip: if the current 7-day window returns no rows for a low-volume code/type, +// fall back to the prior 14 days (<PREV_START> .. <CUR_END>) before giving up. // // Output column 'loc' is a JSON blob {"ClassName":"...","MethodName":"...","LineNumber":N} -// — this is normal. Read it as text. To project the method name only, use +// -- this is normal. Read it as text. To project the method name only, use // parse_json(loc).MethodName // // HARD RULE (per SKILL.md Step 4): if the throw site is in // ExceptionAdapter.{getExceptionFromTokenErrorResponse, exceptionFromAuthorizationResult} // AND the message starts with "AADSTS", the originator is eSTS, not broker. +let curStart = datetime(<CUR_START>); +let curEnd = datetime(<CUR_END>); let codes = dynamic([<CODES_LIST>]); let types = dynamic([<TYPES_LIST>]); android_spans -| where PipelineInfo_IngestionTime between (datetime(<START>) .. datetime(<END>)) +| where PipelineInfo_IngestionTime >= curStart and PipelineInfo_IngestionTime < curEnd | where (array_length(codes) > 0 and error_code in (codes)) or (array_length(types) > 0 and error_type in (types)) | extend loc = tostring(error_location), diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/latency.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/latency.kql index 471fd2ca..e60f053d 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/latency.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/latency.kql @@ -1,13 +1,19 @@ // p50 / p95 / p99 latency on the hot spans. Always merge TDigest before percentile. +// +// Rolling 7-day WoW window. See SKILL.md and bootstrap-report.ps1 for the +// canonical window computation. Two week buckets per span. +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); materialized_view('PerfStatsUpdated') -| where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd | where span_name in ('AcquireTokenSilent','AcquireTokenInteractive', 'GetAccounts','RemoveAccount','ProcessWebsiteRequest') | where span_status == 'OK' +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize p50 = percentile_tdigest(tdigest_merge(responseTimeTDigest), 50, typeof(long)), p95 = percentile_tdigest(tdigest_merge(responseTimeTDigest), 95, typeof(long)), p99 = percentile_tdigest(tdigest_merge(responseTimeTDigest), 99, typeof(long)), reqs = sum(countRequests) - by week = startofweek(EventInfo_Time), span_name -| where week < datetime(<END>) + by week, span_name | order by span_name asc, week asc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/os-version-slice.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/os-version-slice.kql index dcb93df6..e8bb6581 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/os-version-slice.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/os-version-slice.kql @@ -1,8 +1,17 @@ // OS version slice (8th attribution dim). Requires raw android_spans because // ErrorStatsMetrics doesn't carry DeviceInfo_OsVersion. Keep the time window // tight (<= 7 days) to stay under the MCP 240s timeout. +// +// Rolling 7-day window. See SKILL.md and bootstrap-report.ps1. +// +// Inputs: +// <CUR_START> e.g. 2026-07-02 +// <CUR_END> exclusive upper bound, e.g. 2026-07-09 +// <CODES_LIST> e.g. 'invalid_resource', 'null_pointer_error' +let curStart = datetime(<CUR_START>); +let curEnd = datetime(<CUR_END>); android_spans -| where PipelineInfo_IngestionTime between (datetime(<START>) .. datetime(<END>)) +| where PipelineInfo_IngestionTime >= curStart and PipelineInfo_IngestionTime < curEnd | where error_code in (<CODES_LIST>) | summarize devs = dcount(DeviceInfo_Id), cnt = count() diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/reliability-auth-only.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/reliability-auth-only.kql index e9758b21..e90300c3 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/reliability-auth-only.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/reliability-auth-only.kql @@ -1,14 +1,29 @@ -// Auth-only denominator and reliability per week. +// Auth-only denominator and reliability for the WoW window. +// +// Rolling 7-day window. curEnd is EXCLUSIVE (00:00 UTC of the invocation day); +// curStart = curEnd - 7d, prevStart = curEnd - 14d. See SKILL.md and +// assets/scripts/bootstrap-report.ps1 for the canonical window computation. +// +// Inputs: +// <CUR_END> e.g. 2026-07-09 (exclusive) +// <CUR_START> curEnd - 7d, e.g. 2026-07-02 +// <PREV_START> curEnd - 14d, e.g. 2026-06-25 +// +// Output shape: two rows per bucket, week in { prevStart, curStart }. +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let s = materialized_view('SilentAuthStatsAllRequestsMetrics') - | where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) + | where EventInfo_Time >= prevStart and EventInfo_Time < curEnd + | extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize req = sum(countRequests), devHll = hll_merge(countDevicesHll) - by week = startofweek(EventInfo_Time); + by week; let i = materialized_view('InteractiveAuthStatsAllRequestsMetrics') - | where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) + | where EventInfo_Time >= prevStart and EventInfo_Time < curEnd + | extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize req = sum(countRequests), devHll = hll_merge(countDevicesHll) - by week = startofweek(EventInfo_Time); + by week; union s, i | summarize authReq = sum(req), authDev = dcount_hll(hll_merge(devHll)) by week -| where week < datetime(<END>) | order by week asc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/type-subcode-decomposition.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/type-subcode-decomposition.kql index 4454c79f..a9c17743 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/type-subcode-decomposition.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/type-subcode-decomposition.kql @@ -1,13 +1,17 @@ // Sub-code decomposition for an error_type regression card (the "8th dim"). // Shows top error_codes that roll up under each unified_error_type, with WoW devices. +// +// Rolling 7-day WoW window. See SKILL.md and bootstrap-report.ps1. +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let types = dynamic([<TYPES_LIST>]); materialized_view('ErrorStatsMetrics') | extend unified_error_type = MergeUiRequiredExceptions(error_type) -| where EventInfo_Time between (datetime(<PREV_WEEK>) .. datetime(<END>)) +| where EventInfo_Time >= prevStart and EventInfo_Time < curEnd | where unified_error_type in (types) -| extend wk = startofweek(EventInfo_Time) -| where wk < datetime(<END>) +| extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) - by wk, unified_error_type, error_code -| order by unified_error_type asc, wk asc, devs desc + by week, unified_error_type, error_code +| order by unified_error_type asc, week asc, devs desc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-movers.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-movers.kql index 1a53a8f9..4908526f 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-movers.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-movers.kql @@ -1,4 +1,4 @@ -// WoW movers — codes (or types) that moved sharply this week regardless of 60d shape. +// WoW movers -- codes (or types) that moved sharply this window regardless of 60d shape. // // MANDATORY pass alongside `60d-trend-codes.kql` / `60d-trend-types.kql` (per // SKILL.md Step 3b). The 60d bucketer's --peak-floor=10000 EXCLUDES errors @@ -7,10 +7,13 @@ // weeks, or `Code:-11` 937 -> 2,490 devs WoW). Without this pass those spikes // are silently dropped from the report. // +// Rolling 7-day window. See SKILL.md and bootstrap-report.ps1 for the canonical +// window computation. +// // Inputs: -// <CURR_START> Sunday of the reporting week (e.g. 2026-05-03) -// <CURR_END> Sunday after (exclusive, e.g. 2026-05-10) -// <PRIOR_START> Sunday of the baseline week (e.g. 2026-04-26) +// <CUR_END> end of the current window, EXCLUSIVE (e.g. 2026-07-09) +// <CUR_START> curEnd - 7d (e.g. 2026-07-02) +// <PREV_START> curEnd - 14d (e.g. 2026-06-25) // // To run for error_type instead of error_code, copy this query and replace: // - error_code -> MergeUiRequiredExceptions(error_type) (alias as `t`) @@ -19,16 +22,19 @@ // Thresholds (tuneable): // floor: cDev>=500 OR cReq>=5000 (small enough to catch sub-bucketer-floor codes) // move: |dDev%|>=25 OR |dReq%|>=50 (real spike, not noise) -// new-this-wk: pDev==0 OR pReq==0 (never seen before this week) +// new-this-week: pDev==0 OR pReq==0 (never seen before this window) +let curEnd = datetime(<CUR_END>); +let curStart = datetime(<CUR_START>); +let prevStart = datetime(<PREV_START>); let curr = materialized_view('ErrorStatsMetrics') - | where EventInfo_Time between (datetime(<CURR_START>) .. datetime(<CURR_END>)) + | where EventInfo_Time >= curStart and EventInfo_Time < curEnd | where isnotempty(error_code) and error_code != 'success' | summarize cDev = dcount_hll(hll_merge(countDevicesHll)), cReq = sum(countOverall) by error_code; let prior = materialized_view('ErrorStatsMetrics') - | where EventInfo_Time between (datetime(<PRIOR_START>) .. datetime(<CURR_START>)) + | where EventInfo_Time >= prevStart and EventInfo_Time < curStart | where isnotempty(error_code) and error_code != 'success' | summarize pDev = dcount_hll(hll_merge(countDevicesHll)), pReq = sum(countOverall) diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-table-sparkline-series.kql b/.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-table-sparkline-series.kql index 7ef5ce46..9b561575 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-table-sparkline-series.kql +++ b/.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-table-sparkline-series.kql @@ -2,33 +2,47 @@ // // MANDATORY (per SKILL.md Output checklist, v8): the `data-trend` arrays in // the Section 6 (error_code) and Section 7 (error_type) WoW tables must come -// from real data — not be fabricated from a "roughly increasing" pattern. +// from real data -- not be fabricated from a "roughly increasing" pattern. // Past failure mode: small-volume codes (Broker request cancelled, // kdfv2_key_derivation_error, TimeoutCancellationException) were filtered out // by the 60d bucketer's peak-floor, then their sparklines were invented inline // in the WoW table HTML. That's data dishonesty even when the array looks plausible. // // This query returns 8 weekly buckets for every code/type that appears in -// either the WoW movers list OR the 60d trend output. Run it twice — once with -// the codes filter, once with the types filter — and feed the result into the +// either the WoW movers list OR the 60d trend output. Run it twice -- once with +// the codes filter, once with the types filter -- and feed the result into the // WoW-table generator so every row has a real-data trend. // -// Inputs: -// <START> Sunday of week-0 (e.g. 2026-04-12 for an 8-week window ending 2026-06-06) -// <END> Sunday after the reporting week, EXCLUSIVE (e.g. 2026-06-07) -// <CODES> Dynamic list of error_code values whose sparklines we need. -// Build this from the union of: -// * wow-movers-codes.json results -// * 60d-codes regression/spike/improvement bucket members -// For the type variant, swap to `unified_error_type in (<TYPES>)` -// and the MergeUiRequiredExceptions extension. +// NOTE ON WEEK ALIGNMENT: even though the primary WoW section uses a rolling +// 7-day window AND the big 60-day trend chart now ends today (with a partial +// final bar), these per-ROW sparklines deliberately keep 8 COMPLETE Sun-Sat +// weeks. A partial final point in every WoW row would render as a misleading +// dip across the whole table; the decorative per-row trajectory reads cleaner +// on complete weeks. (The big 60d chart can afford one honest partial bar; 30+ +// table rows each ending on a partial point cannot.) +// +// Inputs (distinct from the trend-codes/types <TREND_*> tokens, which now mean +// the LITERAL last 60 days ending today -- these SPARK_* tokens still mean the +// last 8 COMPLETE weeks): +// <SPARK_START> Sunday of week-0 = startofweek(today) - 56d +// (e.g. 2026-05-10 for an 8-week window ending 2026-07-05) +// <SPARK_END> Sunday that OPENS the current in-progress week, EXCLUSIVE. +// Compute as startofweek(today). For a report run on 2026-07-09 +// (a Thu), use 2026-07-05. The `where week < datetime(<SPARK_END>)` +// drops the partial in-progress bucket at the source. +// <CODES> Dynamic list of error_code values whose sparklines we need. +// Build this from the union of: +// * wow-movers-codes.json results +// * 60d-codes regression/spike/improvement bucket members +// For the type variant, swap to `unified_error_type in (<TYPES>)` +// and the MergeUiRequiredExceptions extension. let codes = dynamic([<CODES>]); materialized_view('ErrorStatsMetrics') -| where EventInfo_Time between (datetime(<START>) .. datetime(<END>)) +| where EventInfo_Time between (datetime(<SPARK_START>) .. datetime(<SPARK_END>)) | where error_code in (codes) | summarize devs = dcount_hll(hll_merge(countDevicesHll)), errs = sum(countOverall) by week = startofweek(EventInfo_Time), error_code -| where week < datetime(<END>) +| where week < datetime(<SPARK_END>) | order by error_code asc, week asc diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/agg.js b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/agg.js index 349e8d2d..2e7c2864 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/agg.js +++ b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/agg.js @@ -15,16 +15,23 @@ * * Input shape: a Kusto MCP JSON file produced by: * + * let curEnd = datetime(<CUR_END>); + * let curStart = datetime(<CUR_START>); + * let prevStart = datetime(<PREV_START>); * let codes = dynamic([...]); * materialized_view('ErrorStatsMetrics') - * | where EventInfo_Time between (datetime(<prev_week>) .. datetime(<this_week_end>)) + * | where EventInfo_Time >= prevStart and EventInfo_Time < curEnd * | where error_code in (codes) // or unified_error_type in (types) - * | extend wk = startofweek(EventInfo_Time) - * | where wk < datetime(<reporting_week_end_sunday>) // drop partial end! + * | extend week = iff(EventInfo_Time >= curStart, curStart, prevStart) * | summarize devs = dcount_hll(hll_merge(countDevicesHll)), * errs = sum(countOverall) - * by wk, error_code, <ONE_DIMENSION> - * | order by error_code asc, wk asc, devs desc + * by week, error_code, <ONE_DIMENSION> + * | order by error_code asc, week asc, devs desc + * + * (`week` is now the START datetime of the 7-day window -- one of two values: + * prevStart or curStart. The script sorts wks lexicographically and treats + * the smallest as "prev" and the largest as "cur", so any pair of sortable + * bucket labels works.) * * Usage: * node agg.js <input.json> <error_key> <dim_col> [<dim_col2> ...] [--top=N] [--metric=devs|reqs] @@ -67,7 +74,7 @@ function pct(a, b) { } const { items, schema } = load(file); -const wkIdx = schema.indexOf('wk'); +const wkIdx = schema.indexOf('week'); const errIdx = schema.indexOf(errKey); const valIdx = schema.indexOf(metric === 'devs' ? 'devs' : 'errs'); const dimIdxs = dimCols.map(c => { @@ -79,20 +86,20 @@ const dimIdxs = dimCols.map(c => { return i; }); if (wkIdx < 0 || errIdx < 0 || valIdx < 0) { - console.error(`Required columns missing. schema=${schema.join(', ')} need wk, ${errKey}, ${metric === 'devs' ? 'devs' : 'errs'}`); + console.error(`Required columns missing. schema=${schema.join(', ')} need week, ${errKey}, ${metric === 'devs' ? 'devs' : 'errs'}`); process.exit(2); } -// group: err -> dimkey -> wk -> value +// group: err -> dimkey -> week -> value const m = {}; const wks = new Set(); for (const r of items) { - const wk = r[wkIdx], err = r[errIdx], val = r[valIdx]; + const week = r[wkIdx], err = r[errIdx], val = r[valIdx]; const dimKey = dimIdxs.map(i => (r[i] === null || r[i] === undefined || r[i] === '') ? '(blank)' : r[i]).join(' | '); - wks.add(wk); + wks.add(week); m[err] = m[err] || {}; m[err][dimKey] = m[err][dimKey] || {}; - m[err][dimKey][wk] = (m[err][dimKey][wk] || 0) + val; + m[err][dimKey][week] = (m[err][dimKey][week] || 0) + val; } const sortedWks = [...wks].sort(); if (sortedWks.length < 2) { diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/bootstrap-report.ps1 b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/bootstrap-report.ps1 index 703aa653..4d3a8fcb 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/bootstrap-report.ps1 +++ b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/bootstrap-report.ps1 @@ -4,57 +4,85 @@ .DESCRIPTION Implements SKILL.md Step 1 as a script so the workflow doesn't drift across - runs: - 1. Computes the reporting-week Sunday from the current date (most recent - complete Sun-Sat week unless -ReportingSunday is passed explicitly). - 2. Creates ~/android-oce-reports/_data/<sunday>/ for raw query payloads. - 3. Decides what to do if the target report file already exists: - - If the existing file is an UNFILLED template stub (header dates - still match the canonical template's reference week), silently - re-bootstrap from the template — there's nothing to preserve. - - If the existing file contains real per-week content (the dates - inside differ from the template's reference week), HALT and - require the caller to explicitly delete or rename the file first. - This is the "filename collision rule" from SKILL.md. - 4. Prunes _data/<sunday>/ folders older than -DataRetentionDays (default 60) - so the directory doesn't accumulate stale payloads indefinitely. - -.PARAMETER ReportingSunday - Sunday of the reporting week (yyyy-MM-dd). If omitted, defaults to the most - recent complete Sun-Sat week relative to the system clock. + runs. The reporting window is a ROLLING 7-DAY window ending at start-of-day + (UTC) on -EndDate (defaults to today): + + curStart = EndDate - 7d (inclusive) + curEnd = EndDate (exclusive, == 00:00 UTC of the end date) + prevStart = EndDate - 14d + prevEnd = curStart + + Rationale: the prior + implementation aligned the reporting window on Sun-Sat calendar weeks and + defaulted to "the Sunday of the currently in-progress week". Run on a + Thursday it emitted a 4-day partial window and dropped the last complete + week -- the exact "missed the last 6 days" the developer reported. The + rolling-window model always covers the 7 fully-elapsed days immediately + before the invocation with no user prompting. + + The script also: + 1. Creates ~/android-oce-reports/_data/<end-date>/ for raw query payloads. + 2. Copies the canonical template into + ~/android-oce-reports/oncall-wow-report-<end-date>.html. + 3. Stamps the resolved window into the <title>, the <div class="meta"> + block, and the "Generated <strong>...</strong>" banner so the header + can never drift from what was actually queried (the resolved window + is echoed in the report header for transparency). + 4. Decides what to do if the target report file already exists: + - If the existing file is an UNFILLED template stub (multiple + fingerprint markers still match the canonical template), silently + re-bootstrap -- nothing to preserve. + - Otherwise HALT and require -Force. + 5. Prunes _data/<old-end-date>/ folders older than -DataRetentionDays + (default 60). + +.PARAMETER EndDate + End of the reporting window (yyyy-MM-dd, UTC). Exclusive upper bound: data + is queried up to but not including 00:00 UTC on this date. Defaults to + today (UTC). Example: -EndDate 2026-07-09 on a run at any local time on + 2026-07-09 UTC produces the same window [2026-07-02 00:00, 2026-07-09 00:00). .PARAMETER Force Skip the collision check and overwrite any existing file. .PARAMETER DataRetentionDays - How many days of _data/<sunday>/ folders to keep before pruning. Default 60. + How many days of _data/<end-date>/ folders to keep before pruning. Default 60. .PARAMETER SkillRoot Path to the skill folder. Defaults to the location of this script's parent. .EXAMPLE .\bootstrap-report.ps1 - # Default: latest complete week, halt on collision + # Default: rolling 7 days ending today (UTC), halt on collision. .EXAMPLE - .\bootstrap-report.ps1 -ReportingSunday 2026-05-31 -Force + .\bootstrap-report.ps1 -EndDate 2026-07-09 -Force + # Reproduce the report for a specific window end. .OUTPUTS Prints the absolute path of the newly created report file. + +.NOTES + Breaking change vs pre-fix versions: + * -ReportingSunday parameter has been REMOVED; use -EndDate instead. + (The old name encoded the exact semantic mismatch that caused the bug.) + * Filename is now oncall-wow-report-<end-date>.html (was <sunday>.html). #> [CmdletBinding()] param( - [string]$ReportingSunday, + [string]$EndDate, [switch]$Force, [int]$DataRetentionDays = 60, [string]$SkillRoot ) $ErrorActionPreference = 'Stop' +# --------------------------------------------------------------------------- # Locate the skill folder + canonical template +# --------------------------------------------------------------------------- if (-not $SkillRoot) { - # This script lives at <skill>/assets/scripts/bootstrap-report.ps1, so go up 2 levels - # to reach <skill>/assets/. Templates live at <skill>/assets/templates/. + # This script lives at <skill>/assets/scripts/bootstrap-report.ps1, so go up + # 2 levels to reach <skill>/assets/. Templates live at <skill>/assets/templates/. $SkillRoot = Split-Path -Parent (Split-Path -Parent $PSCommandPath) } $template = Join-Path $SkillRoot 'templates\report-template.html' @@ -62,115 +90,218 @@ if (-not (Test-Path $template)) { throw "Canonical template not found at $template. Pass -SkillRoot if running outside the skill folder." } -# Compute the reporting Sunday -if (-not $ReportingSunday) { - $today = [datetime]::Today - # Most recent Sunday strictly before today, OR today if today is Sunday - $offset = ($today.DayOfWeek.value__ + 7) % 7 # 0..6 days back to the previous Sunday - $sunday = $today.AddDays(-$offset) - # If today is Sunday but it's still early in the day, prefer the prior complete week - if ($today.DayOfWeek -eq [DayOfWeek]::Sunday -and (Get-Date).Hour -lt 6) { - $sunday = $sunday.AddDays(-7) - } - $ReportingSunday = $sunday.ToString('yyyy-MM-dd') +# --------------------------------------------------------------------------- +# Resolve the rolling window +# --------------------------------------------------------------------------- +if (-not $EndDate) { + # UTC "today" so the window boundaries are stable across time zones and + # match Kusto's default datetime semantics (EventInfo_Time is UTC). + $EndDate = [datetime]::UtcNow.Date.ToString('yyyy-MM-dd') +} +try { + $curEnd = [datetime]::ParseExact($EndDate, 'yyyy-MM-dd', + [System.Globalization.CultureInfo]::InvariantCulture, + [System.Globalization.DateTimeStyles]::AssumeUniversal -bor + [System.Globalization.DateTimeStyles]::AdjustToUniversal) +} catch { + throw "Invalid -EndDate '$EndDate'. Use yyyy-MM-dd." } -[void][datetime]::Parse($ReportingSunday) # validate format +$curStart = $curEnd.AddDays(-7) +$prevStart = $curEnd.AddDays(-14) +$prevEnd = $curStart +# 60-day trend spans the LITERAL last 60 days ending curEnd (today), so BOTH +# bounds move with -EndDate (the trend ends today rather than 3-6 days ago on +# the last complete Sunday). The trend CHART shows the +# current in-progress week as its final (partial) bar, but bucket-trends.js still +# computes regression/improvement DELTAS on complete Sun-Sat weeks only -- a +# partial week as "last" would read as a fake -99% improvement. So we resolve two +# distinct things: +# * data/chart window [sixtyDayStart, sixtyDayEnd) = [curEnd-60d, curEnd) +# -> queried and charted (final bucket is the partial week) +# * classification cutoff trendClassEnd = startofweek(curEnd) +# -> passed to bucket-trends.js as --end so the partial +# current week is excluded from the delta math while +# still being drawn. +# Per-week bucketing stays Sun-Sat aligned (Kusto startofweek() is Sunday-based), +# so the first and last buckets are partial by construction. +$curEndDow = [int]$curEnd.DayOfWeek # Sun=0 .. Sat=6 +$sixtyDayStart = $curEnd.AddDays(-60) # literal 60 days ending today +$sixtyDayEnd = $curEnd # exclusive upper bound == today; chart includes the partial current week +$trendClassEnd = $curEnd.AddDays(-$curEndDow) # startofweek(curEnd): weeks >= this are the in-progress (partial) week, excluded from delta classification + +# Sanity check: curEnd is an exclusive 00:00-UTC date boundary and must be today +# (UTC) or earlier. Compare date-to-date -- a sub-day clock slack here would let +# *tomorrow* through as -EndDate when the script runs in the last hour before +# midnight UTC, which would include today's partial data and shift the window. +$todayUtc = [datetime]::UtcNow.Date +if ($curEnd -gt $todayUtc) { + throw "Resolved curEnd $($curEnd.ToString('yyyy-MM-dd')) UTC is in the future (today UTC = $($todayUtc.ToString('yyyy-MM-dd'))). -EndDate must be today or earlier. Refusing to bootstrap a report with a future window." +} + +$curEndStr = $curEnd.ToString('yyyy-MM-dd') +$curStartStr = $curStart.ToString('yyyy-MM-dd') +$prevStartStr = $prevStart.ToString('yyyy-MM-dd') +$prevEndStr = $prevEnd.ToString('yyyy-MM-dd') + +Write-Host "Resolved reporting window (UTC):" +Write-Host " Last 7 days: $curStartStr -> $curEndStr (exclusive upper bound)" +Write-Host " Baseline: $prevStartStr -> $prevEndStr" +Write-Host " 60-day trend: $($sixtyDayStart.ToString('yyyy-MM-dd')) -> $($sixtyDayEnd.ToString('yyyy-MM-dd')) (literal 60d ending today; chart includes current partial week)" +Write-Host " Trend delta cutoff: weeks < $($trendClassEnd.ToString('yyyy-MM-dd')) (startofweek(curEnd); pass as bucket-trends.js --end)" +# NOTE: Console output uses ASCII '->'; the HTML stamp below uses U+2192 arrows +# and U+00B7 middle-dots to match the template's canonical visual style. This +# is safe because $outText is written via [System.Text.UTF8Encoding]::new($false) +# which preserves multi-byte code points end-to-end (per the UTF-8 trap in +# template-readme.md, only the '@...@' heredoc-to-Set-Content path strips them). + +# --------------------------------------------------------------------------- # Paths +# --------------------------------------------------------------------------- $reportDir = Join-Path $env:USERPROFILE 'android-oce-reports' -$dataDir = Join-Path $reportDir "_data\$ReportingSunday" -$out = Join-Path $reportDir "oncall-wow-report-$ReportingSunday.html" +$dataDir = Join-Path $reportDir "_data\$curEndStr" +$out = Join-Path $reportDir "oncall-wow-report-$curEndStr.html" New-Item -ItemType Directory -Force $reportDir | Out-Null New-Item -ItemType Directory -Force $dataDir | Out-Null -# Read the template's reference dates so we can detect "unfilled stub" collisions. -# A reliable signal of "this file is the template stub": MULTIPLE markers all -# still match the template. We check title, the meta-line dates, AND the first -# KPI value — any divergence means real content has been written. +# --------------------------------------------------------------------------- +# Collision detection -- "unfilled stub" vs "real report" +# +# Fail-safe by design: we only silently overwrite when we can POSITIVELY +# identify the existing file as a freshly-bootstrapped, never-populated stub. +# The authoritative signal is the OCE-UNPOPULATED-STUB sentinel that bootstrap +# injects into every stub (see the write step below) and that validate-report.ps1 +# refuses to let a report publish with. A populated (validated) report can never +# carry it, so real work can never be misclassified as a stub. As a second guard +# we also require the first KPI to still hold the template's value. Anything that +# is not a positively-identified stub requires an explicit -Force to overwrite; +# when in doubt we refuse and exit rather than risk clobbering real work. +# --------------------------------------------------------------------------- +$stubSentinel = 'OCE-UNPOPULATED-STUB' $templateText = [IO.File]::ReadAllText($template) -function Get-FingerprintMarkers([string]$text) { - $m = @{} - if ($text -match '<title>([^<]+?)') { $m['title'] = $Matches[1].Trim() } - if ($text -match '
\s*([^<]+)') { $m['metaDate'] = $Matches[1].Trim() } - # NOTE: the "Generated" date is intentionally NOT a fingerprint marker — bootstrap - # stamps it to the actual run date below, so it never matches the template after copy. - # First KPI tile's value (e.g. "10.58 B"). Differs week-to-week. - if ($text -match '
\s*
[^<]+
\s*
([^<]+?)
') { $m['firstKpi'] = $Matches[1].Trim() } - return $m +function Get-FirstKpi([string]$text) { + if ($text -match '
\s*
[^<]+
\s*
([^<]+?)
') { return $Matches[1].Trim() } + return $null } +$templateFirstKpi = Get-FirstKpi $templateText -$templateMarkers = Get-FingerprintMarkers $templateText - -# Collision check if ((Test-Path $out) -and -not $Force) { - $existingText = [IO.File]::ReadAllText($out) - $existingMarkers = Get-FingerprintMarkers $existingText - - # "Unfilled stub" requires ALL markers to match the template AND the file size - # to be within 5% of the template's. ANY divergence (a single value updated, - # a single KPI populated, sections added) means real content exists. - $allMatch = $true - foreach ($k in $templateMarkers.Keys) { - if ($existingMarkers[$k] -ne $templateMarkers[$k]) { $allMatch = $false; break } - } - $sizeRatio = (Get-Item $out).Length / [Math]::Max(1, (Get-Item $template).Length) - $sizeClose = ($sizeRatio -ge 0.95) -and ($sizeRatio -le 1.05) - - $isUnfilledStub = $allMatch -and $sizeClose + $existingText = [IO.File]::ReadAllText($out) + $hasStubSentinel = $existingText.Contains($stubSentinel) + $existingFirstKpi = Get-FirstKpi $existingText + $firstKpiUnchanged = ($null -ne $templateFirstKpi) -and ($existingFirstKpi -eq $templateFirstKpi) + $isUnfilledStub = $hasStubSentinel -and $firstKpiUnchanged if ($isUnfilledStub) { - Write-Warning "Existing $out is an unfilled template stub (all template fingerprints match, size within 5%). Re-bootstrapping silently." + Write-Warning "Existing $out is an unpopulated bootstrap stub (carries the $stubSentinel sentinel and still holds the template's first-KPI value). Re-bootstrapping silently." } else { - $divergence = @() - foreach ($k in $templateMarkers.Keys) { - if ($existingMarkers[$k] -ne $templateMarkers[$k]) { - $divergence += " $k`: template='$($templateMarkers[$k])' existing='$($existingMarkers[$k])'" - } - } - if (-not $sizeClose) { - $divergence += " size: template=$((Get-Item $template).Length) bytes existing=$((Get-Item $out).Length) bytes ratio=$([Math]::Round($sizeRatio,2))x" - } + $reasons = @() + if (-not $hasStubSentinel) { $reasons += " - the $stubSentinel sentinel is absent (a populated/validated report never carries it)" } + if (-not $firstKpiUnchanged) { $reasons += " - the first KPI differs from the template (existing='$existingFirstKpi' template='$templateFirstKpi')" } Write-Error @" -A populated report already exists for the same Sunday bucket: +A report already exists for the same end-date bucket and is NOT a positively-identified unfilled stub: $out -Divergence from the template (which is why this is NOT classified as an unfilled stub): -$($divergence -join "`n") +Why this is not treated as a re-bootstrappable stub: +$($reasons -join "`n") -Per the SKILL.md filename-collision rule, do NOT silently overwrite. Either: - 1. Open the existing report, list its top-3 findings, and confirm what changed - in the new data before regenerating. Then re-run with -Force. +Refusing to overwrite to avoid clobbering real work. Either: + 1. Open the existing report, confirm what it contains, then re-run with -Force to overwrite. 2. Rename / delete the existing file and re-run. "@ exit 2 } } -# Bootstrap +# --------------------------------------------------------------------------- +# Bootstrap: copy the template +# --------------------------------------------------------------------------- Copy-Item $template $out -Force Write-Host "Bootstrapped $out from $template" Write-Host "Data folder: $dataDir" -# Stamp the actual run date into the "Generated ..." banner so the -# report never carries a stale template date (the v8 bug where it read 2026-06-15 -# on a file produced 2026-06-18). This is purely mechanical — today's clock date — -# and has zero off-by-one risk. The reporting-week / baseline / 60-day meta dates -# are still AUTHOR-set (see template-readme.md "Date fields"); bootstrap does not -# touch them because they must be verified against the user's intended week bucket. -# Use UTF8-no-BOM read/write so the report's emojis/arrows survive (the UTF-8 trap). -$today = (Get-Date).ToString('yyyy-MM-dd') +# --------------------------------------------------------------------------- +# Stamp the resolved window into the report header +# +# The template ships with hard-coded dates from a real prior week. We rewrite +# them mechanically so the header always matches what the queries actually +# targeted. This removes the last hand-authored date field and closes the +# window-drift class of bugs entirely (the resolved window is echoed in the +# report header for transparency). +# +# Uses UTF8-no-BOM read/write so the report's emojis/arrows survive (the +# UTF-8 trap called out in template-readme.md). +# --------------------------------------------------------------------------- +function Format-DateHuman([datetime]$d, [switch]$IncludeYear) { + # e.g. "Thu Jul 2, 2026" or "Thu Jul 2" + $dow = $d.ToString('ddd', [System.Globalization.CultureInfo]::InvariantCulture) + $mon = $d.ToString('MMM', [System.Globalization.CultureInfo]::InvariantCulture) + $day = $d.Day + if ($IncludeYear) { return "$dow $mon $day, $($d.Year)" } + return "$dow $mon $day" +} +# Display dates in the header. For a half-open [curStart, curEnd) window the +# calendar dates covered are [curStart, curEnd - 1 day], but the user-facing +# convention (matching the ADO issue "Jul 2 - Jul 9 when run on Jul 9") shows +# the interval endpoints -- so we render curStart -> curEnd literally. +# U+2192 RIGHTWARDS ARROW and U+00B7 MIDDLE DOT, matching the template's style. +$arrow = [char]0x2192 +$dot = [char]0x00B7 +$curLabel = "$(Format-DateHuman $curStart -IncludeYear:$false) $arrow $(Format-DateHuman $curEnd -IncludeYear:$true)" +$prevLabel = "$(Format-DateHuman $prevStart -IncludeYear:$false) $arrow $(Format-DateHuman $prevEnd -IncludeYear:$false)" +# 60-day trend: literal 60 days ending curEnd (today). curEnd is the exclusive +# upper bound, so the last calendar day carrying data is curEnd - 1 (yesterday). +# The final weekly bucket is the current in-progress week (partial) -- see +# bootstrap's $trendClassEnd note and bucket-trends.js --include-partial-end. +$sixtyDayLabel = "$(Format-DateHuman $sixtyDayStart -IncludeYear:$false) $arrow $(Format-DateHuman ($sixtyDayEnd.AddDays(-1)) -IncludeYear:$true)" +$todayStr = (Get-Date).ToUniversalTime().ToString('yyyy-MM-dd') + $outText = [IO.File]::ReadAllText($out) -$outText = [regex]::Replace($outText, 'Generated\s+[^<]*', "Generated $today") + +# 1) ... +$newTitle = "Android Broker $dot On-Call Report $([char]0x2014) Last 7 days ending $curEndStr" +$outText = [regex]::Replace($outText, '[^<]*', "$newTitle") + +# 2) The
block up through the closing
immediately +# before the badge. We use a single-line non-greedy match on the full block. +$newMeta = @" +
+ Last 7 days: $curLabel  vs  $prevLabel  $dot  + 60-day trend: $sixtyDayLabel (last 60 days; final bar in progress)  $dot  + Source: android_spans materialized views  $dot  + Generated $todayStr +
+"@ +# The template's meta div is multi-line; use single-line mode with .*? non-greedy. +$outText = [regex]::Replace($outText, '(?s)
.*?
', $newMeta) + +# 3) Belt-and-suspenders: if for some reason the meta-div rewrite above didn't +# hit (e.g. the template was restructured), still update the Generated banner +# so the file never carries a stale template date. +$outText = [regex]::Replace($outText, 'Generated\s+[^<]*', "Generated $todayStr") + +# 4) Inject the unpopulated-stub sentinel right after the DOCTYPE so a later +# bootstrap can POSITIVELY identify this file as a never-populated stub that +# is safe to silently overwrite. The author replaces the template's KPI / +# table / prose values and DELETES this line while filling the report; +# validate-report.ps1 refuses to pass a report that still carries it, so a +# published report can never be mistaken for a stub. +$stubMarker = "" +if ($outText -notmatch 'OCE-UNPOPULATED-STUB') { + $outText = [regex]::Replace($outText, '()', "`$1`n$stubMarker", 1) +} + [IO.File]::WriteAllText($out, $outText, [System.Text.UTF8Encoding]::new($false)) -Write-Host "Stamped Generated date: $today" +Write-Host "Stamped resolved window into and meta block. Generated=$todayStr." +# --------------------------------------------------------------------------- # Prune old _data folders +# --------------------------------------------------------------------------- $dataRoot = Join-Path $reportDir '_data' if (Test-Path $dataRoot) { $cutoff = (Get-Date).AddDays(-$DataRetentionDays) $oldFolders = Get-ChildItem $dataRoot -Directory | Where-Object { - # Folder name should look like a date; skip the current run's folder $_.FullName -ne $dataDir -and $_.LastWriteTime -lt $cutoff } diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/bucket-trends.js b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/bucket-trends.js index c602fba3..aaa0186d 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/bucket-trends.js +++ b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/bucket-trends.js @@ -1,29 +1,53 @@ #!/usr/bin/env node /** - * bucket-trends.js — Bucket every error code into 60-day trend categories. + * bucket-trends.js -- Bucket every error code into 60-day trend categories. + * + * This tool operates ONLY on the 60-day trend section, which still uses Sun-Sat + * weekly buckets (Kusto startofweek()-aligned). The primary/WoW section uses a + * rolling 7-day window and does NOT go through this script -- it consumes the + * two-bucket outputs of reliability-auth-only.kql / wow-movers.kql / etc. + * directly. * * Input: a Kusto MCP JSON result file from a query of the form: * * materialized_view('ErrorStatsMetrics') - * | where EventInfo_Time between (datetime(<start>) .. datetime(<end_exclusive>)) + * | where EventInfo_Time >= datetime(<trend_start>) and EventInfo_Time < datetime(<trend_end>) * | where isnotempty(error_code) and error_code != 'success' * | summarize errs=sum(countOverall), * devs=dcount_hll(hll_merge(countDevicesHll)) * by week=startofweek(EventInfo_Time), error_code - * | where week < datetime(<reporting_week_end_sunday>) // drop partial end-week! * | order by error_code asc, week asc * -// (Use dcount_hll on countDevicesHll, NOT sum(countDevices) — see ../docs/kusto-cheatsheet.md.) + * (Use dcount_hll on countDevicesHll, NOT sum(countDevices) -- see ../docs/kusto-cheatsheet.md.) + * + * LITERAL 60-day window ending today: + * The 60d trend now spans [today-60d, today), so the query no longer filters the + * partial in-progress week at the source -- the CHART wants it as the final bar. + * Instead this script splits two week sets: + * * classification weeks (complete Sun-Sat only) drive first/last/delta/spike/ + * peak-floor -- a partial week as "last" would read as a fake -99% improvement. + * * display weeks (adds the partial current week) drive the emitted `series` + * arrays and the JSON sidecar, so the sparkline/chart ends today. + * Pass `--end=<startofweek(today)> --include-partial-end` for this behavior. + * Without --include-partial-end the script behaves as before (display == classify). + * + * <end> convention: startofweek(today) -- i.e. the Sunday that OPENS the currently + * in-progress week. Every complete week strictly before that Sunday is classified; + * with --include-partial-end the in-progress week (bucket == <end>) is still charted. + * See assets/scripts/bootstrap-report.ps1 for how this is computed ($trendClassEnd). * * Usage: * node bucket-trends.js <mcp-output.json> * [--start=YYYY-MM-DD] [--end=YYYY-MM-DD] # inclusive start, EXCLUSIVE end (week-bucket) - * [--peak-floor=N] [--metric=devs|reqs] + * [--include-partial-end] [--peak-floor=N] [--metric=devs|reqs] * * --start defaults to the second-earliest week in the data (drops partial start week). * --end defaults to the most recent week, but the script will WARN-AND-DROP any week * where (latest EventInfo_Time in the bucket - week-start) < 6 days, because that - * is a partial end-week and will turn every error into a fake -99% improvement. + * is a partial in-progress week and will turn every error into a fake -99% improvement. + * --include-partial-end keep the partial current week (bucket >= --end) in the emitted + * `series` arrays / JSON sidecar for charting, while still EXCLUDING it from the + * delta/spike/first/last classification. No-op without --end. * * --metric=devs (default) buckets on weekly device counts (catches errors hitting more users) * --metric=reqs buckets on weekly request counts (catches per-device retry storms) @@ -46,7 +70,9 @@ * sparkline-data-generator script). The sidecar shape is: * { * "metric": "devs" | "reqs", - * "weeks": [iso, iso, ...], + * "weeks": [iso, ...], // DISPLAY weeks (incl. partial end) + * "classifyWeeks": [iso, ...], // complete weeks used for deltas + * "includePartialEnd": bool, * "buckets": { * "regression": [ { code, first, last, peak, delta, series: [N,N,...] }, ... ], * "spike": [...], @@ -54,6 +80,8 @@ * "flat": [...] * } * } + * (`series` follows `weeks` (display); first/last/delta/peak + * follow `classifyWeeks` (complete).) */ const fs = require('fs'); @@ -63,6 +91,7 @@ const startArg = (args.find(a => a.startsWith('--start=')) || '').split('=')[1]; const endArg = (args.find(a => a.startsWith('--end=')) || '').split('=')[1]; const metric = ((args.find(a => a.startsWith('--metric=')) || '').split('=')[1] || 'devs').toLowerCase(); const summary = args.includes('--summary'); +const includePartialEnd = args.includes('--include-partial-end'); const jsonArg = (args.find(a => a.startsWith('--json=')) || '').split('=')[1]; if (!['devs', 'reqs'].includes(metric)) { console.error(`--metric must be 'devs' or 'reqs', got '${metric}'`); @@ -74,7 +103,7 @@ const metricIdx = metric === 'reqs' ? 0 : 1; // [errs, devs] tuple const keyCol = ((args.find(a => a.startsWith('--key=')) || '').split('=')[1] || 'error_code'); if (!file) { - console.error('Usage: node bucket-trends.js <mcp-output.json> [--start=YYYY-MM-DD] [--end=YYYY-MM-DD] [--peak-floor=N] [--metric=devs|reqs] [--key=error_code|unified_error_type] [--summary] [--json=path]'); + console.error('Usage: node bucket-trends.js <mcp-output.json> [--start=YYYY-MM-DD] [--end=YYYY-MM-DD] [--include-partial-end] [--peak-floor=N] [--metric=devs|reqs] [--key=error_code|unified_error_type] [--summary] [--json=path]'); process.exit(1); } @@ -91,12 +120,12 @@ if (Array.isArray(schemaRow)) { } else { throw new Error('First row of results.items must be the schema row'); } -const iWeek = colNames.indexOf('week') >= 0 ? colNames.indexOf('week') : colNames.indexOf('wk'); +const iWeek = colNames.indexOf('week'); const iCode = colNames.indexOf(keyCol); const iErrs = colNames.indexOf('errs'); const iDevs = colNames.indexOf('devs'); if (iWeek < 0 || iCode < 0 || iErrs < 0 || iDevs < 0) { - throw new Error(`Schema must include week|wk, ${keyCol}, errs, devs. Got [${colNames.join(', ')}]`); + throw new Error(`Schema must include week, ${keyCol}, errs, devs. Got [${colNames.join(', ')}]`); } const items = d.results.items.slice(1); @@ -137,19 +166,31 @@ if (!endArg && weeks.length >= 4) { } } -const keep = weeks.filter(w => w >= startISO && (endISO ? w < endISO : true) && w !== droppedPartial); +// classKeep = complete Sun-Sat weeks used for delta/spike/first/last classification. +// A partial week here would produce a fake -99% improvement, so it is always excluded. +const classKeep = weeks.filter(w => w >= startISO && (endISO ? w < endISO : true) && w !== droppedPartial); +// displayKeep = weeks emitted in `series` / the JSON sidecar (what the chart draws). +// With --include-partial-end it adds the partial current week (bucket >= endISO up to +// endISO inclusive) so the chart ends today; otherwise it mirrors classKeep. +const displayKeep = (includePartialEnd && endISO) + ? weeks.filter(w => w >= startISO && w <= endISO) + : classKeep; if (!summary) { - console.log('All weeks: ', weeks); - console.log('Trend weeks: ', keep, `(${keep.length} complete)`); - console.log('Metric: ', metric, `(peak floor=${peakFloor.toLocaleString()})`); + console.log('All weeks: ', weeks); + console.log('Classify weeks: ', classKeep, `(${classKeep.length} complete)`); + if (includePartialEnd && displayKeep.length !== classKeep.length) { + console.log('Display weeks: ', displayKeep, `(${displayKeep.length}, incl. partial current week)`); + } + console.log('Metric: ', metric, `(peak floor=${peakFloor.toLocaleString()})`); } -if (keep.length < 4) { - console.warn(`[bucket-trends] WARN: only ${keep.length} kept weeks — trend buckets will be unstable. Need >= 4 for meaningful regression/improvement classification.`); +if (classKeep.length < 4) { + console.warn(`[bucket-trends] WARN: only ${classKeep.length} classify weeks — trend buckets will be unstable. Need >= 4 for meaningful regression/improvement classification.`); } const buckets = { regression: [], spike: [], improvement: [], flat: [] }; for (const [code, wd] of Object.entries(series)) { - const vals = keep.map(w => (wd[w] || [0, 0])[metricIdx]); + const vals = classKeep.map(w => (wd[w] || [0, 0])[metricIdx]); // classification + const displayVals = displayKeep.map(w => (wd[w] || [0, 0])[metricIdx]); // charted series const peak = Math.max(...vals); if (peak < peakFloor) continue; const first = vals[0] || 1, last = vals[vals.length - 1]; @@ -163,7 +204,7 @@ for (const [code, wd] of Object.entries(series)) { else if (delta > 0.15) cat = 'regression'; else if (delta < -0.15) cat = 'improvement'; else cat = 'flat'; - buckets[cat].push({ code, first, last, peak, delta: +(delta * 100).toFixed(1), series: vals }); + buckets[cat].push({ code, first, last, peak, delta: +(delta * 100).toFixed(1), series: displayVals }); } // Compact bucket-count line (always emitted, summary or verbose) @@ -188,7 +229,9 @@ if (jsonArg) { metric, key: keyCol, peakFloor, - weeks: keep, + weeks: displayKeep, + classifyWeeks: classKeep, + includePartialEnd, droppedPartial, buckets: Object.fromEntries( Object.entries(buckets).map(([k, arr]) => [ diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/summarize-attribution.js b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/summarize-attribution.js index 9d86a75c..41a99f62 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/summarize-attribution.js +++ b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/summarize-attribution.js @@ -16,16 +16,19 @@ * --label=shared_dev <shared.json> \ * --label=client_sku <sku.json> * - * Per-file schema: row[0] must include `error_code`, `wk`/`week`, `devs`/`countDevices`, + * Per-file schema: row[0] must include `error_code`, `week`, `devs`/`countDevices`, * and exactly one trailing string column (the dimension value). * - * 2) Union mode (NEW, recommended for 2-week WoW attribution — one query covers all dims): + * 2) Union mode (recommended for WoW attribution -- one query covers all dims): * * node summarize-attribution.js --union <attribution-union.json> * * Expected schema (any column order): * dim string -- short label e.g. 'span', 'calling_app', 'broker_ver' - * wk | week datetime + * week datetime -- for the rolling-window queries this is the + * bucket START datetime (prevStart or curStart); + * the script sorts them lexicographically and + * treats the smallest as prev, largest as cur. * error_code string (or `error_type` — use --key=error_type to switch) * val_string string } EITHER `val_string`+`val_bool` (Kusto union of * val_bool bool } mixed-type slice columns) ... @@ -33,7 +36,7 @@ * devs long (use `dcount_hll(hll_merge(countDevicesHll))` upstream) * errs long (optional — request count, used for retry-storm detection) * - * The union form is what Step 5 of SKILL.md now recommends — 1 round-trip vs 7. + * The union form is what Step 5 of SKILL.md recommends — 1 round-trip vs 7. * See assets/queries/attr-union-by-dim.kql. * * Output: per error_code, per dimension, the top-5 values for each week (prior + curr), @@ -99,11 +102,11 @@ function loadSliceFile({ label, file }) { } const cols = Object.keys(schema); const idxCode = cols.indexOf(keyCol); - let idxWeek = cols.indexOf('wk'); if (idxWeek < 0) idxWeek = cols.indexOf('week'); + const idxWeek = cols.indexOf('week'); let idxDevs = cols.indexOf('devs'); if (idxDevs < 0) idxDevs = cols.indexOf('countDevices'); let idxErrs = cols.indexOf('errs'); if (idxErrs < 0) idxErrs = cols.indexOf('countOverall'); if (idxCode < 0 || idxWeek < 0 || idxDevs < 0) { - throw new Error(`${file}: schema must include ${keyCol}, wk|week, devs|countDevices. Got [${cols.join(', ')}]`); + throw new Error(`${file}: schema must include ${keyCol}, week, devs|countDevices. Got [${cols.join(', ')}]`); } // Find the dim column. When schema was provided as an array (run-kql.ps1) we // don't have type info, so fall back to "any remaining column" (typically the @@ -118,11 +121,11 @@ function loadSliceFile({ label, file }) { const map = {}; for (const r of rows.slice(1)) { - const code = r[idxCode], wk = r[idxWeek]; + const code = r[idxCode], week = r[idxWeek]; const dim = (r[idxDim] === null || r[idxDim] === '') ? '(blank)' : r[idxDim]; const devs = r[idxDevs] || 0; const errs = idxErrs >= 0 ? (r[idxErrs] || 0) : 0; - const slot = ((map[code] ||= {})[wk] ||= {})[dim] ||= { devs: 0, errs: 0 }; + const slot = ((map[code] ||= {})[week] ||= {})[dim] ||= { devs: 0, errs: 0 }; slot.devs += devs; slot.errs += errs; } return { label, map }; @@ -134,8 +137,8 @@ function loadUnion(file) { const rows = d.results.items; const schemaRaw = rows[0]; // Two schema shapes are supported: - // (a) Object form (MCP tool): { dim: 0, wk: 1, ... } — keys are column names - // (b) Array form (REST helper assets/scripts/run-kql.ps1): ['dim', 'wk', ...] + // (a) Object form (MCP tool): { dim: 0, week: 1, ... } — keys are column names + // (b) Array form (REST helper assets/scripts/run-kql.ps1): ['dim', 'week', ...] // Detect and normalize to an object map { colName -> index }. let schema; if (Array.isArray(schemaRaw)) { @@ -150,7 +153,7 @@ function loadUnion(file) { const idx = name => cols.indexOf(name); const idxDim = idx('dim'); const idxCode = idx(keyCol); - let idxWeek = idx('wk'); if (idxWeek < 0) idxWeek = idx('week'); + const idxWeek = idx('week'); let idxDevs = idx('devs'); if (idxDevs < 0) idxDevs = idx('countDevices'); let idxErrs = idx('errs'); if (idxErrs < 0) idxErrs = idx('countOverall'); // Kusto auto-renames duplicate column names from union branches: a column @@ -165,14 +168,14 @@ function loadUnion(file) { idx('val_bool') >= 0 ? idx('val_bool') : idx('val_string_bool'); if (idxDim < 0 || idxCode < 0 || idxWeek < 0 || idxDevs < 0 || idxValS < 0) { - throw new Error(`Union file ${file}: schema must include dim, ${keyCol}, wk|week, devs|countDevices, val_string|val|val_string_string (and optionally val_bool|val_string_bool). Got [${cols.join(', ')}]`); + throw new Error(`Union file ${file}: schema must include dim, ${keyCol}, week, devs|countDevices, val_string|val|val_string_string (and optionally val_bool|val_string_bool). Got [${cols.join(', ')}]`); } - // perDim[label].map[code][wk][dimVal] = { devs, errs } + // perDim[label].map[code][week][dimVal] = { devs, errs } const byDim = {}; for (const r of rows.slice(1)) { const label = r[idxDim]; const code = r[idxCode]; - const wk = r[idxWeek]; + const week = r[idxWeek]; const valS = r[idxValS]; const valB = idxValB >= 0 ? r[idxValB] : null; let v; @@ -182,7 +185,7 @@ function loadUnion(file) { const devs = r[idxDevs] || 0; const errs = idxErrs >= 0 ? (r[idxErrs] || 0) : 0; const target = byDim[label] ||= { label, map: {} }; - const slot = ((target.map[code] ||= {})[wk] ||= {})[v] ||= { devs: 0, errs: 0 }; + const slot = ((target.map[code] ||= {})[week] ||= {})[v] ||= { devs: 0, errs: 0 }; slot.devs += devs; slot.errs += errs; } return Object.values(byDim); @@ -194,7 +197,7 @@ const slices = unionFile ? loadUnion(unionFile) : inputs.map(loadSliceFile); const universe = {}; for (const s of slices) { for (const [code, wks] of Object.entries(s.map)) { - for (const wk of Object.keys(wks)) ((universe[code] ||= {})[wk] = true); + for (const week of Object.keys(wks)) ((universe[code] ||= {})[week] = true); } } const codes = Object.keys(universe).sort(); diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/validate-report.ps1 b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/validate-report.ps1 index e13494b3..c96cd6ff 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/validate-report.ps1 +++ b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/validate-report.ps1 @@ -4,7 +4,7 @@ .DESCRIPTION Runs all required pre-publish checks per SKILL.md "Output checklist": - 1. No stale-template tokens ({{...}} placeholders or "EXAMPLE CONTENT BELOW" sentinel). + 1. No stale-template tokens ({{...}} placeholders, "EXAMPLE CONTENT BELOW" sentinel, or the OCE-UNPOPULATED-STUB bootstrap sentinel). 2. No `devs` / `reqs` in user-facing text (only allowed inside <pre><code> KQL blocks). 3. No U+FFFD (Unicode replacement character) — catches mojibake from emoji edits. 4. Section 2 callouts are siblings, NOT nested. Tracks <div> open/close depth @@ -24,6 +24,13 @@ 9c. .dim-pct content guard — every dim-pct cell must be a short bare percentage (no "(...)" annotations, <= 9 chars). A long pct value starves the flex name column and collapses real labels to "1...". + 10. Fabricated-sparkline heuristic: data-trend arrays whose peak sits far + below the bucketer's peak-floor are flagged as likely hand-rolled. + 11. Rolling-window header integrity: the meta-line "Last 7 + days: <curStart> → <curEnd>" dates must match the filename's end-date + (curEnd = end-date, curStart = end-date - 7d). Prevents a stale + template stub from being published as if it were fresh, and catches + any hand-edit that broke the auto-stamp. Exits with non-zero status if any HARD check fails (stale tokens, devs/reqs leak, U+FFFD, unbalanced div depth, missing layout-guard CSS). @@ -34,7 +41,7 @@ .EXAMPLE .\validate-report.ps1 - .\validate-report.ps1 -Path C:\path\to\oncall-wow-report-2026-05-03.html + .\validate-report.ps1 -Path C:\path\to\oncall-wow-report-2026-07-09.html #> [CmdletBinding()] param( @@ -70,13 +77,13 @@ Write-Host "Validating: $Path" Write-Host ("Size: {0:N0} bytes" -f (Get-Item $Path).Length) Write-Host "" -# ---- 1. Stale tokens / EXAMPLE sentinel ---- -$stale = Select-String -Path $Path -Pattern '\{\{|EXAMPLE CONTENT BELOW|EXAMPLE_' +# ---- 1. Stale tokens / EXAMPLE sentinel / unpopulated-stub sentinel ---- +$stale = Select-String -Path $Path -Pattern '\{\{|EXAMPLE CONTENT BELOW|EXAMPLE_|OCE-UNPOPULATED-STUB' if ($stale.Count -gt 0) { Add-Fail "Stale template tokens found ($($stale.Count)). First few:" $stale | Select-Object -First 5 | ForEach-Object { Write-Host " L$($_.LineNumber): $($_.Line.Trim().Substring(0, [Math]::Min(110, $_.Line.Trim().Length)))" } } else { - Pass "No stale {{...}} tokens or EXAMPLE sentinel" + Pass "No stale {{...}} tokens, EXAMPLE sentinel, or unpopulated-stub sentinel" } # ---- 2. devs / reqs in user-facing text ---- @@ -299,7 +306,7 @@ foreach ($m in $trendMatches) { $arrStr = $m.Groups[1].Value $vals = $arrStr.Split(',') | ForEach-Object { try { [double]$_.Trim() } catch { 0 } } if ($vals.Count -lt 6) { continue } - # Filter 1: trend with all values < 100 is suspicious (real codes don't sit at 30-50 devices/wk for 8 weeks) + # Filter 1: trend with all values < 100 is suspicious (real codes don't sit at 30-50 devices/week for 8 weeks) $maxVal = ($vals | Measure-Object -Maximum).Maximum if ($maxVal -lt 100) { $suspectCount++ @@ -311,11 +318,51 @@ foreach ($m in $trendMatches) { # Skip this; too easy to false-positive on genuinely monotonic real series like no_tokens_found. } if ($suspectCount -gt 0) { - Add-Warn "$suspectCount data-trend array(s) have peak value < 100 (suspicious — real WoW-table series usually peak >= 100 devices/wk). Likely fabricated. First: [$suspectFirst]. Source from assets/queries/wow-table-sparkline-series.kql instead." + Add-Warn "$suspectCount data-trend array(s) have peak value < 100 (suspicious — real WoW-table series usually peak >= 100 devices/week). Likely fabricated. First: [$suspectFirst]. Source from assets/queries/wow-table-sparkline-series.kql instead." } else { Pass "No suspicious low-peak data-trend arrays detected" } +# ---- 11. Rolling-window header integrity ---- +# The meta line must read "Last 7 days: <weekday> <Mon> <D>[, YYYY] -> <weekday> +# <Mon> <D>, YYYY", and those dates must be self-consistent with the filename's +# end-date (curEnd = filename date, curStart = curEnd - 7d). +$filename = Split-Path $Path -Leaf +if ($filename -match '^oncall-wow-report-(\d{4}-\d{2}-\d{2})\.html$') { + $fnEnd = [datetime]::ParseExact($Matches[1], 'yyyy-MM-dd', + [System.Globalization.CultureInfo]::InvariantCulture) + $fnStart = $fnEnd.AddDays(-7) + + # Match: "Last 7 days: <weekday> <Mon> <D>[, YYYY]? -> <weekday> <Mon> <D>, YYYY" + # -> may be U+2192 arrow or ASCII "->". + $metaRe = 'Last\s+7\s+days:\s*(?:<[^>]+>)?\s*(\w{3})\s+(\w{3})\s+(\d{1,2})(?:,\s*(\d{4}))?\s*(?:[\u2192]|->|->)\s*(\w{3})\s+(\w{3})\s+(\d{1,2}),\s*(\d{4})' + if ($content -match $metaRe) { + $s = "$($Matches[2]) $($Matches[3]), $($Matches[8])" # e.g. "Jul 2, 2026" (year from end side) + $e = "$($Matches[6]) $($Matches[7]), $($Matches[8])" + try { + $mStart = [datetime]::ParseExact($s, 'MMM d, yyyy', + [System.Globalization.CultureInfo]::InvariantCulture) + $mEnd = [datetime]::ParseExact($e, 'MMM d, yyyy', + [System.Globalization.CultureInfo]::InvariantCulture) + # If the start ended up after the end (year boundary), roll the start back a year. + if ($mStart -gt $mEnd) { $mStart = $mStart.AddYears(-1) } + if ($mStart -ne $fnStart) { + Add-Fail "Meta-line 'Last 7 days' start = $($mStart.ToString('yyyy-MM-dd')) but filename implies $($fnStart.ToString('yyyy-MM-dd')) (filename end $($fnEnd.ToString('yyyy-MM-dd')) - 7d). Header is out of sync with filename -- did you edit dates by hand? Re-run bootstrap-report.ps1." + } elseif ($mEnd -ne $fnEnd) { + Add-Fail "Meta-line 'Last 7 days' end = $($mEnd.ToString('yyyy-MM-dd')) but filename says $($fnEnd.ToString('yyyy-MM-dd'))." + } else { + Pass "Meta-line rolling-window dates match filename ($($mStart.ToString('yyyy-MM-dd')) -> $($mEnd.ToString('yyyy-MM-dd')))" + } + } catch { + Add-Warn "Meta-line dates matched pattern but failed to parse: '$s' / '$e'. Skipping integrity check." + } + } else { + Add-Warn "Meta-line 'Last 7 days: ... -> ...' pattern not found. Either the report predates the rolling-window rewrite or the header was hand-edited. Re-run bootstrap-report.ps1 to restore the auto-stamped meta line." + } +} else { + Add-Warn "Filename '$filename' does not match 'oncall-wow-report-YYYY-MM-DD.html'; skipping meta-line date consistency check." +} + Write-Host "" if ($failures.Count -eq 0) { Write-Host "All hard checks passed." -ForegroundColor Green diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/visual-smoke.ps1 b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/visual-smoke.ps1 index c4ca9272..dc8076c7 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/scripts/visual-smoke.ps1 +++ b/.github/skills/oncall-weekly-telemetry-report/assets/scripts/visual-smoke.ps1 @@ -30,7 +30,7 @@ .EXAMPLE .\visual-smoke.ps1 - # checks the latest report + writes ~/android-oce-reports/_visual/oncall-wow-report-<sunday>.png + # checks the latest report + writes ~/android-oce-reports/_visual/oncall-wow-report-<end-date>.png .EXAMPLE .\visual-smoke.ps1 -Path C:\path\to\report.html diff --git a/.github/skills/oncall-weekly-telemetry-report/assets/templates/report-template.html b/.github/skills/oncall-weekly-telemetry-report/assets/templates/report-template.html index 20a1654e..d4249b96 100644 --- a/.github/skills/oncall-weekly-telemetry-report/assets/templates/report-template.html +++ b/.github/skills/oncall-weekly-telemetry-report/assets/templates/report-template.html @@ -1,14 +1,19 @@ <!DOCTYPE html> <!-- TEMPLATE - see assets/templates/template-readme.md for usage. This file IS a real prior-week - report kept as a structural reference. To produce a new week's report, copy this - file to ~/android-oce-reports/oncall-wow-report-<sunday>.html and edit IN PLACE, - replacing prior-week dates, KPI values, table rows, attribution-card prose, and - PR citations with the current week's data. The CSS in <head> is canonical - do - not restyle per week. Authors mark unfinished sections with the literal sentinel string the validator greps for (see assets/templates/template-readme.md). --> + report kept as a structural reference. To produce a new report, run + assets/scripts/bootstrap-report.ps1 which copies this file to + ~/android-oce-reports/oncall-wow-report-<end-date>.html and STAMPS the + resolved rolling-7-day window into the <title>, meta line, and Generated + banner. You then edit IN PLACE, replacing prior-window KPI values, table + rows, attribution-card prose, and PR citations with the current window's + data. The CSS in <head> is canonical - do not restyle per run. Header + dates (title, meta line, Generated) are stamped by bootstrap - do not + hand-edit them. Authors mark unfinished sections with the literal sentinel + string the validator greps for (see assets/templates/template-readme.md). --> <html lang="en"> <head> <meta charset="UTF-8"> -<title>Android Broker · On-Call Weekly Report — Week of May 3, 2026 +Android Broker · Weekly On-Call Report