Skip to content

feat(replay): Add agent-readable replay timelines#913

Open
dcramer wants to merge 13 commits intomainfrom
codex/replay-agent-analysis
Open

feat(replay): Add agent-readable replay timelines#913
dcramer wants to merge 13 commits intomainfrom
codex/replay-agent-analysis

Conversation

@dcramer
Copy link
Copy Markdown
Member

@dcramer dcramer commented May 4, 2026

Add path-aware replay discovery and deterministic replay summaries so agents can investigate user behavior from Session Replay without reading raw rrweb frames.

Replay Discovery

replay list now supports --path, --entry-path, and --exit-path for parsed URL pathname matching. --url remains a broader replay-search text filter, while --friction exposes indexed error, warning, rage-click, and dead-click cohorts.

Replay Evidence

replay events lists normalized events with offsets, URL path/query fields, filters, and JSONL output. replay summarize returns route flow, counts, timings, friction signals, and notable events for evidence-backed analysis.

Generated Skill

Replay command docs and committed skill references are regenerated from command metadata and fragments; generated skill content was not hand-authored.

Refs GH-907

dcramer and others added 2 commits May 4, 2026 19:25
Paginate replay recording segment downloads so long replays are fully
available to replay view and future replay inspection commands. The API
caps recording segment pages at 100 segments, so the helper now follows
Link cursors and stops when the replay metadata count is satisfied.

**Replay API Compatibility**

Remove `ota_updates` from the replay list field set because the live
replay index endpoint rejects it as invalid. The replay schemas still
tolerate `ota_updates` when a payload includes it.

**Local Smoke Testing**

Add `bun run cli` as a lightweight runner for live CLI checks without
regenerating docs and SDK first.

Validated with focused replay and explore tests, typecheck, lint, and a
live read-only smoke test on a 471-segment replay that fetched five
segment pages: 100, 100, 100, 100, 71.

Refs GH-907

---------

Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Miguel Betegón <miguelbetegongarcia@gmail.com>
Add path-aware replay discovery, normalized event listing, and deterministic replay summaries so agents can inspect session behavior without relying on raw replay frames.

Generate skill and docs output from command metadata. Add focused coverage for path matching, event normalization, summary signals, and JSONL output.

Refs GH-907

Co-Authored-By: OpenAI Codex <noreply@openai.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://cli.sentry.dev/_preview/pr-913/

Built to branch gh-pages at 2026-05-06 20:29 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Codecov Results 📊

6704 passed | Total: 6704 | Pass Rate: 100% | Execution Time: 0ms

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +18
Passed Tests 📈 +18
Failed Tests
Skipped Tests

All tests are passing successfully.

✅ Patch coverage is 82.28%. Project has 13925 uncovered lines.
✅ Project coverage is 76.89%. Comparing base (base) to head (head).

Files with missing lines (8)
File Patch % Lines
src/lib/replay-events.ts 73.86% ⚠️ 160 Missing
src/lib/replay-summary.ts 79.41% ⚠️ 139 Missing
src/commands/replay/event/list.ts 87.55% ⚠️ 33 Missing
src/commands/replay/shared.ts 66.67% ⚠️ 26 Missing
src/lib/replay-search.ts 58.33% ⚠️ 25 Missing
src/commands/replay/target.ts 78.57% ⚠️ 18 Missing
src/commands/replay/summarize.ts 94.29% ⚠️ 16 Missing
src/lib/api/replays.ts 84.21% ⚠️ 6 Missing
Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    76.66%    76.89%    +0.23%
==========================================
  Files          303       310        +7
  Lines        57997     60254     +2257
  Branches         0         0         —
==========================================
+ Hits         44463     46329     +1866
- Misses       13534     13925      +391
- Partials         0         0         —

Generated by Codecov Action

Comment thread src/commands/replay/summarize.ts Outdated
Comment thread src/commands/replay/list.ts Outdated
Render missing replay summary durations without a seconds suffix and make --problem-only distinct from --friction by limiting it to indexed errors and warnings.

Co-Authored-By: OpenAI Codex <codex@openai.com>
Comment thread src/lib/replay-events.ts Outdated
Classify replay Log frames with mixed-case error levels as errors so --kind error and summary signal detection agree.

Co-Authored-By: OpenAI Codex <codex@openai.com>
Comment thread src/lib/replay-search.ts
Comment thread src/commands/replay/list.ts Outdated
dcramer and others added 2 commits May 4, 2026 12:10
Match root route filters against child paths and avoid adding multiple generic URL search clauses for positional replay route filters.

Co-Authored-By: OpenAI Codex <codex@openai.com>
Represent replay routes as chronological visits instead of unique path aggregates. Add bounded route timing fields, next-path context, per-visit event counts, and explicit user-interaction metadata so agents can reason about repeated navigation paths.

Split replay event counts for clicks, taps, inputs, focuses, blurs, and scrolls. This avoids overloading input counts and keeps the summary JSON useful for generalized replay analysis.

Refs GH-907

Co-Authored-By: OpenAI Codex <noreply@openai.com>
Comment thread src/commands/replay/list.ts Outdated
dcramer and others added 2 commits May 4, 2026 12:44
Expose replay platform, SDK, replay type, and recording parser stats from replay summarize. This keeps the summary output honest for non-web or sparsely parsed recordings without adding a new command surface.

Include the generated replay skill reference updates from the summary schema change.

Refs GH-907

Co-Authored-By: OpenAI Codex <noreply@openai.com>
Fetch additional replay pages when client-side filters are active so --limit applies to the filtered result set instead of only the first server page. Keep the loop bounded by the shared pagination limit and preserve the final server cursor for navigation.

Refs GH-907

Co-Authored-By: OpenAI Codex <noreply@openai.com>
Comment thread src/commands/replay/list.ts Outdated
dcramer and others added 2 commits May 4, 2026 12:53
When --url is provided with route path flags, use the explicit URL value as the single server-side URL prefilter and leave path/entry/exit semantics to client-side filtering. This avoids accidental AND narrowing from duplicate url: search tokens.

Refs GH-907

Co-Authored-By: OpenAI Codex <noreply@openai.com>
Store a mid-page cursor when client-side replay filters fill a result page before the underlying API page is exhausted. This keeps subsequent -c next calls from skipping matching replays on path or friction-filtered searches.

Refs GH-907
Co-Authored-By: OpenAI Codex <codex@openai.com>
Comment thread src/commands/replay/event/list.ts
dcramer and others added 2 commits May 4, 2026 13:07
Reject --before and --after unless --around is also present. This prevents replay event list from silently ignoring window options and fails before fetching replay data.

Refs GH-907
Co-Authored-By: OpenAI Codex <codex@openai.com>
Fetch multiple API pages when an unfiltered replay list asks for more than the API page size. Reuse mid-page cursor bookmarks so pagination can resume without duplicating rows.

Refs GH-907

Co-Authored-By: OpenAI Codex <codex@openai.com>
@dcramer dcramer marked this pull request as ready for review May 5, 2026 00:17
Comment thread src/commands/replay/list.ts Outdated
Comment on lines +323 to +325
nextCursor: string | undefined;
}
): ReplayPageResult {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The cursor encoding/decoding logic incorrectly handles mid-page cursors on the first page. An undefined server cursor is encoded as "" and then decoded back to undefined, causing pagination to restart.
Severity: HIGH

Suggested Fix

Modify decodeReplayCursor to preserve an empty string for serverCursor instead of converting it to undefined. The expression serverCursor: serverCursor || undefined should be changed to serverCursor: serverCursor to distinguish a mid-page cursor on the first page from a request that has no cursor at all.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/commands/replay/list.ts#L323-L325

Potential issue: The pagination logic has a flaw when creating a mid-page cursor on the
first page of client-filtered results. The `serverCursor` is `undefined`, which
`encodeReplayCursor` converts to an empty string (e.g., `"|replayId"`). However,
`decodeReplayCursor` converts this empty string back to `undefined`. This causes the
subsequent fetch to request page 1 again instead of the next page. If the
`afterReplayId` from the cursor is no longer present in the re-fetched page 1,
`replayStartIndex` returns 0, leading to duplicate replays being emitted.

Did we get this right? 👍 / 👎 to inform future reviews.

Copy link
Copy Markdown
Member

@BYK BYK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a fan of the API. Looks like we spread the search syntax arguments to custom flags instead of reusing the existing helpers and UX with -q/--search. I'm also skeptical about the --jsonl flag. That shouldn't be necessary our existing --json output is capable of emitting JSONL by default when the output system is fed json data through the yield iterations.

Comment thread docs/src/fragments/commands/replay.md Outdated
--around 01:23 --json

# Emit newline-delimited JSON for large timelines
sentry replay events my-org/346789a703f6454384f1de473b8b9fcc --json --jsonl
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our --json mode should default to JSONL in this case?

**Flags:**
- `-k, --kind <value>... - Event kind filter (navigation, click, tap, input, focus, blur, scroll, viewport, mutation, dom-snapshot, breadcrumb, network, console, error, span, web-vital, memory, video, mobile, unknown)`
- `-u, --url <value> - Filter events by current or target URL substring`
- `--path <value> - Filter events by parsed URL pathname`
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can make this an optional positional argument instead as it looks quite common and looks like a natural sub-section?

- `-k, --kind <value>... - Event kind filter (navigation, click, tap, input, focus, blur, scroll, viewport, mutation, dom-snapshot, breadcrumb, network, console, error, span, web-vital, memory, video, mobile, unknown)`
- `-u, --url <value> - Filter events by current or target URL substring`
- `--path <value> - Filter events by parsed URL pathname`
- `-q, --contains <value> - Filter events by text in labels, messages, URLs, selectors, or data`
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-q is reserved for the short version of --search so we should either change it or change --contains to --search?

- `-u, --url <value> - Filter events by current or target URL substring`
- `--path <value> - Filter events by parsed URL pathname`
- `-q, --contains <value> - Filter events by text in labels, messages, URLs, selectors, or data`
- `--selector <value> - Filter events by selector substring`
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not clear how --selector and --contains work together or how they defer?

Comment on lines +24 to +28
- `--from <value> - Start offset (seconds, 90s, 01:23, or 1:02:03)`
- `--to <value> - End offset (seconds, 90s, 01:23, or 1:02:03)`
- `--around <value> - Center an evidence window around this offset`
- `--before <value> - Window before --around (default: 10s)`
- `--after <value> - Window after --around (default: 30s)`
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use the existing -t/--period notation for these:

-t, --period <period> | Time range: "7d", "2026-04-01..2026-05-01", ">=2026-04-01" (default: "7d")

Comment thread script/generate-docs-sections.ts Outdated
Comment on lines +135 to +143
return [
...new Set(
route.commands.map((cmd) =>
cmd.path.startsWith(prefix)
? cmd.path.slice(prefix.length)
: (cmd.path.split(" ").at(-1) ?? route.name)
)
),
];
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Array.from(new Set(....)) as that's way more efficient

Comment thread src/commands/replay/index.ts Outdated
fullDescription:
"Search and inspect Session Replays from your Sentry organization.\n\n" +
"Commands:\n" +
" event Inspect normalized events from a replay (alias: events)\n" +
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to undertand the difference between replay list and replay event list. Care to elaborate?

Comment thread src/commands/replay/list.ts Outdated
const COMMAND_NAME = "replay list";
const SIMPLE_SEARCH_VALUE_RE = /^[^\s:"]+$/;

function encodeReplayCursor(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we already have generic cursor management commands? Do replays have different cursors?

Comment thread src/commands/replay/list.ts Outdated
}

function quoteSearchValue(value: string): string {
return SIMPLE_SEARCH_VALUE_RE.test(value) ? value : JSON.stringify(value);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same search syntax we use in explore? If yes we should be using existing parsers and validators for them.

Reduce replay list and event commands to the reviewed MVP surface. Use shared search, pagination, and JSON output hooks instead of bespoke filters and JSONL flags.

Co-Authored-By: OpenAI Codex <codex@openai.com>
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 17b8e28. Configure here.

Comment thread src/lib/replay-search.ts
return replayUrlPathMatches(urls.at(-1), path);
}
return urls.some((url) => replayUrlPathMatches(url, path));
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused exported function and type are dead code

Low Severity

replayMatchesPath and ReplayPathMatchMode are new exports that are never imported or referenced anywhere in the codebase. The PR description mentions --path, --entry-path, and --exit-path filters on replay list, but none of those flags were actually added to the list.ts command. This function appears to be scaffolding for unimplemented features, left behind as dead code.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 17b8e28. Configure here.

[`Open the org-scoped replay instead: ${command} ${org}/${replayId}`]
);
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate validateReplayProjectScope with differing behavior

Medium Severity

Two validateReplayProjectScope implementations now coexist: the new synchronous export in shared.ts and the original async local function in view.ts. They have materially different behavior — view.ts throws when a replay has no project_id association and falls back to a getProject() API call, while shared.ts silently returns in both cases. Future bug fixes applied to one will likely miss the other, and the behavioral gap can confuse users who get different validation outcomes from replay view vs replay events or replay summarize.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 17b8e28. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants