Skip to content

Ambient state snapshots, build-attributed sessions, and span tooling that can't lie - #152

Open
kyleve wants to merge 12 commits into
mainfrom
cursor/periscope-ambient-snapshots-3fed
Open

Ambient state snapshots, build-attributed sessions, and span tooling that can't lie#152
kyleve wants to merge 12 commits into
mainfrom
cursor/periscope-ambient-snapshots-3fed

Conversation

@kyleve

@kyleve kyleve commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Three related things, all in service of being able to read a log weeks later and know what it meant: the system state behind every event, the build behind every session, and span surfaces that don't render claims their data can't support.

Closes nine backlog entries — the ambient-snapshot P0 in Shared/Periscope/TODOs.md, the LogSession build-naming P2 in the root TODOs.md, and seven span-tooling / docs quick wins.

Ambient state snapshots

Every event already referenced one SDLogSession row for "which build, on which device". It now also references one SDAmbientSnapshot row for "what was the system doing" — network, thermal state, power mode, lifecycle — so correlating an error with connectivity stops being a timestamp hunt.

  • AmbientEvent.reporting (.state / .occurrence) draws the line between a lasting condition and a passing moment. A memory warning is the one built-in .occurrence: the app is not "in a memory warning" afterwards, so it must not stick to every later record.
  • AmbientSnapshot is the latest value of every stateful kind, and its identity is the dedupe key — applying(_:) returns self whenever nothing actually moved, so a run of records sharing one state shares one stored row. folding(_:into:) makes an empty snapshot unrepresentable: a row describing the system while carrying no values would be worse than an honest nil.
  • The pipeline folds and stamps in the existing in-lock buffer path, which now returns the record as buffered so the journal and the sinks get the stamped copy. Folding happens before stamping, so an ambient event carries the state it announces rather than the one it replaced.
  • The crash journal carries it, so records that only exist because of a crash still say what the system was doing.
  • Storage is one row per distinct state plus an indexed ambientSnapshotID, and retention takes unreferenced snapshots with it.
  • Thermal state and low-power mode report at started(), so their state isn't unknown until it next changes.

A session can name the build it came from

LogSession.current() read only CFBundleShortVersionString / CFBundleVersion, and the Where app pins both in the manifest — so every developer build read v1.0 (1) and weeks-old logs couldn't be tied to the code that produced them.

LogSession.attributes ([LogSessionAttributeKey: String]) is the seam: the host app fills it at bootstrap, keeping Periscope below the Where modules rather than reaching up for a build stamp. Where's stamp-build-info.sh now writes WhereConfiguration / WhereSwiftOptimizationLevel / WhereSwiftCompilationMode beside the commit keys, BuildInfo reads them back, and the viewer's session picker shows commit + optimization level.

The optimization level is the load-bearing field, not the commit: a span duration from an -Onone build says nothing about the shipping app, and the configuration alone can't answer it (a Debug configuration can be compiled -O). So SpanHistoryView also gained a build scope — all builds, this session, or every session built at the current one's level — and labels the active scope, because a p95 that silently pools an unoptimized build with an optimized one measures nothing.

Span tooling that can't lie

  • SpanNode carried parallel ended / exitMode / duration optionals, so a span whose end payload failed to decode rendered an exit chip beside a "running" duration — a reading that cannot be true, since the exit comes from an indexed column and the duration from the payload. One Outcome replaces them, and its Timing distinguishes an end that measured nothing (an orphan) from one whose payload wouldn't decode.
  • Span history labels a recovered bucket as recovered. A SpanEnded whose payload won't decode still groups (by its message), but the row says the name isn't the recorded one instead of minting what looks like a real span kind.
  • Every read model logs its failures to PeriscopeToolsLog.failures — deliberately OSLog, not Periscope: these surfaces reload on every store commit, so logging into the store they read would turn one corrupt row into a refresh loop.
  • The orphan sweep stops parsing payloads to decide relaunches. SpanRelaunchPolicy persists as a column on the began row. A row predating the column falls back to its payload, and a payload that can't be read can't prove the span asked to survive — so it's closed, with the failure logged rather than silently deciding against the policy.
  • SpanTreeRow reads \.logRowDensity instead of hard-coding comfortable, so the viewer's density picker actually reaches span-tree rows.

Notable decisions

  • Two ambient stamping sites, not one. The drop report is synthesized during the drain and never passes through the buffer path, so it is stamped where it is constructed — otherwise the one record that marks a gap in the history couldn't say what the system was doing when the gap opened.
  • No app-lifecycle startup baseline. Emitting one means reading UIApplication.shared.applicationState at launch, which reads .background even for a user tap under the UIScene lifecycle — the trap LifecycleReason.undetermined exists to avoid. A baseline there would stamp a confidently wrong value on every early record; the first real transition fills it correctly a moment later. A test pins the omission so it reads as deliberate.
  • An unstamped bundle claims nothing. BuildInfo.logSessionAttributes is empty rather than reporting the stamp script's unknown placeholder as a configuration — a session that can't name its build should read as unidentified, not as a build called "unknown".
  • Decode tolerance where builds meet. AmbientEvent goes to v2 and LogSession gains a hand-written init(from:), because synthesized decoding throws on a missing key and older rows/journals don't have the new fields. Both have a test that feeds the decoder the old shape.
  • Migration is verified, not assumed. The new columns are optional so SwiftData can infer a lightweight migration. A store written by the pre-change schema (from a git worktree at that commit) was opened under the new one: old rows read back with ambient state honestly absent, and later writes committed with zero write failures.
  • degradeSpanBegan is a test seam for degraded on-disk shapes. Nothing a test can write produces a payload that won't decode or a missing policy column, so the sweep's fallback branch was unreachable from a test without it.

Testing

tuist test Stuff-iOS-Tests passes in full (the multi-bundle scheme, not just the isolated bundles). New coverage spans the snapshot value type, pipeline stamping (including spans, the drop report, and live observers), journal round-trip and crash recovery, store dedupe / reads / orphan pruning, session attribute round-tripping, the build-scope filter, the span outcome model under unreadable payloads, and the relaunch column plus its payload fallback.

Open in Web Open in Cursor 

kyleve added 11 commits July 28, 2026 15:02
Closes the first step of the ambient-state-snapshot P0: the snapshot needs
to know which events describe a lasting condition (fold them in) and which
describe an instant (don't). `AmbientEvent.reporting` draws that line, and
a memory warning becomes the one built-in `.occurrence` — the app is not
"in a memory warning" afterwards, so it must not stick to every later
record.

`eventVersion` goes to 2. Decoding is hand-written for one load-bearing
reason: v1 rows have no `reporting` key at all and synthesized decoding
throws on a missing key rather than defaulting. Every v1 ambient event was
a state change, so absence decodes as `.state`, covered by a test that
feeds the decoder the v1 shape.
Pure value-type groundwork for the P0: no pipeline is wired to it yet.

A snapshot is the latest value of every stateful ambient kind, and its
identity is the dedupe key — `applying(_:)` returns `self` whenever nothing
actually moved, so a run of records sharing one system state will share one
stored row instead of one row per record.

`folding(_:into:)` handles the "nothing observed yet" case so an empty
snapshot is unrepresentable: a row claiming to describe the system while
carrying no values would be worse than an honest absence.

`AmbientKind: CodingKeyRepresentable` keeps the values dictionary encoding
as a JSON object keyed by kind rather than a flat alternating array. It is
deliberately not `RawRepresentable`, which could change how a kind encodes
inside an `AmbientEvent` payload and invalidate stored rows.
The point of the P0: any event — not just the ambient ones — can now be
joined to what the system was doing when it happened, the same way it is
already joined to its session.

Folding and stamping both happen inside the existing in-lock buffer path,
which returns the record *as buffered* so the journal and the sinks receive
the stamped copy rather than the pre-stamp original. Folding runs before
stamping, so an ambient event carries the state it announces instead of the
one it replaced.

The drop report needs its own stamp: it is synthesized during the drain and
never passes through the buffer path, so it would otherwise be the one
record that couldn't say what the system was doing when the gap opened.
A record whose only copy is the journal should still say what the system
was doing when it was emitted, so `LogJournalRecord` carries the stamped
snapshot and an end-to-end test asserts it survives emit → disk → recover.

The field is `Optional` on purpose, and a test pins that: journals are
written before an upgrade and ingested after one, so an entry from a build
that predates ambient state has no `ambient` key at all. Optional means
synthesized decoding tolerates that instead of throwing away the journal.

Ingest maps the snapshot onto its row in the next commit, with the column
to put it in.
Completes the durable half of the P0. Events reference their ambient state
the way they already reference their session: a `SDAmbientSnapshot` row per
distinct state plus an indexed `ambientSnapshotID` on the event, so a run of
events that shared one system state costs one row rather than one copy each.
Snapshot values are a dictionary attribute rather than a JSON blob, so
reading a snapshot back has no decode step and therefore no failure mode to
swallow. Journal ingest maps recovered records onto their rows too.

Retention takes unreferenced snapshots with it — one row per distinct state
means they otherwise accumulate for as long as the app keeps changing
network, thermal, or power state.

Both new columns are optional so SwiftData can infer the migration, and that
is verified rather than assumed: a store written by the previous commit's
schema (from a worktree at that commit) opens under the new one, its rows
read back with ambient state honestly absent, and subsequent writes with
snapshots commit with zero write failures.

`LogRecord.stamped(ambient:)` is a DEBUG `@_spi(Testing)` seam: store tests
hand records to `write(_:)` directly, with dates they choose, so they can't
obtain a stamped record from a live pipeline. One test does go end to end
through a real pipeline, and another through crash-journal recovery.
Without a baseline the snapshot has nothing to say about either until the
first transition — and a device that launches hot and stays hot, or a session
that runs entirely in Low Power Mode, never posts one at all. Both read
nonisolated `ProcessInfo` state, so `started()` needs no actor hop, matching
how the accessibility source already emits its summary.

The app-lifecycle source deliberately gets no baseline, and a test now pins
that: reading `UIApplication.applicationState` at launch reports `.background`
even for a user tap under the UIScene lifecycle — the trap
`LifecycleReason.undetermined` exists to avoid — so a baseline would stamp a
confidently wrong value onto every early record of a normal launch. The first
real transition fills it in honestly.
Stamping state onto records only pays off if a developer can see it. The
event detail view resolves the snapshot on demand — one row serves many
events, so queries don't join it per row — and the export resolves the whole
set once and embeds each event's state as a JSON object.

A referenced snapshot that isn't found is reported rather than omitted, in
both surfaces: retention only drops *unreferenced* snapshots, so a missing
one is a real inconsistency and must not read as "nothing was known about
the system".
The version and build number can't identify a build when the manifest
pins both, so every developer build's sessions read `v1.0 (1)` and
week-old events can't be tied to the code that produced them. A recorded
duration has the same problem from the other side: without the
optimization level there's no telling a measurement that says something
about the shipping app from one taken off an `-Onone` build.

`LogSession` grows an `attributes` dictionary keyed by a typed
`LogSessionAttributeKey`, with well-known keys for the commit, its
clean/dirty status, the configuration, the optimization level, and the
compilation mode. PeriscopeCore can't reach WhereCore, so it's a seam the
host app fills rather than something Periscope reads: the stamp script
now writes `CONFIGURATION`, `SWIFT_OPTIMIZATION_LEVEL`, and
`SWIFT_COMPILATION_MODE` into the app's Info.plist beside the commit
keys, `BuildInfo` reads them back as a `Compilation` value, and
`bootstrapLogging` passes `BuildInfo.logSessionAttributes` to
`LogSession.current`. The viewer's session picker names the commit and
optimization level instead of the pinned version pair.

An unstamped bundle claims nothing rather than claiming it was built from
a commit named `unknown`, so a session that can't identify its build
reads as unidentified. `LogSession` decodes a payload without
`attributes` as an empty set — the crash journal ingests entries written
by earlier app versions — and `SDLogSession.attributes` is optional so
existing rows take SwiftData's lightweight migration.

Closes the "a LogSession can't name the build it came from" P2 in the
root TODOs.
A p99 pooled half from an unoptimized developer build and half from an
optimized one describes neither, and nothing in the number tells the
reader it happened. Now that a session names its optimization level, the
history can separate them.

`SpanHistoryView` gains a build scope — all builds, this session, or every
session built at the same optimization level — resolved against the
sessions the store actually recorded, so a store whose sessions never
named a level doesn't offer to group by one. The list header names what
the percentiles cover, and the empty state distinguishes "nothing
recorded" from "nothing in this scope".

Re-scoping filters the ends already accumulated instead of refetching, so
narrowing stays as live and as cheap as the unscoped reading. `all` stays
the default: a reader who hasn't chosen sees every run, and narrowing is
an explicit act. An unresolvable scope admits no sessions rather than
widening back to every build, so a label can't say one thing while the
data says another.

`PeriscopeStore.currentSession` is the new seam this reads — the session
writes are attributed to, `nil` before one exists rather than
synthesizing one a later write would adopt.
Closes the "span-tooling quick wins" plan step.

Three things the span surfaces got wrong, all of them silent:

- Every read model swallowed its failures. A store read that threw, or a
  payload that wouldn't decode, set `.failed` (or quietly grouped a row by
  its message) with nothing written anywhere a developer could see it.
  They now report to `PeriscopeToolsLog.failures`, an OSLog channel rather
  than Periscope itself: these surfaces reload on every store commit, so
  logging into the store they read would turn one corrupt row into a
  refresh loop.

- `SpanNode` carried parallel `ended` / `exitMode` / `duration` optionals,
  so a span whose end payload failed to decode rendered an exit chip
  beside a "running" duration — a reading that cannot be true, since the
  exit comes from an indexed column and the duration from the payload.
  One `Outcome` replaces them, and its `Timing` tells an end that
  measured nothing apart from one whose payload wouldn't decode.

- `SpanTreeRow` read `stylesheet.row.comfortable` directly, ignoring the
  density it was handed.

The orphan sweep also stops parsing payloads to decide relaunches:
`SpanRelaunchPolicy` is persisted as a column on the began row, and the
sweep filters survivors from it. A row written before the column falls
back to its payload, and a payload that can't be read can't prove the
span asked to survive — so it's closed, with the failure logged rather
than silently deciding against the policy. `degradeSpanBegan` is the test
seam for that pair of degraded shapes; nothing a test can write produces
them.
Closes the "PeriscopeCore/PeriscopeTools docs" plan step.

The invariants an agent can't re-derive from the code: that an ambient
event declares whether it's a state or an occurrence (and why folding an
occurrence in would make every later record claim the app was mid-memory-
warning); that a snapshot's identity only changes when a value moves,
which is what makes "one row per distinct state" true; that a session's
build attributes come from the app because Periscope sits below it; that
the relaunch sweep reads a column and logs when it can't; that the tools
log their own failures to OSLog rather than into the store they read; and
that a timing reading has to name the builds it pools.

Also fixes a doc claim that was already wrong: the crash-durability
section read as though the journal covered the whole process lifetime. It
doesn't — it opens with the store, and `PeriscopeStore.make` is `async`.
That was its own backlog item.

Closes eight backlog entries across the Periscope and root TODOs, and
adds the two missing `*+Display` test files that another one asked for.
@cursor cursor Bot changed the title Ambient state snapshots: join any event to what the system was doing Ambient state snapshots, build-attributed sessions, and span tooling that can't lie Jul 28, 2026
Preserve build-attributed Periscope sessions while adopting scoped log-store routing and demo mode from main.

Co-authored-by: Cursor <cursoragent@cursor.com>
/// `CodingKeyRepresentable` so dictionaries keyed by it encode as JSON
/// objects instead of the flat alternating key/value array a dictionary with
/// non-string keys falls back to.
struct StringCodingKey: CodingKey {

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, what?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems weird!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant