Skip to content

parser: slice midnight-straddling turns instead of dropping them - #857

Draft
avs-io wants to merge 1 commit into
getagentseal:mainfrom
avs-io:fix/turn-range-filter-midnight
Draft

parser: slice midnight-straddling turns instead of dropping them#857
avs-io wants to merge 1 commit into
getagentseal:mainfrom
avs-io:fix/turn-range-filter-midnight

Conversation

@avs-io

@avs-io avs-io commented Jul 29, 2026

Copy link
Copy Markdown
Member

Overlaps #853, which came first. @KENSHI601 opened #853 on 2026-07-28 fixing this defect in the two parseProviderSources loops, and the diagnosis and repro here are theirs.

This branch is a superset of #853: it carries the same two-loop fix plus the Claude Code path and the day-aggregator reconciliation described below. Either this lands and #853 closes as included, or #853 lands and the delta comes as a follow-up. Maintainer's choice — flagging the overlap rather than quietly duplicating it.

Relates to #852.

The defect

Both parseProviderSources (Codex, OTel, network providers) and scanProjectDirs (Claude Code) range-filtered a whole turn by its first call's timestamp. A turn spanning local midnight — common for long autonomous sessions — had every call after the boundary silently dropped from the new day: codeburn today reported no usage until the turn ended, and daily history attributed the whole turn's cost to the turn-start day.

Delta over #853

#853 fixes the two parseProviderSources loops. This branch additionally carries:

1. The Claude Code path. scanProjectDirs has the structurally identical whole-turn drop, filtering on turn.assistantCalls[0].timestamp. Claude Code is the highest-traffic provider in this tool, so the reported symptom persists for most users until this path is fixed too. Handled by classifiedTurnSlicedToRange, which slices assistantCalls and leaves category/subCategory/retries/hasEdits as classified from the complete turn — they are computed once ahead of the date filter, for carriedBranch/PR-ref tracking reasons already in the code, and are turn-level judgments rather than per-call sums.

2. A timestamp re-anchor on a sliced turn. day-aggregator.ts buckets by turn.timestamp rather than per-call timestamps — deliberately, per ae18b1f. Without re-anchoring the sliced turn to its first surviving call, the retained post-midnight calls are still filed under the pre-slice day, so they reappear in parseAllSessions output but remain invisible in codeburn today, which is the reported symptom. Verified against getDateRange('today') and aggregateProjectsIntoDays under a simulated UTC+8 clock.

Fold-anchor safety

The PR-linked subagent fold anchor — a parent whose own turns fall entirely out of range is kept as a zero-cost anchor so an in-range child can still attribute — is unchanged. carriedBranch, carriedPrRefs and spawnPrSets are built from cachedFile.turns, the full pre-slice list, before the date filter runs. A slice yields a non-empty call list only when the turn genuinely has an in-range call, so a fully out-of-range turn still filters to nothing. The anchor test passes identically before and after the change, which is the correct obligation: the mechanism was not broken and must not break.

Testing

Two regression files built from @KENSHI601's synthetic repro, made deterministic across machine timezones rather than dependent on the runner's local zone: tests/parser-turn-midnight-range.test.ts (provider path) and tests/parser-claude-turn-midnight-range.test.ts (Claude Code path).

Each covers: a turn spanning the boundary attributes post-boundary calls to the new day and pre-boundary calls to the prior day; a same-day control turn is unchanged; a turn entirely outside the range is still excluded; the unfiltered monthly/total aggregate does not double-count; and aggregateProjectsIntoDays with getDateRange('today') reconciles under a simulated UTC+8 clock. The Claude Code file adds the fold-anchor case.

All confirmed failing before the change. Full suite 2475 passed, tsc --noEmit clean.

Out of scope

  • The issue's optional per-call attribution in aggregateProjectsIntoDays (dateKey(call.timestamp) instead of the turn day) is deliberately not included. Once turns are sliced per range, a day-scoped query already sees only that day's calls. Reintroducing raw per-call bucketing on a wide unsliced parse — the common path for history and menubar rollups — would resurrect the reconciliation bug ae18b1f fixed, where a midnight-straddling turn split across two days and history.daily stopped reconciling to the live headline.
  • filterProjectsByDateRange / turnIsInDateRange, the dashboard's in-memory re-narrowing over an already-wide parse, has the same shape but operates on turns already merged into ProjectSummary. Not covered here; a candidate follow-up.

Credit

Root-cause diagnosis, the synthetic repro, and the fix shape are @KENSHI601's, in issue #852 and PR #853.

@avs-io avs-io changed the title parser: keep in-range calls of a midnight-straddling turn instead of dropping it parser: extend the midnight turn-slice fix to the Claude Code path (delta over #853) Jul 29, 2026
…d of dropping it

Both parseProviderSources (Codex/OTel/network providers) and scanProjectDirs
(Claude Code, the highest-traffic provider) range-filtered a whole turn by
its first call's timestamp. A turn spanning local midnight — common for
long-running autonomous sessions — had every call after the boundary
silently dropped from the new day: `codeburn today` reported no usage until
the turn ended, and daily history attributed the whole turn's cost to the
turn-start day.

Slice each turn to only its in-range calls instead of dropping it wholesale,
and re-anchor `timestamp` to the first surviving call so day-aggregator's
turn-anchored day bucketing (which keys off `timestamp`, not per-call
timestamps) attributes the retained calls to the day they actually occurred,
not the pre-slice turn's original day.

- parseProviderSources: turnSlicedToRange slices a CachedTurn's raw calls,
  applied in both the servedSources and durableSources loops (both had the
  identical bug).
- scanProjectDirs: classifiedTurnSlicedToRange slices a ClassifiedTurn's
  assistantCalls post-classification. category/subCategory/retries/hasEdits
  stay exactly as classified from the complete turn (turn-level judgments,
  not per-call sums) — only assistantCalls and timestamp are trimmed. The
  PR-linked subagent fold-anchor mechanism (a parent whose own turns are
  fully out of range still folds an in-range child with zero parent spend)
  is untouched: it depends on carriedBranch/spawnPrSets built from the FULL
  pre-slice turn list, and a slice only ever produces a non-empty
  assistantCalls when the turn genuinely has in-range calls, so it can't
  turn a real anchor into a spend-bearing turn or vice versa.
- Both slicers share a small `callsInRange` predicate; the two composite
  objects (CachedTurn vs ClassifiedTurn) are built separately since their
  shapes genuinely differ.

Fixes getagentseal#852. Repro, root-cause diagnosis, and the initial fix shape (filter
calls inside the turn rather than dropping it, applied to both provider
loops) are from the issue reporter (KENSHI601), verified against the current
parser and extended to the Claude Code path and the day-aggregator
timestamp reconciliation.
@iamtoruk
iamtoruk force-pushed the fix/turn-range-filter-midnight branch from dd110d9 to 1643aa9 Compare July 30, 2026 00:39

@iamtoruk iamtoruk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The today fix is genuine, but as it stands this trades a visible bug for a silent one: periods longer than a day lose usage. Requesting changes on a proven conservation violation.

The blocker, reproduced end-to-end through buildDurablePeriod (the shared headline builder behind CLI/TUI/menubar): turn starting 07-27 23:57 with calls at 23:58 ($0.006) and 00:10 ($0.012). Truth for week/month = 2 calls / $0.018.

period main this PR
today 0 / $0 (the bug you're fixing) 1 / $0.012 ✓
week 2 / $0.018 ✓ 1 / $0.006
month 2 / $0.018 ✓ 1 / $0.006

Mechanism: the backfill parse now slices the turn at the backfillStart..yesterdayEnd boundary, so the cached yesterday keeps only pre-midnight calls — but aggregateProjectsIntoDays stays turn-anchored, so in the period parse the whole turn buckets under yesterday and the today bucket gets zero. unionDaysForPeriod = cache (≤ yesterday) + live (today): the post-midnight calls are in neither. Per-call slicing and per-turn day bucketing are incompatible — the "out of scope" note on aggregateProjectsIntoDays is exactly where the gap opens. On a real month of data: 12 straddling turns, $8.98 silently gone.

Second blocker — half-applied across surfaces: filterProjectsByDateRange (dashboard Today/7-Days) and filterProjectsByDays (menubar/history) still whole-turn-drop, so the same data yields different Today numbers per surface — the exact divergence class the durable-totals parity suite exists to prevent (which has no midnight-straddling case; one should land with whichever PR carries this fix).

Also worth stating: turn-level aggregates (editTurns, categories, workflow) now report on BOTH sides of a split — a turn whose only Edit happened entirely on day N reports editTurns=1 on day N+1 too. If that's the intended semantics it needs documenting; summed-across-days turn counts inflate.

Given your own note that #853 should take precedence: agreed — suggest closing this in favor of landing the conservation-correct version once, there. Two things to carry over: (a) #853 currently does NOT re-anchor timestamp after slicing, so the two implementations diverge on the split rule — whichever survives should pin down one rule; (b) #853 doesn't touch scanProjectDirs, so its exposure at the cache-backfill boundary is structurally the same question and needs the same regression test (a straddling-turn buildDurablePeriod case asserting day-N + day-N+1 == whole-range for calls, cost, and tokens — I have a failing test ready to contribute).

Side-finding while auditing: parseSessionFile (~:1832-1846) is dead code carrying a fourth copy of the old filter and the historically-important "orphan assistant calls" comment — worth a separate cleanup so the next person doesn't have to reconcile four filter sites.

@avs-io avs-io changed the title parser: extend the midnight turn-slice fix to the Claude Code path (delta over #853) parser: slice midnight-straddling turns (superset of #853: adds Claude Code path) Jul 30, 2026
@avs-io avs-io changed the title parser: slice midnight-straddling turns (superset of #853: adds Claude Code path) parser: slice midnight-straddling turns instead of dropping them Jul 30, 2026
KENSHI601 added a commit to KENSHI601/codeburn that referenced this pull request Jul 30, 2026
Extends the per-call range filter so multi-day periods stop losing
usage (review on getagentseal#857):

- Re-anchor a sliced turn's timestamp to its first surviving call in
  parseProviderSources, so every turn slicer shares one split rule.
- scanProjectDirs (Claude Code path): slice per call instead of
  dropping the whole turn on its first assistant timestamp; category/
  subCategory/retries/hasEdits stay classified from the full turn.
- aggregateProjectsIntoDays: bucket cost/calls/tokens (and the model,
  project, provider-slice rollups built from them) under each call's
  own day; turn-level stats (categories, editTurns, oneShotTurns) stay
  turn-anchored. This is the conservation fix: cache (<= yesterday) +
  live (today) unions now sum to the whole range for straddling turns.
- buildJsonReport's dailyMap fallback follows the same per-call rule
  so the no-durable path can't diverge from durable.days.
- filterProjectsByDateRange (dashboard) and filterProjectsByDays
  (menubar/history) slice per call instead of dropping whole turns.

Adds the straddling-turn buildDurablePeriod case to the durable-totals
parity suite (day-N + day-N+1 == whole-range calls/cost/tokens,
verified to fail without the fix), covers the today view and the
surface filters, and makes the suite hermetic on machines with real
provider data.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants