Skip to content

Fix Codex model attribution for large turn contexts#1012

Closed
BarryYangi wants to merge 1 commit into
steipete:mainfrom
BarryYangi:codex/codex-subagent-large-turn-context-model
Closed

Fix Codex model attribution for large turn contexts#1012
BarryYangi wants to merge 1 commit into
steipete:mainfrom
BarryYangi:codex/codex-subagent-large-turn-context-model

Conversation

@BarryYangi
Copy link
Copy Markdown

Summary

  • Keep Codex turn_context lines up to the existing 256 KiB scanner limit instead of truncating them at 32 KiB.
  • Add a regression test for a large Codex turn_context whose model metadata must be used for subsequent token rows.

Root cause

Codex subagent sessions can write large turn_context JSONL lines because the payload includes role instructions, skill context, and other runtime metadata. The Codex cost scanner already allows lines up to 256 KiB, but it only preserved a 32 KiB prefix before parsing.

When a turn_context line is larger than 32 KiB, CostUsageJsonl.scan marks it as truncated and the scanner skips the line entirely. If later token_count rows do not carry their own model metadata, currentModel is never populated and the row falls back to gpt-5.

Local reproduction context

I reproduced this with CodexBar 0.26.1 / build 63 on macOS using local Codex Desktop session logs.

Observed environment/details:

  • CodexBar CLI: CodexBar 0.26.1
  • Codex session source: Codex Desktop
  • Codex CLI recorded in session metadata: 0.131.0-alpha.9
  • Affected rows were subagent sessions under ~/.codex/sessions/2026/05/15/
  • The affected subagent turn_context.payload.model was gpt-5.5
  • The following token_count rows had no info.model / info.model_name
  • One representative turn_context line was 34,789 bytes, just over the scanner's old 32 KiB prefix limit

After clearing CodexBar's cost cache and refreshing with 0.26.1, the daily cost cache still bucketed the affected 2026-05-15 subagent usage under gpt-5:

"2026-05-15": {
  "gpt-5.5": [369490655, 354968960, 1078961],
  "gpt-5": [9847428, 8968960, 98161]
}

The gpt-5 bucket above came from seven subagent session files whose source turn_context.payload.model was gpt-5.5. This changes the estimated cost because the affected tokens are priced as gpt-5 instead of gpt-5.5.

Fix

Use the existing maxLineBytes value as the preserved prefix for Codex scanner lines. This keeps the current 256 KiB guardrail while allowing large-but-valid turn_context rows to be parsed and used for model attribution.

Validation

  • swift test --filter CostUsageScannerPriorityTests
  • swift test --filter CostUsageScanner

@BarryYangi BarryYangi marked this pull request as ready for review May 17, 2026 15:52
@BarryYangi
Copy link
Copy Markdown
Author

BarryYangi commented May 17, 2026

Additional reproduction note: this was reproduced with CodexBar 0.26.1 / build 63 after clearing the Codex cost cache and refreshing. The affected Codex Desktop subagent logs had turn_context.payload.model = gpt-5.5, but the turn_context JSONL line was about 34.8 KiB, so the old 32 KiB prefix caused the scanner to skip it and fall back to gpt-5 for later model-less token_count rows.

@steipete
Copy link
Copy Markdown
Owner

Thanks for the clear repro here. This is the same underlying Codex turn_context attribution bug as #1014/#1013, and #1014 has now landed as 036b497.

That fix keeps the retained JSONL prefix for oversized rows, recovers the model from long turn_context rows including nested payload.info.model, ignores JSON-looking prompt examples, and bumps the Codex local cost cache to codex-v8.json so stale misattributed caches rebuild.

Closing this PR as superseded by #1014 rather than merging two overlapping branches. Appreciate the detailed reproduction notes.

@steipete steipete closed this May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants