feat(act): measure realized savings for defer-* actions in act report - #862
Open
AVSRPA1KR wants to merge 1 commit into
Open
feat(act): measure realized savings for defer-* actions in act report#862AVSRPA1KR wants to merge 1 commit into
AVSRPA1KR wants to merge 1 commit into
Conversation
The defer-enable / defer-alwaysload / defer-threshold plan kinds (part 2 of the deferral-coverage work) applied and undid correctly but were invisible to `act report` — it returned "not measurable: no baseline captured at apply time", because report.ts never captured a baseline for them or knew how to compute their realized delta. Wire them in, mirroring the mcp-remove path since deferral has the same effect (MCP tool-def schema leaves the upfront prefix): - needsConfigBaseline + captureBaseline now cover the defer-* kinds. Servers are the named set for defer-alwaysload, or the observed MCP surface for defer-enable / defer-threshold (which re-enable deferral across everything). Per-session tokens use observed tool counts, or the 5-tools x 400 fallback, exactly like mcp-remove. - A new deferRow computes realized savings as per-session prefix tokens times the post-apply sessions where deferral actually became active — detected by the same deferred-tools-inventory signal the mcp-deferral-off detector uses. Sessions begun before the client restarted still run deferral-off and are excluded; if none benefited, the row reports "not yet in effect" with zero realized rather than claiming a saving that has not taken hold. Records applied before this change (no baseline) keep the existing "no baseline captured at apply time" note, so nothing regresses. 11 tests cover the measured, partial, not-yet-in-effect, no-sessions, empty-baseline, and missing-baseline paths, plus baseline capture for each kind. Verified live end to end: apply captures the baseline, and act report reports realized savings scaled to the sessions that adopted deferral.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #631 (part 2 of the deferral-coverage work): make the
defer-*actions measurable inact report.The gap
The
defer-enable/defer-alwaysload/defer-thresholdplan kinds from #631 apply, journal, and undo correctly, butact reportcouldn't measure them — it returned "not measurable: no baseline captured at apply time", becausereport.tsnever captured a baseline for these kinds nor knew how to compute their realized delta. The measurement piece the design called for was left unwired.The fix
Deferral has the same effect as
mcp-remove— MCP tool-def schema leaves the upfront prefix — so this mirrors that path, with one inversion in the realized signal:needsConfigBaseline+captureBaselinenow cover thedefer-*kinds. Servers are the named set fordefer-alwaysload, or the observed MCP surface fordefer-enable/defer-threshold(which re-enable deferral across everything). Per-session tokens use observed tool counts, or the 5-tools × 400 fallback, exactly likemcp-remove.deferRowcomputes realized savings as per-session prefix tokens × the post-apply sessions where deferral actually became active — detected by the same deferred-tools-inventory signal themcp-deferral-offdetector uses.ENABLE_TOOL_SEARCHis read at process start, so sessions begun before the client restarted still run deferral-off and are excluded; if none benefited, the row reports "not yet in effect" with zero realized, rather than claiming a saving that hasn't taken hold.Records applied before this change (no baseline) keep the existing "no baseline captured at apply time" note, so nothing regresses.
Tests
11 new tests cover measured, partial, not-yet-in-effect, no-sessions, empty-baseline, and missing-baseline paths, plus baseline capture for each kind. Verified end to end locally: applying a
defer-enablecaptures the baseline, andact reportreports realized savings scaled to the sessions that adopted deferral (and honestly reports "not yet in effect" when none have).Diff is limited to
src/act/report.ts(+80) andtests/act-report.test.ts(+140).tsc --noEmitclean; the full act suite passes.