fix: enforce dispatch routing to the matched specialist (0.50.0) - #61
Merged
Conversation
Sending a task to general-purpose when a tp-* specialist covers it is the
most expensive mistake available in a session. Same question, measured:
57,921 tokens through general-purpose against 18,677 through tp-run —
3.1x, ~39k wasted per launch, most of it startup cost a general agent
pays before doing any work.
The matcher was never the problem; it named the right agent on five of
seven real descriptions. The suggestion just never landed. It went out as
permissionDecision=allow with the advice in the reason, and an allow is
something the model may ignore — it did, on ten of eleven dispatches in a
measured session.
Reads are disciplined because a wasteful read comes back denied.
Dispatches were not, because they came back allowed. Deny mode (the
default) now blocks a high-confidence match and names the agent to use.
The dead `&& ctx.force` clause that made the deny tier unreachable is
gone with it.
Matching now reads the prompt as well as the description. Descriptions
run two or three words ("Reuse check") and score low; the prompt carries
the task and takes the same dispatch to high. Without it the blocking
tier would almost never fire. containsEscape reads the prompt too, so an
escape written there still works now that the prompt can trigger a block.
Low-confidence matches still only advise, escapes still pass,
TOKEN_PILOT_MODE=advisory still disables blocking, tp-* dispatches are
untouched. The v0.31.0 test asserting the old advise-only behaviour is
updated deliberately, not deleted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Dispatch routing now blocks a high-confidence match instead of quietly suggesting it. Sending work to
general-purposewhen a tp-* specialist covers it is the most expensive mistake available in a session, and until now nothing stopped it.Why
Measured on the same question, twice:
general-purposetp-run3.1x — about 39k wasted per launch, most of it startup cost a general agent pays before doing any work.
The matcher was never broken. It named the correct specialist on five of seven real descriptions. The suggestion just never landed: it went out as
permissionDecision=allowwith the advice in the reason, and an allow is something the model may ignore. Over one measured session it did exactly that — ten of eleven dispatches went togeneral-purposewhile the matcher was naming a specialist every time.Reads are disciplined because a wasteful read comes back denied. Dispatches were not, because they came back allowed. That is the whole gap.
How
&& ctx.forceclause that made this tier unreachable goes with it —A || B || (C && D && A)is justA || B.promptas well as thedescription. Real descriptions are two or three words ("Reuse check") and score low; the prompt carries the task and takes the same dispatch to high. Without this the blocking tier would almost never fire.containsEscapereads the prompt too, so an escape written there still works now that the prompt can trigger a block.Risk
This changes default behaviour: some dispatches that used to pass will now be refused with instructions. That is the intent, and the blast radius is bounded on four sides — low-confidence matches still only advise, escape phrases still pass,
TOKEN_PILOT_MODE=advisorydisables blocking entirely, and tp-* dispatches are never touched.Verified on a built dist:
general-purpose+ "review these changes for duplication" comes back denied namingtp-pr-reviewer; the same call with "ad-hoc" comes back allowed. Full suite 1445 green.Notes for reviewer
The v0.31.0 test asserting the old advise-only behaviour is updated, not deleted — the old expectation was deliberate, so the change is deliberate too, and the test now records why.
Worth a second opinion: the confidence bar for blocking is score ≥ 3 or a quoted trigger phrase. Set it lower and legitimate dispatches start getting refused; set it higher and this stops firing again. Three is what the measured descriptions clustered around, but it is a judgement call, not a derived number.