fix(config): map the small tier for the Claude CLI runtime - #557
Merged
Conversation
An agent declaring model_tier: small was unrunnable on anthropic + claude_cli, and did not degrade: the run aborted at plan time after the selection phase had already been paid for, and every reviewer that did resolve never ran. Agents travel between runtimes through shared catalogs, so a tier the other complete runtimes serve should not be a runtime-specific hard failure. Map small to claude-haiku-4-5. The CLI serves it and internal/pricing already prices it, so "unset" was not a design so much as an override every operator had to write identically. A built-in is a floor, not a recommendation — a profile that wants a stronger model for small still says so in llm.model_map, which is where a taste judgment belongs. anthropic_api and pi_rpc keep empty built-in maps by design, so the unmapped error stays reachable and now names the entry that fixes it instead of only the tier that failed. Closes #556
piekstra-dev
approved these changes
Aug 6, 2026
piekstra-dev
left a comment
piekstra-dev
left a comment
Collaborator
There was a problem hiding this comment.
Automated PR Review
Reviewed commit: 707fd414d2f4
Profile: reviewer - Posting as: piekstra-dev
Summary
| Reviewer | Findings |
|---|---|
| go:implementation-tests | 0 |
| policies:conventions | 0 |
Reviewer Coverage
| Reviewer | Status | Inspected | Skipped | Constraints |
|---|---|---|---|---|
| go:implementation-tests | complete_broad | internal/cmd/configcmd/configcmd.go, internal/cmd/configcmd/configcmd_test.go, internal/cmd/initcmd/initcmd_test.go, internal/config/config.go, internal/config/config_test.go, internal/pipeline/pipeline_test.go, internal/stagemodel/resolver.go, internal/stagemodel/resolver_test.go | unavailable | unavailable |
| policies:conventions | complete_broad | README.md, internal/cmd/configcmd/configcmd.go, internal/config/config.go | unavailable | No local or linked cli-common/.github convention docs were present in the provided review context, so this review relied on in-repo consistency (README.md, config.go, configcmd.go) rather than an external policy diff. |
0 PR discussion threads considered. 0 summarized; 0 resolved.
Completed in 2m 54s | $1.46 | claude-sonnet-5 | cr 0.10.268
| Field | Value |
|---|---|
| Model | claude-sonnet-5 |
| Reviewers | go:implementation-tests, policies:conventions |
| Engine | claude_cli · claude-sonnet-5 |
| Reviewed by | cr · piekstra-dev |
| Duration | 2m 54s wall · 3m 50s compute |
| Cost | $1.46 |
| Tokens | 92 in / 15.0k out |
Per-workstream usage
| Workstream | Model | In | Out | Cache read | Cache create | Cost | Duration |
|---|---|---|---|---|---|---|---|
| orchestrator-selection | claude-sonnet-5 | 6 | 2.8k | 48.4k | 15.1k | $0.15 | 32s |
| go:implementation-tests | claude-sonnet-5 | 48 | 7.4k | 883.7k | 43.6k | $0.64 | 1m 59s |
| policies:conventions | claude-sonnet-5 | 32 | 4.5k | 614.8k | 49.0k | $0.55 | 1m 05s |
| orchestrator-rollup | claude-sonnet-5 | 6 | 377 | 63.0k | 17.7k | $0.13 | 12s |
piekstra-dev
approved these changes
Aug 6, 2026
piekstra-dev
left a comment
piekstra-dev
left a comment
Collaborator
There was a problem hiding this comment.
Automated PR Review
Reviewed commit: 707fd414d2f4
Profile: reviewer - Posting as: piekstra-dev
Summary
| Reviewer | Findings |
|---|---|
| go:implementation-tests | 0 |
| policies:conventions | 0 |
| documentation:docs | 0 |
Reviewer Coverage
| Reviewer | Status | Inspected | Skipped | Constraints |
|---|---|---|---|---|
| go:implementation-tests | complete_broad | internal/cmd/configcmd/configcmd.go, internal/cmd/configcmd/configcmd_test.go, internal/cmd/initcmd/initcmd_test.go, internal/config/config.go, internal/config/config_test.go, internal/pipeline/pipeline_test.go, internal/stagemodel/resolver.go, internal/stagemodel/resolver_test.go | unavailable | Git history/diff tooling was unavailable in this environment (git commands were blocked), so review was done by reading the PR-head checkout directly rather than a line-level diff. |
| policies:conventions | complete_broad | README.md, internal/cmd/configcmd/configcmd.go, internal/config/config.go | unavailable | unavailable |
| documentation:docs | complete_broad | README.md | unavailable | unavailable |
0 PR discussion threads considered. 0 summarized; 0 resolved.
Completed in 2m 41s | $2.09 | claude-sonnet-5 | cr 0.10.268
| Field | Value |
|---|---|
| Model | claude-sonnet-5 |
| Reviewers | go:implementation-tests, policies:conventions, documentation:docs |
| Engine | claude_cli · claude-sonnet-5 |
| Reviewed by | cr · piekstra-dev |
| Duration | 2m 41s wall · 5m 46s compute |
| Cost | $2.09 |
| Tokens | 120 in / 20.8k out |
Per-workstream usage
| Workstream | Model | In | Out | Cache read | Cache create | Cost | Duration |
|---|---|---|---|---|---|---|---|
| orchestrator-selection | claude-sonnet-5 | 6 | 2.4k | 23.1k | 41.3k | $0.29 | 28s |
| go:implementation-tests | claude-sonnet-5 | 46 | 8.1k | 784.1k | 53.6k | $0.68 | 1m 51s |
| policies:conventions | claude-sonnet-5 | 26 | 3.9k | 450.0k | 43.9k | $0.46 | 1m 51s |
| documentation:docs | claude-sonnet-5 | 36 | 5.9k | 566.4k | 45.9k | $0.54 | 1m 22s |
| orchestrator-rollup | claude-sonnet-5 | 6 | 414 | 62.4k | 17.5k | $0.13 | 12s |
piekstra-dev
approved these changes
Aug 6, 2026
piekstra-dev
left a comment
piekstra-dev
left a comment
Collaborator
There was a problem hiding this comment.
Automated PR Review
Reviewed commit: 707fd414d2f4
Profile: reviewer - Posting as: piekstra-dev
Summary
| Reviewer | Findings |
|---|---|
| go:implementation-tests | 0 |
| policies:conventions | 0 |
| documentation:docs | 0 |
Reviewer Coverage
| Reviewer | Status | Inspected | Skipped | Constraints |
|---|---|---|---|---|
| go:implementation-tests | complete_broad | internal/cmd/configcmd/configcmd.go, internal/cmd/configcmd/configcmd_test.go, internal/cmd/initcmd/initcmd_test.go, internal/config/config.go, internal/config/config_test.go, internal/pipeline/pipeline_test.go, internal/stagemodel/resolver.go, internal/stagemodel/resolver_test.go | unavailable | unavailable |
| policies:conventions | complete_broad | README.md, internal/cmd/configcmd/configcmd.go, internal/config/config.go | unavailable | unavailable |
| documentation:docs | complete_broad | README.md | unavailable | unavailable |
0 PR discussion threads considered. 0 summarized; 0 resolved.
Completed in 2m 39s | $2.21 | claude-sonnet-5 | cr 0.10.268
| Field | Value |
|---|---|
| Model | claude-sonnet-5 |
| Reviewers | go:implementation-tests, policies:conventions, documentation:docs |
| Engine | claude_cli · claude-sonnet-5 |
| Reviewed by | cr · piekstra-dev |
| Duration | 2m 39s wall · 4m 36s compute |
| Cost | $2.21 |
| Tokens | 126 in / 18.9k out |
Per-workstream usage
| Workstream | Model | In | Out | Cache read | Cache create | Cost | Duration |
|---|---|---|---|---|---|---|---|
| orchestrator-selection | claude-sonnet-5 | 6 | 1.2k | 72.3k | 25.5k | $0.19 | 15s |
| go:implementation-tests | claude-sonnet-5 | 56 | 8.4k | 1.3M | 74.4k | $0.98 | 2m 02s |
| policies:conventions | claude-sonnet-5 | 28 | 4.5k | 509.9k | 45.5k | $0.50 | 1m 01s |
| documentation:docs | claude-sonnet-5 | 30 | 4.4k | 441.1k | 24.2k | $0.34 | 1m 05s |
| orchestrator-rollup | claude-sonnet-5 | 6 | 389 | 83.7k | 28.1k | $0.20 | 11s |
piekstra-dev
previously approved these changes
Aug 7, 2026
piekstra-dev
left a comment
piekstra-dev
left a comment
Collaborator
There was a problem hiding this comment.
Automated PR Review
Reviewed commit: 707fd414d2f4
Profile: reviewer - Posting as: piekstra-dev
Summary
| Reviewer | Findings |
|---|---|
| go:implementation-tests | 0 |
| policies:conventions | 0 |
| documentation:docs | 0 |
Reviewer Coverage
| Reviewer | Status | Inspected | Skipped | Constraints |
|---|---|---|---|---|
| go:implementation-tests | complete_broad | internal/cmd/configcmd/configcmd.go, internal/cmd/configcmd/configcmd_test.go, internal/cmd/initcmd/initcmd_test.go, internal/config/config.go, internal/config/config_test.go, internal/pipeline/pipeline_test.go, internal/stagemodel/resolver.go, internal/stagemodel/resolver_test.go | unavailable | unavailable |
| policies:conventions | complete_broad | README.md, internal/cmd/configcmd/configcmd.go, internal/config/config.go | unavailable | Shared cli-common/docs and .github conventions were not present in the review context (no local sibling checkout and no fetched copy), so this review is limited to repo-local conventions and internal consistency. |
| documentation:docs | complete_broad | README.md | unavailable | unavailable |
0 PR discussion threads considered. 0 summarized; 0 resolved.
Completed in 2m 53s | $2.75 | claude-sonnet-5 | cr 0.10.268
| Field | Value |
|---|---|
| Model | claude-sonnet-5 |
| Reviewers | go:implementation-tests, policies:conventions, documentation:docs |
| Engine | claude_cli · claude-sonnet-5 |
| Reviewed by | cr · piekstra-dev |
| Duration | 2m 53s wall · 5m 06s compute |
| Cost | $2.75 |
| Tokens | 128 in / 21.3k out |
Per-workstream usage
| Workstream | Model | In | Out | Cache read | Cache create | Cost | Duration |
|---|---|---|---|---|---|---|---|
| orchestrator-selection | claude-sonnet-5 | 6 | 1.2k | 85.9k | 47.4k | $0.33 | 16s |
| go:implementation-tests | claude-sonnet-5 | 48 | 9.6k | 913.1k | 55.6k | $0.75 | 2m 10s |
| policies:conventions | claude-sonnet-5 | 32 | 4.9k | 654.0k | 64.7k | $0.66 | 1m 06s |
| documentation:docs | claude-sonnet-5 | 36 | 5.3k | 832.1k | 67.4k | $0.73 | 1m 16s |
| orchestrator-rollup | claude-sonnet-5 | 6 | 414 | 108.7k | 38.8k | $0.27 | 16s |
TestRenderConfigTextExactHomeShape pins cr config show's exact text, which now names the built-in small model instead of <unset>. Missed locally because `make test` runs without -count=1, so this package's pre-change result was served from Go's test cache; CI, which builds clean, caught it in three jobs.
monit-reviewer
approved these changes
Aug 7, 2026
monit-reviewer
left a comment
monit-reviewer
left a comment
There was a problem hiding this comment.
Automated PR Review
Reviewed commit: 38cfdc88de97
Profile: claude-monit-reviewer - Posting as: monit-reviewer
Summary
| Reviewer | Findings |
|---|---|
| go:implementation-tests | 0 |
| policies:conventions | 0 |
Reviewer Coverage
go:implementation-tests— complete (broad); inspected 9 assigned files (10 inspected across reviewers):internal/cmd/configcmd/configcmd.go,internal/cmd/configcmd/configcmd_test.go,internal/cmd/initcmd/initcmd_test.go,internal/config/config.go,internal/config/config_test.go,internal/pipeline/pipeline_test.go,internal/stagemodel/resolver.go,internal/stagemodel/resolver_test.go,internal/view/config_test.go; skipped: none; constraints: nonepolicies:conventions— complete (broad); inspected 5 assigned files (10 inspected across reviewers):README.md,internal/cmd/configcmd/configcmd.go,internal/cmd/configcmd/configcmd_test.go,internal/config/config.go,internal/stagemodel/resolver.go; skipped: none; constraints: none
Inspected files (10)
README.mdinternal/cmd/configcmd/configcmd.gointernal/cmd/configcmd/configcmd_test.gointernal/cmd/initcmd/initcmd_test.gointernal/config/config.gointernal/config/config_test.gointernal/pipeline/pipeline_test.gointernal/stagemodel/resolver.gointernal/stagemodel/resolver_test.gointernal/view/config_test.go
0 PR discussion threads considered. 0 summarized; 0 resolved.
Completed in 3m 52s | ~$1.57 (est.) | claude-sonnet-5 | cr 0.10.281
| Field | Value |
|---|---|
| Model | claude-sonnet-5 |
| Reviewers | go:implementation-tests, policies:conventions |
| Engine | claude_cli · claude-sonnet-5 |
| Reviewed by | cr · monit-reviewer |
| Duration | 3m 52s wall · 3m 45s compute |
| Cost | ~$1.57 (est.) |
| Tokens | 34 in / 7.5k out |
Per-workstream usage
orchestrator-selection— claude-sonnet-5- In: 6
- Out: 3.4k
- Cache read: 92.2k
- Cache create: 78.0k
- Cost: ~$0.37 (est.)
- Duration: 43s
go:implementation-tests— claude-sonnet-5- In: 12
- Out: 2.2k
- Cache read: 314.9k
- Cache create: 79.8k
- Cost: ~$0.43 (est.)
- Duration: 2m 21s
policies:conventions— claude-sonnet-5- In: 10
- Out: 1.5k
- Cache read: 234.0k
- Cache create: 75.2k
- Cost: ~$0.37 (est.)
- Duration: 28s
orchestrator-rollup— claude-sonnet-5- In: 6
- Out: 384
- Cache read: 97.9k
- Cache create: 95.3k
- Cost: ~$0.39 (est.)
- Duration: 12s
piekstra
marked this pull request as ready for review
August 7, 2026 11:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #556
The failure
An agent declaring
model_tier: smallis unrunnable onanthropic+claude_cli, and it does not degrade — the run aborts at plan time:The selection phase had already run and been paid for; the reviewers that did resolve never ran. Hit live: a shared agent catalog gained a
small-tier reviewer, and every review on the Claude CLI runtime failed identically until an operator wrote amodel_mapoverride.The change
Map
smalltoclaude-haiku-4-5forclaude_cli. The README previously documented the tier asunset, so this reverses a documented choice deliberately, on three grounds:codex_cli,openai_api). Agents travel between runtimes through shared catalogs, so a tier those runtimes serve should not be a runtime-specific hard failure.internal/pricingalready prices it, sounsetwas less a design than an override every operator had to write identically.smallstill says so inllm.model_map— that is a taste judgment, and it belongs in config, not in the tool's defaults. (The machine that hit this mapssmall: claude-sonnet-5for exactly that reason.)anthropic_apiandpi_rpckeep empty built-in maps by design, so the unmapped-tier error stays reachable — and it now names the entry that fixes it (add llm.model_map.small to the profile's LLM runtime) rather than only the tier that failed. Same wording incr config llm models resolve.Not doing: fallback to the nearest mapped tier
#556 raised this as a secondary option and I am deliberately not taking it. With the built-in map complete, a partial map is now always operator-authored — and silently substituting a model an operator chose not to map is worse than a clear error that names the fix. The tiers stay honest:
smallmeans a small model actually ran.Test churn
Five tests pinned the previous
unsetcontract, and each was updated to the behavior it was really guarding rather than deleted:TestBuiltInModelMapIsProviderAdapterSpecific/TestConfigLLMModelsListAndResolve— expected values.TestInitProfileV2LLMRuntimeSelectionRefreshesModelMapFields/TestInitProfileV2ModelMapInputsDraftOverridesAndClears— the wizard now prefills the small field with the built-in, so the "type an override" flow clears it first, exactly like medium and large.TestResolveFirstAvailableUsesFirstConfiguredTierandTestDryRunFastPreflightResolvesEveryReviewerBeforeLLMused claude_cli's unmapped small as their vehicle for "tier is missing". They now use a runtime with no built-in map plus an explicit partialmodel_map, so both still test fallback and preflight against a genuinely unmapped tier.New:
TestResolveStageModelMapsSmallTierForClaudeCLIpins the mapping, and the unmapped-tier test now also asserts the error namesllm.model_map.small.make lintclean.make testpasses exceptTestPiRPCReviewerExtensionLoadsInInstalledPi, which fails identically onmainin this environment (the locally installedpirejects--no-builtin-tools/--no-approve).