ADR-0022: One model policy for every Anthropic call
Status: Accepted Date: 2026-10-06 Context owner: Saar Related: ADR-0010 Spotter · ADR-0014 workout parse · Spotter behavior · workout-parse behavior
TL;DR
apps/api/src/ai/model-policy.ts names every task that calls Anthropic as a role (MODEL_POLICY). Each role has a model and a call shape: output cap, thinking, effort and tool choice. Call sites send requestParams(MODEL_POLICY.<role>) and never name a model. To change a model, edit that one file. cost.util.ts has the rates, and a spec fails when a model the policy can call has no rate, or when a role asks a model for a call shape the model refuses. The sonnet-5 PostHog flag is gone. Sonnet 4.5 is retired.
Context
- Model ids were set in eleven places:
DEFAULT_MODEL,SONNET_5_MODEL,QUICK_ROUTE_MODEL,PARSE_MODEL,PARSE_DAY_PART_MODEL,PARSE_ESCALATION_MODEL,MOVEMENT_REWRITE_MODEL,MINISITE_TRANSLATION_MODEL, and literal ids in compaction and canonical enrich. The call shape (isSonnet5,extractionCallShape) was worked out from the model string. - The
sonnet-5flag (Spotter loop and dashboard paste on Sonnet 5) was at 100% rollout. That left two code paths and no decision about which model is standard. - Sonnet 5.5 and Opus 5.5 reject a forced
tool_choice. Sonnet 5.5’s “no thinking” setting isthinking: between_tools. Sonnet 5 accepts a forced tool when thinking is disabled. These were probed against the API on 2026-10-05.
Decision
Roles (2026-10-06)
| Role | Model | Call shape |
|---|---|---|
spotterLoop | Sonnet 5.5 | adaptive thinking, effort low, 16,384 cap, tool_choice per turn |
spotterQuickRoute / spotterQuickAnswer / spotterTitle / spotterCompaction | Haiku 4.5 | any tool for the router; plain text for the others |
parsePaste (whole text) | Sonnet 5 | thinking disabled, forced tool |
parseDayPart | Sonnet 5.5 | between_tools, auto tool |
parseEscalation | Sonnet 5.5 | adaptive thinking, effort medium, auto tool |
parseMovementRewrite / parseResolutionConfirmation / exerciseAliases | Haiku 4.5 | forced tool / plain text |
exerciseVideoScoring / exerciseCues / minisiteTranslation | Sonnet 5.5 | between_tools, auto tool, system line “Respond only by calling the X tool.” (callToolOnly) |
Evidence (live eval, 2026-10-05)
| Task | Result |
|---|---|
| Spotter loop: 21 scenarios × 2 runs, tool-call checks | Sonnet 5.5 at low effort: 95% pass, $0.0178/turn, TTFT p50 1.1s. Sonnet 5: 90%, $0.0288, 1.9s. Sonnet 5.5 at default effort: 90%, $0.0280, 1.7s |
| Dashboard paste: 15 goldens × 2 runs | Sonnet 5: edit distance 0.392, p50 4.2s. Sonnet 5.5: 0.388, 6.7s, about 10% more expensive. Haiku on the 4 simplest goldens scored the same as Sonnet but needed a validation retry on 3 of 8 runs |
| Day-grain part reads (#516) | Sonnet 5.5 got 44/44 parts right. Sonnet 5 failed 5 parts in 12 days. Haiku 4.5 failed a part on every day |
| Video ranking | Sonnet 5.5 put 18/18 candidates on the right side of the threshold. Sonnet 4.5: 34/36. Haiku 4.5: 30/36 |
| Cues | Haiku 4.5 wrote less expert cues than Sonnet 5.5 |
| Minisite translation (he/ru) | Haiku 4.5 wrote Hebrew imperatives in the masculine singular. Sonnet 5.5 kept the gender-neutral plural the product copy uses |
| Quick route, title, compaction | Haiku 4.5 kept. Nothing cheaper fits, and quality held |
Spotter continuity
- A run that answers a pending tool call (confirmation or picker) finishes on the model that made the call.
spotterLoopCall(model of the last assistant turn)returns the policy’s loop call or one ofEARLIER_SPOTTER_LOOP_CALLS: Sonnet 5 with adaptive thinking, or Sonnet 4.5 with no thinking. A thinking model can continue a tool turn only from that turn’s own signed thinking blocks. - Thinking blocks are signed by the model that wrote them. When
AgentOrchestrator.loopHistoryreplays history, an assistant turn keeps its thinking blocks only if the run’s call thinks and the turn was written by the same model. The stored rows keep the blocks either way. - Prompt cache. The cache is per model, so after deploy the first turn of every conversation writes a new cache prefix (tool list, system prompt, and history) on Sonnet 5.5. Each prefix is written once and read back afterwards. A conversation whose history had Sonnet 5 thinking blocks also loses those blocks from its replayed prefix.
Retirements
sonnet-5flag. The key, the gate (selectFlaggedModel,isSonnet5) and the flag-off branches are removed. The key is listed with the retired keys infeature-flags.ts, so it is never reused.- Sonnet 4.5. No role uses it. It is listed under
RETIRED_MODELS, and it stays inMODEL_RATESfor two reasons: a pending tool call it made can still be finished on it, and its usage that is already recorded can still be priced. An id with no rate is priced at Sonnet 4.5 rates, the most expensive in the table.
Consequences
- Good: the standard is one diff, and a review shows every role it touches. Pricing and call-shape compatibility are checked by tests, not caught in production.
- Good: Spotter costs about 40% less per turn, with a lower first-token latency.
- Cost: Sonnet 5.5 with
autotool choice can answer without calling the tool. The prompt tells it to call the tool, and the existing “no tool_use” failure paths still apply (the call throws, or the batch comes back empty). It called the tool in 6/6 cue runs, 3/3 video runs and 3/3 translation runs. - Cost: a model swap is no longer flag-gated per org. A rollback is a code change to the policy.
- Cost: every conversation pays one cache write on its first turn after deploy.
Alternatives considered
- Keep per-call-site constants. This is how call shapes drifted from models (
isSonnet5) and how a model with no rate was priced at the wrong rates. - Keep the flag as a kill switch. It was at 100%, and both its branches put models on different call shapes. Reverting the policy file is the kill switch.
- Sonnet 5.5 everywhere. It matched Sonnet 5 on the dashboard paste but was slower and more expensive, and it cannot take the forced tool that the paste prompt is built around.