A cheap failed patch is not cheaper than an expensive accepted patch. Token pricing is an input cost. Engineering teams pay for outcomes: an issue resolved, a test repaired, a migration completed, or a review comment that catches a real defect.
Coding agents amplify the mismatch because they are loops. They read context, call tools, write files, run tests, inspect errors, and try again. Cached prompts can lower the bill; long reasoning traces and repeated failures can raise it. The same per-token price can produce very different task economics.
Token price
What one unit of text costs
Task cost
What one benchmark attempt costs
Solved-task cost
Task cost adjusted for success
Accepted-result cost
Solved cost plus review, tools, and escalation
A better cost formula
The cleanest production metric is total workflow cost divided by accepted results. Before you have private evaluation data, a public benchmark can provide a rough normalization:
Screening proxy
Example: a $0.20 task cost at 50% success becomes a $0.40 normalized cost. This assumes independent attempts and equal task difficulty. Real retries are neither, so use the number to shortlist models—not approve a budget.
We use Terminal-Bench 2.1 when available and Terminal-Bench Hard otherwise because both are closer to tool-using coding work than isolated function generation. The numerator is Artificial Analysis cost per Intelligence Index task, not a terminal-only bill. That mismatch is why every value below is labeled a proxy.
Normalized cost table
Cost adjusted for terminal success
Filtered to models with a Coding Index of at least 25 and both cost and terminal data.
| Model | Coding | Terminal | Task cost | Expected attempts | Normalized proxy | Response |
|---|---|---|---|---|---|---|
| Devstral 2 Mistral · Open | 31.3 | 19% | $0.000 | 5.26× | $0.000 | 10s |
| North Mini Code Cohere · Open | 36.5 | 31% | $0.000 | 3.23× | $0.000 | 26s |
| Command A+ Cohere · Open | 27.8 | 25% | $0.000 | 4.00× | $0.000 | 14s |
| Devstral Small 2 Mistral · Open | 29.3 | 17% | $0.000 | 5.88× | $0.000 | 9s |
| MiMo-V2.5 Xiaomi · Open | 56.8 | 42% | $0.010 | 2.38× | $0.02 | 37s |
| DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek · Open | 56.2 | 36% | $0.02 | 2.78× | $0.06 | 57s |
| MiMo-V2.5-Pro Xiaomi · Open | 60.2 | 43% | $0.03 | 2.33× | $0.07 | 49s |
| DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek · Open | 59.4 | 46% | $0.04 | 2.17× | $0.10 | 1.3m |
| Gemma 4 31B (Non-reasoning) Google · Open | 33.2 | 30% | $0.03 | 3.33× | $0.11 | 15s |
| Gemini 3.1 Flash-Lite Google · Proprietary | 34.7 | 24% | $0.04 | 4.17× | $0.17 | 8s |
| Gemma 4 26B A4B (Reasoning) Google · Open | 39.3 | 14% | $0.03 | 7.14× | $0.21 | N/A |
| GPT-5.6 Terra (low) OpenAI · Proprietary | 58.1 | 44% | $0.10 | 2.27× | $0.23 | 6s |
The first row is not automatically “best.” A value leader can be the right default for routine work and the wrong choice for a risky migration. Devstral 2 is the highest-ranked open-weight option in the normalized set shown here. MiMo-V2.5-Pro has the highest Coding Index among the comparable rows. Those are two different buying situations.
Route by failure cost, not one global leaderboard
Routine tier
Low-cost capable model
Tests, docs, mechanical edits, and bounded transformations with automated verification.
Complex tier
Strong coding model
Multi-file implementation, debugging, and tasks where a retry is still affordable.
Escalation tier
Frontier model + review
Security, architecture, destructive changes, or ambiguous work with high failure cost.
Routing changes the economics more than shaving a few cents from every request. A default model can handle the large easy middle while uncertainty, repeated test failure, or sensitive file paths trigger escalation. A deterministic script should replace the model entirely when the task does not need judgment.
Measure accepted-result cost on your own repositories
- Select 30–100 recent tasks across routine, complex, and high-risk buckets.
- Freeze the agent harness, tool permissions, context policy, and retry budget.
- Record model tokens, cache tokens, tool costs, wall-clock time, retries, and human review minutes.
- Define acceptance before testing: tests pass, patch is relevant, review finds no critical issue, and no prohibited action occurs.
- Calculate total workflow cost per accepted result and report confidence intervals by task bucket.
- Keep failed traces. Repeated failure modes should change routing, prompts, tools, or model choice.
Include human time
A model that produces plausible but subtly wrong patches can look excellent on API spend and terrible on engineering economics. Price review minutes at a consistent internal rate. Otherwise the cheapest model wins by exporting cost to humans.
What this analysis does not prove
- The normalized proxy is not measured cost per solved Terminal-Bench task; the numerator comes from the broader Intelligence Index.
- Expected attempts assumes independent retries, which real agents rarely satisfy.
- Benchmark success does not include your review standard, tools, repository, or security policy.
- Provider pricing and cache behavior can change after the snapshot.
- Missing data can exclude strong models from the comparison.
Use the table to decide what to test. Use your private accepted-result cost to decide what to deploy. For the broader economics of agents, read Cost per Task Is the New Agentic AI Model Benchmark.