The best coding model changes when the benchmark changes.

Use LiveCodeBench for code generation, SWE-bench for repository repair, Terminal-Bench for tool-using agents, and SciCode for scientific programming. If you are choosing a production coding assistant, use at least one repository benchmark, one terminal or agent benchmark, and a private evaluation from your own code.

A coding benchmark is a compressed description of a job. When people ask “which model is best at coding?”, they often combine autocomplete, algorithmic problem solving, code review, debugging, repository search, terminal use, test repair, architecture, and frontend taste into one word. No single public benchmark covers that surface.

What each coding benchmark actually measures

HumanEval still appears in many comparison articles because it is simple and familiar. It is now more useful as a historical baseline than a selection benchmark. Modern frontier models can saturate narrow function-completion tests while still failing to navigate an unfamiliar repository or recover from a broken test run.

One dataset, four different ranking questions

The tables below use the same model dataset but sort it by four different metrics. If the first-place model changes, the benchmark is not necessarily contradicting itself. The question changed.

A composite Coding Index is an efficient shortlist. It should not be the final decision. If your product is a terminal agent, a two-point composite advantage can matter less than a ten-point Terminal-Bench difference. If your users mostly ask for small Python functions, the reverse may be true.

SWE-bench scores models and systems unless you control the harness

SWE-bench Verified contains 500 human-validated, solvable GitHub issues. That makes it far more realistic than isolated code generation. It also makes the evaluation a system test: an agent must inspect a repository, decide what to open, create a patch, and often interpret tests.

The official leaderboard therefore separates the full system leaderboard from a bash-only language-model view using mini-SWE-agent. Even the bash-only view tracks harness releases because tool-calling and prompt changes affect comparability. Compare models only under the same agent and release. Compare products on the full leaderboard only when the product—not the base model—is what you are buying. See the official SWE-bench Verified notes.

Build a coding scorecard around failure, not prestige

The right scorecard starts with what an incorrect answer costs. A failed autocomplete is cheap. A plausible patch merged into a production repository is not. Weight your evaluation accordingly.

The weights above are a starting hypothesis, not a universal formula. Change them for your workflow, preregister the acceptance rule, and keep the failed outputs. The failure clusters usually teach more than the average score.

Methodology and sources

• Live leader tables use Artificial Analysis metrics exposed through the WhatLLM data layer.
• Benchmark descriptions use official benchmark documentation where available.
• SWE-bench’s 500-task count and harness caveats come from its official Verified documentation.
• Scores with missing coverage are excluded rather than imputed.
• Model variants are deduplicated by base name within each top-five table.

For the current multi-benchmark ranking, continue to Best LLM for Coding.