Know what’s
behind the number.

1. Agent Fit: capability for a specific job

Agent Fit is our weighted mean of LiveBench category scores, on a 0–100 scale. The source reports 23 tasks grouped into seven categories. We first average the task scores within each category, then apply the chosen profile’s weights. All 61 configurations use the same 2026-06-25 question set. This date identifies the benchmark release; it is not the release date of every model.

A higher score means better performance on this selected task mix. It is not a percentage of successful agent runs. The weights are our editorial choices, made explicit below.

Swipe the table to see all four task profiles →

LiveBench Overall remains visible as the equal-weight mean of all seven categories. Mathematics has no separate weight in our task profiles; it remains visible for readers who need quantitative problem solving.

2. Practical Value: capability with a budget

We combine Agent Fit with affordability. Affordability measures how a model’s estimated cost ranks against all models with prices in this review: cheaper models rank higher. The default gives 70% to capability and 30% to affordability. You can change both the token workload and this tradeoff.

The monthly estimate is calls × (input tokens × input rate + output tokens × output rate) / 1,000,000. For example, $2 input and $10 output rates with 10,000 input tokens, 2,000 output tokens and 1,000 calls produce a $40 estimate.

Include all billed reasoning in output. Estimates exclude caching, cache writes, search or other tool fees, retries, image/audio charges, hosting, taxes, unrecorded long-context premiums and special service tiers. The verified prompt-length tiers stored in the model snapshot apply to the full request. Haiku 5.5 crosses its tier above 100,000 input tokens; the model page shows both rates. Input and output token counts are assumed identical across models even though tokenizers differ. We do not estimate success-adjusted cost without real workflow measurements.

We prefer checked direct standard API rates for the OpenAI and Anthropic models listed in the snapshot. Other prices are labeled OpenRouter catalog rates. Model pages retain both observations when direct and catalog prices differ. Context and tool-support fields refer to the catalog endpoint; verify them with the endpoint you purchase.

3. Evidence Consensus: combine two kinds of evidence

10 configurations have an explicitly matched Arena text-preference result and a LiveBench result. Within that same group, we turn each result into a relative rank from 0 to 100, called a percentile, then average the two ranks equally. This prevents directly averaging an Arena rating around 1,500 with a benchmark percentage around 80.

Percentile = 100 × (number below + (number tied − 1) / 2) / (cohort size − 1). Identical values share a midrank. A one-model cohort or an all-equal cohort receives 50. Affordability uses the same rule with lower costs ranking higher.

This is a 10-configuration comparison, not a whole-market intelligence ranking. Missing pairs receive no Consensus score. Filter controls never change the reference cohort. An updated snapshot can change percentiles even if an individual raw score stays the same.

Arena measures human preference, which can reward style. LiveBench measures performance on objective tasks. They are different signals, and combining them is an editorial choice. We display Arena’s published ± range and vote count on profiles; those uncertainty values are not propagated into a confidence interval for our aggregate. Preliminary results remain labeled. Small rank differences should not be treated as decisive.

The chart sweet spot: a quick visual shortlist

The cyan area highlights models with a score at or above the 75th percentile and an estimated cost at or below the median of the currently visible, comparable models. Both thresholds use linear interpolation. The area is shown when at least three models have a positive cost and a score for the chosen metric. A model does not have to sit inside it to be worth considering.

The three numbered picks come from the efficient frontier: no visible model is both cheaper and at least as capable, with one strict improvement. Models inside the shaded zone come first. For models outside it, we measure the distance to the nearest edge of the zone: the cost excess on a normalized logarithmic scale and the score shortfall on a 0–1 scale.

Equal zone distances are resolved by distance to the high-score, low-cost corner, using log-cost normalized between the cheapest and most expensive visible models and a score gap from 100. Further ties favor the higher score, then the model name. Equal costs contribute no cost-distance difference. The numbered list can include models just outside the zone; they remain labeled as nearby tradeoffs.

The markers, zone and nearby model summaries change with your chart metric, workload and filters. They are a visual aid within the visible selection, separate from the published aggregate ratings. When Evidence Consensus is selected, the comparison is limited to configurations with matched evidence. Zero-cost workloads and missing observations are not placed on the logarithmic chart.

Model identity and missing evidence

Each model profile preserves the exact LiveBench row identifier, reasoning configuration, catalog ID, and matched Arena identifier when present. Only explicit configuration matches enter Consensus. A default or unknown reasoning setting is not assumed to equal “high” or “max.”

Some LiveBench identifiers and display labels disagree about effort. We expose that discrepancy and exclude those configurations from Consensus. Qwen3.8 Max’s version-specific catalog mapping is unconfirmed, so its price and catalog capabilities are withheld. No approximate value is silently inserted.

Missing means unavailable or unverified, never zero. A model without a price receives neither a cost estimate nor a Practical Value score. The same applies when the supplied input plus billed output exceeds its advertised catalog context window. Tool-calling support is a declared feature, not a tested reliability rating. “Open weights” describes availability of parameters, not a blanket license to use them.

The efficient frontier connects models for which no visible model is both cheaper and at least as capable, with one strict improvement. It depends on the selected metric, workload and filters. It is not a recommendation to choose an extreme without testing.

Current evidence downloads

Download the current benchmark snapshot ↓ · Release watch, source checks and upgrade costs ↗

Direct-price observation dates appear in the data per model; unchanged historical sources retain their original dates.

Sources and dates

Download the September 23 archive (JSON) to inspect the observations, source links and source-file hashes. Third-party observations remain attributed to their publishers.

Snapshot reviewed on 2026-10-15. Eight research and data sources inform this section; they do not all contribute to every score. Source update dates and our retrieval date are separate. Artificial Analysis was researched as a design reference; its proprietary scores and speed measurements are not republished or used in these formulas.

What these rankings cannot tell you

The selection contains models covered by the chosen LiveBench release, including older versions still useful for comparison. It is not an exhaustive inventory of every model. Newly launched models, private models and specialized image/audio models may be absent. Current availability and retirement status must be checked with the provider.

We do not publish a throughput or latency ranking here because this snapshot lacks a consistently matched and reusable performance dataset. Those measurements depend on endpoint, load, prompt length, caching and reasoning effort. We also do not treat the older Berkeley and Vectara evaluations as measurements of newer model releases.

Benchmark questions can become familiar to models, category coverage is incomplete, and the source’s testing tools may differ from yours. These scores say nothing definitive about security, privacy, data residency, production reliability or suitability for a sensitive application.

Updates and corrections

A daily review is scheduled for 05:30 Europe/Vilnius. Each published update is a versioned snapshot after source, identity and calculation checks; a checked source can still have older results. The current scoring methodology is version 1.1. Source changes should never silently rewrite an older observation’s date.

To report a correction, contact AI Agent Store with the model name, field, current source URL and observation date. The calculator and rating formulas contain no paid-placement or affiliate-commission input.