Methodology

How the number is made

Solvency publishes one number that does not appear on any pricing page: what it costs to get a task finished.

cost_per_attempt     = loops × (tokens_in × input_price + tokens_out × output_price)
cost_per_solved_task = cost_per_attempt / pass_rate

Two cost bases, never mixed

Where a benchmark publishes a per-task cost it actually observed, that figure is used directly and no Solvency assumption touches it — no loop count, no per-loop token estimate, no efficiency multiplier. Where only a pass rate is published, cost comes from a loop model that is an assumption, and is labelled as one. The two are reported separately and are never averaged. Tests assert that changing an assumption cannot move a measured row.

The frontier

The scatter on the calculator plots pass rate against cost per solved task. The frontier joins the measured models that no other measured model beats on both axes at once — cheaper and at least as likely to solve the task. Modelled points are drawn hollow and are never on the line, because a modelled cost and a measured cost are not the same kind of number.

Retry variants

Three are computed. The calculator leads with naive (cost / p): unlimited independent retries.

• capped — min(1/p, K) × cost + residual × (1−p)^K; only meaningful with a real human-takeover cost set.
• truncatedGeometric — (E[N] × cost + (1−p)^K × residual) / P(solved), the rigorous truncated-geometric form.

Two proven results: with no human-takeover cost, truncatedGeometric reducesexactly to naive; and capped understatescost for low pass rates — at p = 0.2 it bills 3 attempts instead of 5 and books nothing for the 51% of tasks still unsolved, which makes weak models look cheap.

Assumption health

frontier_efficiency (0.6× loops on heavy tasks for frontier-class models) isdemoted to fallback. It has no published source, and it applies only to rows with no measured cost. Untick it in the calculator's Assumptions drawer to see unadjusted results. Solvency's own measured runs will replace it outright.

Task tiers (light / moderate / heavy) are modelling assumptions inherited from a public cost calculator that states they are not industry-standard benchmarks; Solvency inherits that caveat verbatim. They move modelled rows only.

Rules

• Prices are verified against the provider's own pricing page on the date recorded. Prices recalled from memory are never used.
• Missing is missing. A number that does not exist is reported as missing, never estimated, and a missing cache price is never treated as zero.
• Every modelled parameter is labelled an assumption, carries its provenance, and is adjustable.
• Benchmark figures are third-party, cited and linked, and are not redistributed.
• Measured rows are harness + model pairs, not properties of the bare model; the harness travels with every row.

What this does not measure

A METR randomised controlled trial found experienced open-source developers were19% slower using early-2025 AI tools while believing they were20% faster. Cost per solved task measures benchmark cost. It is not a productivity measure, and the two must not be conflated.