When the Cheapest Model Stops Being the Cheapest

A low API bill is useful only in the context of the work it buys. An unsuccessful answer can still consume tokens, invoke tools, and require a person to inspect it. Those costs do not disappear because the answer was rejected.

Open the accepted-task calculator. The default inputs reproduce workflow A below; enter the second column to compare workflow B.

Use the same task set and acceptance rule

Consider the same 100 tasks with one attempted result per task in each workflow. For this example, each task can contribute at most one accepted result. The acceptance rule is fixed before comparing the alternatives. All prices, review times, and acceptance counts below are illustrative.

Model spending alone favors A by $8. Review costs $24 for A and $12 for B, so total evaluated spending is $36 and $32 respectively. Dividing those totals by the accepted counts gives $0.45 for A and approximately $0.337 for B.

The denominator changes the comparison as much as the model bill. Dividing $36 by 100 would yield $0.36 per attempt, but the workflow produced only 80 accepted results. The 20 unsuccessful attempts remain part of the spending required to obtain those results.

Reproduce the comparison

function costPerAccepted({ accepted, model, tools, minutes, hourly }) {
  if (accepted <= 0) return null;
  return (model + tools + minutes / 60 * hourly) / accepted;
}
const a = costPerAccepted({ accepted: 80, model: 10, tools: 2, minutes: 60, hourly: 24 });
const b = costPerAccepted({ accepted: 95, model: 18, tools: 2, minutes: 30, hourly: 24 });
console.log(a.toFixed(3), b.toFixed(3)); // 0.450 0.337

This example changes both review effort and accepted output count. It does not isolate a causal effect of model choice. Its purpose is to show which measurements are missing from a token-price comparison, not to demonstrate that a more expensive model is always more economical.

Find the condition that reverses the choice

Hold workflow B’s $20 combined model and tool bill and 95 accepted results constant. To stay below A’s $0.45 per accepted result, B’s total spend must be below $42.75. That leaves $22.75 for review, or 56.875 minutes at $24 per hour.

At the assumed 30 review minutes, B clears that threshold. Above roughly 56.9 minutes, A becomes cheaper under the remaining fixed assumptions. This boundary is more useful than a generic claim that one model is better value: it tells you exactly what to measure in your own workflow.

A labor rate of zero answers a different question, namely cash API expenditure without an assigned review cost. That can be appropriate for a personal experiment, but it should not be confused with the operating cost of a team whose time is constrained.

Count retries without multiplying successes

A real task may need several attempts. Include every attempt’s model and tool spending, plus all review time, but count the final accepted task once. Retrying the same task three times does not create three units of useful output.

Keep unfinished and abandoned work in the measurement window rather than discarding it from the cost numerator. Document whether human edits are permitted by the acceptance rule. An answer accepted only after substantial editing is not economically equivalent to an answer accepted unchanged.

When no result is accepted, the unit cost is undefined rather than zero. The calculator shows that explicitly. A small sample with one success is also not a stable estimate of future operating costs, even though division produces a precise-looking number.

Carry the result into product pricing

Once you have measured an accepted-action cost, use it in AI Plan Stress Test. That connects engineering performance to the usage allowance a subscription can finance.

Keep separate scenarios when task difficulty changes. A model can be economical for extraction and poor value for debugging. Pooling those tasks into one acceptance rate can conceal the decision you actually need to make.

Sources and method

Both workflows are original illustrative assumptions, not named model benchmarks. The calculation counts the same categories for both alternatives and uses a common task set and acceptance rule. The default scenario and the worked comparison are covered by the repository’s editorial regression tests.

• Cost per Accepted Task calculation and limitations
• AI Plan Stress Test
• ByteCosts methodology