How to Calculate LLM API Cost

Step 1: define the billing unit

Start with one unit that maps to a product decision. Useful units include:

One model request
One completed task
One conversation
One active user per month
One customer account per month
One thousand documents processed

A request is useful for technical accounting. A completed task is often better for workflows that retry or call several models. Cost per active user is useful for pricing and gross-margin decisions.

Do not begin with total monthly spend alone. A total cannot explain whether growth, longer prompts, power users, or failures caused the change.

Step 2: collect current rates by category

For each exact model and processing mode, record the published unit price for every category the workload uses:

Ordinary input tokens
Cached-input reads
Cache creation or writes when priced separately
Output tokens
Batch, priority, or realtime processing
Audio, image, search, tool, or other feature charges
Fine-tuned model rates when applicable

Store the source URL, currency, unit, model identifier, and date checked. Do not copy a family-level headline onto a different model version. Provider terms and price tiers can change.

The provider pricing index helps compare source-backed rates, but the provider’s current documentation remains the final contract for billing.

Step 3: measure input and output separately

For a simple token-priced call:

input cost = input tokens ÷ 1,000,000 × input price per million

output cost = output tokens ÷ 1,000,000 × output price per million

request cost = input cost + output cost

Use actual API usage fields when available. Input can include system instructions, user text, conversation history, tools, examples, and retrieved documents. Output is what the model generates under the provider’s accounting rules. Read input tokens versus output tokens before building the ledger.

Do not multiply total tokens by a blended rate unless the input-output ratio is fixed and clearly documented. Separate categories make the model auditable when behavior changes.

Step 4: add prompt-cache categories

When prompt caching is enabled, use the categories reported by the API:

cache cost = cache-write tokens × write rate + cache-read tokens × read rate

total input-side cost = uncached input cost + cache cost

A request hit rate is not enough. Savings depend on the number of tokens reused. One small cache hit and one large miss should not be averaged as a 50 percent token saving.

Model cache expiry, misses, and prefix changes. The prompt-caching definition explains the difference between cache writes, reads, and response caching.

Step 5: calculate calls per completed task

A user action may create more than one billable request. Include:

Validation or classification calls
Main generation
Tool-selection calls
Tool-result follow-up calls
Safety or moderation calls when separately charged
Reranking
Fallback models
Retries after errors or invalid output
Evaluation or judge calls

For each call type i:

task model cost = sum(request count_i × average request cost_i)

If 10 percent of tasks require one extra attempt, the average call count rises to at least 1.10 before other workflow stages are included.

Track successful and failed calls. A request that does not produce a usable customer result can still consume tokens.

Step 6: use distributions instead of one average

Calculate at least three workload profiles:

Profile	Purpose
Median	Represents ordinary usage
95th percentile	Reveals margin pressure from heavy users
Worst credible case	Tests caps, abuse controls, and bill shock

For each profile, measure requests, input tokens, output tokens, cache reuse, retries, and model routing. A single average can hide a small group that creates most of the cost.

The Scenario Studio is designed for these side-by-side assumptions.

Step 7: convert request cost into monthly cost

For one homogeneous route:

monthly route cost = monthly requests × average cost per request

For several routes or models:

monthly inference cost = sum(monthly volume_i × unit cost_i)

Then add non-inference product costs that scale with the feature:

Embeddings and vector search
Storage and data transfer
Speech or image processing
Observability and trace retention
Evaluation runs
Queueing or serverless execution
Human review and support

Keep fixed costs and variable costs separate. Variable cost supports unit economics; fixed cost supports budget and break-even analysis.

Worked example with illustrative rates

Assume a text feature has the following modeled workload. These values are illustrative, not current provider quotes.

Assumption	Value
Monthly completed tasks	100,000
Average calls per task	1.15
Input tokens per call	2,400
Output tokens per call	600
Illustrative input rate	$2 per million tokens
Illustrative output rate	$8 per million tokens

One call costs:

2,400 ÷ 1,000,000 × $2 + 600 ÷ 1,000,000 × $8 = $0.0096

Average model cost per completed task:

1.15 × $0.0096 = $0.01104

Monthly model cost:

100,000 × $0.01104 = $1,104

Now suppose retrieval, observability, and storage add $0.0018 per task:

monthly variable AI cost = 100,000 × ($0.01104 + $0.0018) = $1,284

The example demonstrates the method only. Replace every rate and usage assumption with current source data and measured product behavior.

Step 8: calculate cost per active user

If the feature has U monthly active users:

AI cost per active user = monthly variable AI cost ÷ U

For plan-level economics:

gross margin = (revenue - AI COGS - other COGS) ÷ revenue

Calculate by plan or customer segment. Enterprise accounts, free users, and automated users can have very different request distributions.

Use the AI cost calculator for a fast monthly estimate and the per-user margin calculator when connecting usage to pricing.

Step 9: model routing and fallback explicitly

For traffic split among models:

routed cost = sum(traffic share_i × cost per request_i)

Add router-classification cost and fallback probability. Shares should sum to one for the initial route, while retries and fallbacks are additional conditional calls.

A cheaper model does not save money if it fails often enough to trigger premium fallbacks. Measure accepted-result cost:

cost per accepted result = total workflow cost ÷ accepted results

This metric aligns cost with quality.

Step 10: reconcile estimates with invoices

An estimate becomes reliable through reconciliation:

Sum logged provider usage by day and model.
Apply the rate table used by the model.
Compare the result with provider billing exports.
Investigate taxes, credits, discounts, rounding, unlogged calls, and non-token products.
Version rate changes rather than rewriting history.
Update workload percentiles from production data.

Do not force the model to match an invoice by hiding unexplained differences inside a generic multiplier. Keep a reconciliation adjustment visible until the cause is known.

Launch checklist

Before launch, confirm:

Exact model identifiers and source URLs are recorded.
Input, cached input, and output are separate.
Hidden application context is included.
Retries and fallbacks are measured.
Median and tail users are modeled.
Non-token services are included.
Cost caps and alerts exist.
The free tier has a worst-case budget.
Logs can be reconciled with billing.
Product pricing is tested against per-user COGS.

Frequently asked questions

How do I calculate cost per LLM request?

Multiply each input, cached-input, and output token count by its matching unit rate, then add any feature charges and conditional calls. Use provider-reported usage when possible.

Should failed requests be included in LLM cost?

Yes. A failed or rejected product outcome may still contain billable model work. Track provider errors, application retries, invalid structured output, and user-visible failures separately.

Can I use an average token count for budgeting?

Use an average for a first estimate, but also model percentiles and worst credible cases. Heavy users, long contexts, and automation can dominate the monthly bill.

How often should API prices be updated?

Check on a defined schedule and when providers announce model or pricing changes. Keep effective dates so historical estimates and invoices remain reproducible.

How to Calculate LLM API Cost per Request, User, and Month