Trust & method
AI cost FAQ: estimating, caching, batch, margins, and sources
Answers to the most common AI cost questions, grounded in the ByteCosts pricing index and its public formulas: how to estimate cost per user, why output tokens usually dominate, how prompt caching and its break-even work, when the batch API beats realtime, how power users break flat-plan margins, when API pricing beats a subscription, how accurate the estimates are, where the data comes from, how often it's updated, and how to cite it. The throughline: list price rarely decides cost - output volume, cache hit rate, retries, and the power-user tail do. Each answer links to the calculator that turns the idea into a number for your workload, and to the methodology behind the data.
FAQ summary
The questions on this page cover the decisions teams usually need before they can budget an AI feature:
- These answers cover the questions teams ask most when budgeting an AI product. Each is grounded in the ByteCosts pricing index and the shared cost engine, and links to the calculator that turns the principle into a number for your own traffic.
- Where a figure is an estimate rather than an official list price, the answer says so. For the sourcing and grading behind the data, see the methodology; for the underlying terms, see the glossary.
- Use the FAQ to choose the right calculator, then model the actual seats, requests, token mix, cache hit rate, retry rate, and pricing source for your workload.
Frequently asked questions
How do I estimate AI app cost per user?
Start from one request: input tokens × input rate + output tokens × output rate, normalized to USD per 1M tokens. Multiply by requests per user per day and days per month to get per-user cost, then by monthly active users for total spend. The catch is averages hide the heavy tail - a small share of power users can be many times the median and dominate the bill. So model the average and the heavy users separately, and add retry overhead, prompt-cache effects, and any gateway fee, none of which appear in a provider's list price. Use the AI cost calculator at /tools/ai-cost-calculator for the simple case and Scenario Studio at /tools/scenario-studio when seats, caching, retries, and a model mix matter.
Why can output tokens dominate LLM cost?
Output tokens are almost always priced several times higher than input - often 3–5×. So even when a workload sends far more input than it generates, a modest amount of expensive output can be the larger half of the bill, and reasoning or verbose responses make it worse because they emit many more output tokens per call. Input-heavy patterns like RAG and classification are the exception, where input genuinely leads. The practical takeaway: control response length, cap reasoning where you can, and compare models on their output rate, not just the headline input price. See the glossary entry for output tokens at /glossary and model it at /tools/ai-cost-calculator.
How do prompt caching savings work?
Caching lets you reuse a processed prompt prefix - system prompt, tool definitions, shared context - instead of re-billing it at the full input rate every request. A cache read costs about 0.1× the input rate, but writing to the cache costs a premium: roughly 1.25× for a 5-minute TTL or 2× for a 1-hour TTL. That means caching only pays off above a break-even reuse count: a prefix has to be read back enough times to recoup the write premium. The lever that decides whether caching helps is your cache hit rate. A cold or rarely-reused prefix can actually cost more. Model your prefix size, hit rate, and TTL at /tools/cache-savings.
When is batch API cheaper than realtime?
The batch API trades latency for a discount - typically around 50% off realtime rates - by returning results within a window that can be up to 24 hours. It's cheaper whenever the work is latency-tolerant: summarization, classification, extraction, evals, and offline pipelines. It is not an option for interactive features where a user is waiting on the response, which must use realtime at full price. A common pattern is to route background jobs to batch and only user-facing calls to realtime. Quantify the split and the savings for your own mix at /tools/batch-vs-realtime.
How do power users break AI SaaS margins?
Usage-based cost scales with how much each user does, but flat per-seat pricing collects the same revenue from everyone. When the top 1% of users run many times the median - and on AI products they often do - their token cost can exceed what they pay, and a handful of accounts can erase the gross margin of an entire plan. Per-user cost averages hide this completely. The fix is to model the heavy tail explicitly, consider usage caps or metered overage, and price with the power-user cost in view, not the average. See the power user and gross margin entries in the glossary at /glossary and stress-test a plan at /tools/scenario-studio.
Is API pricing cheaper than subscription pricing?
It depends entirely on volume. Pay-as-you-go API cost is linear in requests; a subscription is a flat monthly fee. The two cross exactly once: below the crossover the API is cheaper, above it the subscription wins. So there's a real break-even request volume, and which side you're on is the whole question - light or spiky usage favors the API, heavy steady usage favors a flat plan. Note that two pay-as-you-go models, by contrast, never cross, because both are linear through the origin; the cheaper-per-request one wins at every volume. Find your subscription-vs-API crossover at /tools/scenario-studio.
How accurate are ByteCosts estimates?
List prices are read from providers' own pages and dated, with a confidence grade from A+ to E - so the input rates are as accurate as the source they came from. The estimates a calculator produces are exactly as accurate as your inputs: token counts, request volume, retry rate, and cache hit rate are assumptions you supply, and real workloads vary. Treat calculator output as a well-grounded planning figure, not a quote - validate against your provider's actual billing once you have traffic. The formulas themselves are public and shared across every tool, so the math is consistent and inspectable. See how prices are graded at /methodology.
Where does ByteCosts pricing data come from?
Prices come from each provider's official pricing and documentation pages, normalized into one index and dated. The index currently covers 4,312 priced models across 152 providers, plus 310 officially-extracted rows, 299 of them graded A or A+. Precedence is strict: a provider's pricing page beats its docs, which beats normalized third-party data, and a price is never added unless the source is clear and current. You can trace any figure from the sources page at /sources, and read the full sourcing and grading method at /methodology.
How often is ByteCosts updated?
ByteCosts is refreshed by a manual, network-free command the operator runs locally - never automatically by the live site, a deploy, or CI. That's a deliberate cost-safety choice: it keeps the project a static site that never calls a provider API and never incurs runtime cost. The "last updated" date shown across the site (currently July 9, 2026) tracks the committed dataset's own timestamp, not the deploy time. Because updates are manual, a provider may change a price before ByteCosts reflects it - so for a commitment, confirm the current rate on the provider's source page, linked from /sources.
Can I cite ByteCosts data?
Yes. Every generated page carries a citation block with a title, the last-updated date, and the canonical URL, plus a Markdown mirror and links to the relevant data reference pages. Cite the page URL and the last-updated date so readers can see the dataset snapshot you used. Because prices are list prices and refreshed manually, note the date in your citation and link to the relevant provider source for the live rate. Start with /data/llm-pricing-json or /data/llm-pricing-csv for the export schema before consuming raw files.
ByteCosts AI cost FAQ. ByteCosts. Updated July 9, 2026. https://bytecosts.com/faq/