AI Fundamentals
Input Tokens vs Output Tokens: What Counts, What Costs, and Why
Input tokens are the tokenized content a model receives, including application-added instructions and context. Output tokens are the tokens the model generates. Providers may price the two categories differently, and both can consume context-window capacity. Accurate cost models must therefore record them separately instead of multiplying one combined token total by a blended price.
What counts as input tokens
Input tokens are not limited to the text typed into a chat box. They can include every token sent to the model as part of the request:
- System and developer instructions
- The current user message
- Earlier conversation turns
- Few-shot examples
- Tool or function definitions
- Retrieved documents in a RAG pipeline
- Structured data, markup, or image-related text metadata
- Previous tool results returned to an agent
- Any application wrapper added before transmission
The exact accounting fields and names vary by API. Some providers use terms such as prompt tokens, input tokens, cache creation tokens, or cached input tokens. The stable principle is that the model must process the supplied context before it can generate a response.
A short visible question can therefore produce a large input count. A customer may enter ten words while the application adds a long system prompt, a tool schema, chat history, and several retrieved passages.
What counts as output tokens
Output tokens are generated by the model. For a text response, they represent the tokenized completion returned by the model. Tool-call arguments, structured output, reasoning-related usage, or other model-specific categories may be reported separately or included according to the provider’s API contract.
Output length is usually controlled by a maximum-output setting, a stop condition, or the model’s own end-of-sequence decision. A maximum is a ceiling, not a prediction. The model may stop earlier.
Because autoregressive generation produces one token after another, longer outputs increase generation time. They also affect cost when the output rate differs from the input rate.
The basic cost equation
For a provider with separate input and output rates:
cost = input tokens ÷ 1,000,000 × input price + output tokens ÷ 1,000,000 × output price
Suppose an illustrative request uses 8,000 input tokens and 1,000 output tokens. If the illustrative rates are $2 per million input tokens and $8 per million output tokens:
- Input cost:
8,000 ÷ 1,000,000 × $2 = $0.016 - Output cost:
1,000 ÷ 1,000,000 × $8 = $0.008 - Total:
$0.024
These numbers are examples, not current provider quotes. Always substitute the exact model and price tier from the provider’s current pricing documentation. The LLM API cost guide covers cached tokens, retries, and monthly volume.
How caching changes input accounting
Prompt caching can create additional input categories. A repeated prompt prefix may be billed or reported differently from uncached input. Some systems distinguish cache creation from cache reads. Others automatically identify reusable prefixes and report cached tokens in usage metadata.
Do not subtract cached tokens from total input without understanding the provider’s fields. A sound ledger records:
uncached input + cache writes + cache reads + output
Each category is multiplied by its own published rate. Read what prompt caching is before modeling savings.
How input and output share context capacity
The context window defines how much tokenized information the model can use under its API rules. Input and generated output are related because the model must attend to the supplied context and the tokens generated so far.
A request that uses nearly all available capacity for input may leave less room for output, depending on the model and endpoint. Applications should validate both:
input tokens + requested output allowance + required overhead <= supported limit
Provider documentation is the authority for the exact constraint. Do not assume every model treats advertised context and maximum output identically.
Input-heavy and output-heavy workloads
Different products have different token shapes.
Input-heavy examples
- Summarizing long documents
- RAG over several retrieved passages
- Reviewing a large code diff
- Multi-turn conversations with extensive history
- Classifying records with long source text
Output-heavy examples
- Generating articles or reports
- Producing long code files
- Multi-step reasoning responses
- Creating synthetic data
- Expanding outlines into detailed prose
The ratio drives the bill. A model that is economical for short classification may be expensive for long-form generation if output pricing is high. Compare models against the actual distribution of input and output, not a blended token number.
Measure the hidden input
Production logging should capture more than total tokens. At minimum, record:
- Model and provider
- Route or product feature
- Input tokens
- Cached-input categories
- Output tokens
- Request success or retry status
- Latency
- Tenant or plan using a non-PII identifier
Then examine median, 90th percentile, and 95th percentile usage. Large contexts and automated users can dominate spend even when the average appears safe.
The context-window cost calculator helps isolate the price of additional prompt context. Use the AI cost calculator to convert request-level usage into monthly COGS.
Common accounting mistakes
Counting only user text. This misses system prompts, tools, history, and retrieval.
Multiplying all tokens by one rate. Input, cached input, and output may have different prices.
Using the maximum output as actual usage. A configured limit is not the number generated.
Ignoring failed attempts. A retry can create another billable request even if the user sees one answer.
Combining models into one average. Routing and fallbacks should be calculated per model before totals are combined.
Frequently asked questions
Are prompt tokens the same as input tokens?
They usually describe the same broad side of the request, but field names and subcategories vary by provider. Follow the usage schema for the specific API and preserve cached or specialized categories separately.
Does conversation history count as input tokens?
Yes, when the application sends that history back to the model. A user interface may display an ongoing conversation, but the model only sees the context included in the current request or maintained by the provider’s conversation mechanism.
Do tool calls create output tokens?
Tool-call names and arguments are model-generated content and can contribute to output usage according to the API’s accounting rules. Tool results sent back to the model can then contribute to a later request’s input usage.
Which token category usually costs more?
There is no universal rule. Many providers publish different input and output rates, but the relationship depends on the exact model, processing mode, and price tier. Verify the current source page.
Sources
Input Tokens vs Output Tokens: What Counts, What Costs, and Why. ByteCosts. Updated 2026-06-21. https://bytecosts.com/blog/input-tokens-vs-output-tokens/