LLM API Costs: What Your AI Feature Costs Per User
LLM API costs explained: how token pricing works, what an AI feature costs per user per month, and the levers that cut the bill before you ship.
Most AI features cost cents per user per month, not dollars. LLM API costs are usage-based: you pay per million tokens of input and output, output is priced at roughly five times input, and the same workload can swing twenty-fold depending on which model tier you choose. Once you can estimate tokens per request, you can estimate the bill before writing any code.
This guide is for founders and small engineering teams adding an AI feature to a product and trying to budget for it. It covers how token pricing works, what the main models cost today, worked examples of cost per user, the places bills get out of hand, and the levers that cut spend without making answers worse.
Key takeaways
- Cost per request is (input tokens × input rate) + (output tokens × output rate). Every other pricing feature is a discount or a surcharge on those two numbers.
- Output is the expensive half. Capping response length is usually the fastest saving available.
- Cached input costs a tenth of fresh input or less, so a long system prompt should be cached rather than resent on every call.
- Work that nobody is waiting for belongs in a batch queue: both major providers price batched tokens at half the standard rate.
- Bills rarely break because of traffic. They break because of agent loops, retries and context that nobody capped.
How do LLM API costs work?
You pay for text in and text out, counted in tokens and billed per million. A token is a fragment of text, usually a few characters long. Everything you send counts as input — the system prompt, retrieved documents, conversation history and the user's message — and everything the model writes counts as output.
Three details matter more than the headline rate. Output costs several times input on every mainstream model, so an assistant that writes paragraphs costs more than a classifier reading the same material. Input you have sent before can be served from a cache at a steep discount. And the whole conversation is resent on every turn, so a long chat gets more expensive per message as it goes, not just longer.
Running cost is also a different budget from build cost. If you are still sizing the project rather than the monthly invoice, our breakdown of what an MVP costs to build covers that side of the question.
What is a token, and how do you estimate them?
Start with roughly 1,500 tokens per 1,000 words of English and refine later. Anthropic's documentation puts a token at approximately 3.5 English characters on its earlier models, and notes that Claude 4.7 and later use a newer tokenizer producing about 30 percent more tokens for the same text (Anthropic's glossary of model terms).
That 30 percent is a useful warning. Token counts are not comparable across providers, or even across generations from one provider, so a spreadsheet built on last year's model will drift when you upgrade. When the number genuinely matters, count tokens with the model you plan to use before launch rather than after the first invoice.
What do LLM APIs cost in 2026?
List prices cluster into three tiers that sit roughly a factor of twenty apart. The table below shows published rates in US dollars per million tokens for the current flagship families.
| Model | Input | Output | Cached input | Typical job |
|---|---|---|---|---|
| gpt-6-luna | $0.10 | $0.50 | $0.01 | Classification, routing, high volume |
| Claude Haiku 5.5 | $0.10 | $0.50 | $0.01 | Extraction and tagging, prompts under 100k tokens |
| gpt-6.1-sol | $2.00 | $10.00 | $0.10 | General product features |
| Claude Sonnet 5.5 | $2.00 | $10.00 | $0.10 | General product features |
| Claude Opus 5.5 | $4.00 | $20.00 | $0.20 | Long agentic tasks |
| gpt-6-astra | $10.00 | $50.00 | $1.00 | Hard reasoning, low volume |
Rates come from OpenAI's published API pricing and Anthropic's pricing documentation, both of which change often enough to be worth rechecking before a number reaches a board deck. Read the shape rather than the exact figures: output is five times input across the board, cached input is a tenth of fresh input or better, and the small tier is around a twentieth of the mid tier.
How much does an AI feature cost per user?
For most product features, somewhere between a fraction of a cent and a few cents per user per month. The way to know is to multiply tokens per request by requests per user, then compare the result to what that user pays you.
Three worked examples, all at mid-tier list prices of $2 per million input tokens and $10 per million output tokens:
- Support assistant. 2,000 input tokens (instructions plus one help-centre article) and 400 output tokens per answer works out at $0.008 an answer. Ten thousand answers a month is $80. The same volume on the small tier is $4.
- Document summary. A twenty-page document is roughly 15,000 tokens in and 800 tokens out: about 3.8 cents per document, or $38 for a thousand documents a month.
- Retrieval chat. Eight retrieved passages, a system prompt and a few turns of history come to about 6,000 input tokens, with 500 tokens out. That is 1.7 cents a message, so a user who sends twenty messages a month costs about 34 cents.
Against a $29 subscription, 34 cents is roughly one percent of revenue, and that ratio is the number worth tracking. Token spend only becomes a real problem when it is a meaningful share of what the feature earns, or when a free tier lets strangers spend it for you.
Two costs sit outside the token bill: embeddings and a vector store if the feature retrieves anything, and the ordinary hosting the rest of your app already needs.
Where do LLM bills actually blow up?
Almost always in one of four places, and ordinary traffic growth is not one of them.
Agent loops with no ceiling
An agent that decides its own next step might call tools four times or forty. Each iteration resends a growing transcript, so cost per task rises faster than the number of steps. A hard iteration cap and a maximum runtime are the only dependable defence; our guide to shipping AI agents with guardrails covers the rest of that pattern.
Context that grows every turn
Resending the full conversation is the default behaviour of every chat API. A session that starts at 1,000 input tokens can be sending 25,000 by turn thirty, so the thirtieth answer costs many times the first. Summarise older turns, or drop everything outside a fixed window.
Long-context price cliffs
Some models change rate once a prompt crosses a threshold. Claude Haiku 5.5 is priced at $0.10 per million input tokens for prompts up to 100,000 tokens and $0.50 above that, a fivefold jump that one oversized document can trigger. Reasoning tokens, generated when a model is allowed to think before answering, are billed as output as well.
Tools billed on top of tokens
Built-in tools carry their own line items. OpenAI charges $10 per thousand web search calls and $2.50 per thousand file search calls, plus $0.10 per GB per day for stored files, with retrieved content still billed at the model's token rates. A feature that searches the web on every page view is a tool bill with a model attached.
How do you cut LLM costs without hurting quality?
Work through these in order. The early items are cheap to implement and hard to get wrong.
- Cap output. Set
max_tokensand ask for the length you actually want. Output is the five-times-input side of the bill. - Cache the stable prefix. Put the system prompt and any fixed reference material at the front of the request and mark it for caching. Anthropic prices a five-minute cache write at 1.25× the input rate and a cache hit at a tenth or less, so caching pays for itself after a single hit.
- Route by difficulty. Send routine requests to the small tier and escalate only the ones that fail a confidence check. With twenty times between tiers, even a crude router pays for itself.
- Batch anything asynchronous. Nightly enrichment, bulk classification and back-fills do not need an answer in two seconds, and batched tokens cost half.
- Trim what you retrieve. Six well-chosen passages beat twenty mediocre ones on both cost and answer quality; our walkthrough of retrieval pipelines covers chunking and re-ranking.
- Cache answers, not just prompts. Repeated questions can be served from your own store with no model call at all.
One caution on caching: a cache write costs more than a normal input token, so caching a prompt that is used once is a small loss rather than a saving. It pays off on prefixes that are reused across requests.
Should you host your own model instead?
Not at early-stage volume. An API bill is variable and starts near zero; a GPU bill is fixed and starts the moment the instance boots, whether anyone uses the feature or not.
| Approach | What you pay for | Fits when |
|---|---|---|
| Hosted API | Tokens used, nothing idle | Traffic is variable or still unknown |
| Open model on managed GPUs | Every hour the instance is up | One narrow, constant, high-volume task |
| Your own hardware | Machines, power and an engineer to mind them | Volume is large and predictable, or data cannot leave |
The honest trigger for revisiting this is a token bill that is both large and boring: the same prompt shape, in high volume, every day. Before that, the engineering time costs more than the tokens.
How do you track AI cost per user?
Record the token counts the API already returns, store them next to the user and the feature, and convert to money when you query. Providers return input, output and cached token counts on every response; discarding them is how teams end up guessing at a four-figure invoice.
cost_usd = (usage.input_tokens * INPUT_RATE
+ usage.cached_tokens * CACHED_RATE
+ usage.output_tokens * OUTPUT_RATE) / 1_000_000Two figures deserve a dashboard: cost per active user and cost per completed task. Add an alert for any single request above a threshold you pick deliberately, and set the provider's own spend limit as a backstop. This belongs with your other production signals, which our guide to logging, metrics and alerting on a small team covers in full.
Frequently asked questions
How much should I budget for the first month?
Measure tokens per request on a handful of real examples, multiply by the traffic you expect, then double it. Early features almost always send more context than planned, and the first weeks include your own testing. A feature with a few hundred users and no agent loop rarely exceeds a low two-figure monthly bill.
Is the cheap model good enough?
For classification, routing, extraction, tagging and short rewrites, usually yes, and the gap to the mid tier is about twenty times the price. Reach for a larger model when the task needs multi-step reasoning, long documents held in mind at once, or writing a customer will read closely. Test both on the same twenty real inputs before deciding.
Do I pay for the system prompt on every call?
Yes, unless it is cached. A 2,000-token system prompt sent on every one of 100,000 monthly requests is 200 million input tokens, which is $400 at mid-tier rates and around $20 if the prefix is served from cache. That is the single clearest argument for caching.
What stops one user running up the bill?
A per-user quota counted in your own database, a rate limit on the endpoint, a hard output cap, and an upper bound on input length. Free tiers need all four, because an unauthenticated AI endpoint is an open invitation.
Why did my bill grow faster than my usage?
Context growth is the usual answer: longer conversations, more retrieved passages, or a system prompt that gained a few hundred tokens with each new instruction. Compare tokens per request month over month, not just request counts, and the cause is normally obvious within minutes.
What to settle before you ship the feature
Estimate tokens per request, multiply by expected traffic, compare the result to what the feature earns, and put caps on output, context and loop length before the first real user arrives. Those four steps account for most of practical AI cost control, and all of them are easier to do now than after an invoice arrives.
If you would rather have senior engineers build and instrument the feature for you, AlgoSmiths works on fixed-price projects with a working demo every week. Our AI, ML and data engineering work is where LLM features get built with monitoring and fallbacks in place.