When you send a prompt to an LLM, the model reads everything at once — your system prompt, conversation history, retrieved documents, and your question. This is like walking into a restaurant and reading the entire menu. It takes a moment, but the restaurant can serve many readers simultaneously. Input tokens are processed in parallel in a single forward pass through the model.

The model’s response is generated one token at a time, sequentially. Each token depends on every token before it. This is like a chef preparing your meal — course by course, each dish building on the last. The chef can only work on one dish at a time, and you’re paying for every minute of their attention. Output tokens require a full forward pass per token.

Input processing runs the entire prompt through the model in one parallel operation. Whether your prompt is 100 tokens or 10,000 tokens, it’s processed in a single batch. Output generation requires a complete forward pass for every single token. A 500-token response means 500 separate inference passes through the model’s billions of parameters.

During generation, the model must maintain a KV-cache (key-value cache) that stores the internal state of every token generated so far. This cache grows with each new token and consumes GPU memory throughout the entire generation process. Longer responses require more memory for longer periods.

This is the fundamental constraint: token 47 cannot be generated until token 46 exists. The autoregressive architecture means output generation is inherently sequential and cannot be parallelized. No amount of hardware can make output generation as efficient as input processing — it’s a mathematical limitation of how transformer models work.

Most models charge 4–8x more for output than input. The outlier is DeepSeek V3.2 at only 1.6x — likely a competitive pricing strategy to gain market share. The typical ratio across the industry is 4–5x. This means a request that generates 1,000 output tokens costs the same as reading 4,000–5,000 input tokens.

GPT-5 Nano, GPT-4.1 Nano, GPT-4o mini, DeepSeek V3.2. These models are remarkably capable for their price. They handle classification, extraction, summarization, simple Q&A, and routing decisions. For a customer support bot handling 10,000 conversations/month, budget models cost $2–5/month. They cover 60–70% of production workloads.

Claude Haiku, GPT-5, GPT-4.1, GPT-5.4, Claude Sonnet, Claude Opus. The workhorses for coding assistance, complex analysis, multi-step reasoning, and creative writing. A code assistant for 20 developers costs $30–50/month at this tier. These models handle the remaining 25–35% of tasks that need more sophistication.

o1-pro, o3-pro. Reserved for hard math, science, and coding problems that require extended reasoning chains. These models generate thousands of invisible “thinking tokens” (covered in Chapter 3). A single complex reasoning task can cost $0.50–$5.00. Use sparingly and only when cheaper models genuinely can’t handle the task.

All major providers quote prices as dollars per million tokens ($/M). When you see “$2.50 / $10.00” it means $2.50 per million input tokens and $10.00 per million output tokens. This convention exists because individual tokens cost fractions of a cent — per-million makes the numbers readable.

That $0.0125 per request seems trivial. But at 10,000 requests/day, it’s $125/day or $3,750/month. At 100,000 requests/day, it’s $37,500/month. The per-request cost is always small; the monthly bill is where reality hits. Always multiply by your expected daily volume to get the real picture.

Your system prompt is sent with every single request. A 2,000-token system prompt across 100,000 daily requests means 200M input tokens/day. At $2.50/M, that’s $500/day just for system prompts. Prompt caching (Chapter 6) can reduce this by 75–90%, but only if you know to look for it.

In multi-turn conversations, the entire history is re-sent with each message. A 20-turn conversation accumulates all previous messages as input tokens. By turn 20, you might be sending 15,000+ input tokens per request — even if the user’s latest message is only 50 tokens.

Asking the model to “explain in detail” or “be thorough” can double or triple output length without improving quality. Since output tokens are 4–8x more expensive, verbose instructions have an outsized cost impact. Be specific about desired output length.

Using Claude Opus ($5/$25) for tasks that GPT-4o mini ($0.15/$0.60) handles equally well means paying 33x more for input and 42x more for output. The quality difference for simple classification or extraction tasks is often negligible.

Here’s the full restaurant analogy: Reading the menu = input tokens (cheap, parallel). The chef cooking = output tokens (expensive, sequential). A prix fixe menu = subscription pricing (fixed cost regardless of consumption). Ordering the tasting menu = reasoning models with thinking tokens (Chapter 3). Sending food back = retry loops in agents (Chapter 7).

Chapter 3 reveals the hidden multipliers that make your actual bill much higher than the sticker price suggests. Reasoning tokens, quadratic attention scaling, context surcharges, and batch discounts — the iceberg beneath the surface of API pricing.

Output tokens cost 3–8x more than input because generation is sequential while reading is parallel. Task type determines your bill — summarization (input-heavy) is cheap, drafting (output-heavy) is expensive. The 3,000x price range across models means tier selection is the highest-leverage decision. Always calculate monthly cost at production volume, not per-request cost.