Think of an LLM like a taxi. The moment you get in and tell the driver where to go, the meter starts running. Every block you travel (every chunk of text processed) adds to the fare. A token is that “block” — the fundamental billing unit of AI. You pay for every token that goes into the model (your question, context, documents) and every token that comes out (the model’s response).

A token is not a word, a character, or a sentence. It’s a sub-word unit — roughly 3–4 characters, or about 0.75 words in English. LLMs don’t read text the way humans do. They break everything into these small chunks before processing. The word “economics” is 1 token. The word “tokenization” is 2 tokens. The phrase “AI cost engineering” is 4 tokens.

Most modern LLMs use Byte Pair Encoding to decide how to split text. BPE starts with individual characters and iteratively merges the most frequently occurring pairs from a massive training corpus. After training, the algorithm produces a vocabulary of 50,000–100,000 tokens. Common words like “the” or “is” become single tokens. Rare or long words get split into smaller recognized pieces.

OpenAI uses Tiktoken (cl100k_base for GPT-4, o200k_base for GPT-4o and later). Anthropic uses their own BPE variant. Google uses SentencePiece. The same sentence can produce different token counts on different models — “The quick brown fox jumps over the lazy dog” might be 9 tokens on GPT-4o but 11 on another model.

Not all text is created equal in token economics. Common English prose averages about 1.3 tokens per word — relatively efficient. But code averages 1.5–3 tokens per word because of special characters, indentation, and syntax. JSON and YAML are the worst offenders at 2–4 tokens per word, because every brace, bracket, colon, and quotation mark consumes a token.

If your application sends tool schemas, API responses, or structured data to the model, you’re paying a format tax. A single MCP tool definition in JSON can consume 500+ tokens. An agent with 20 tools loaded starts every conversation having already spent 10,000+ tokens just on tool definitions — before the user says a word.

You don’t need a tokenizer to estimate costs. These rough conversions work for English text with GPT-4o’s tokenizer:

Context windows define the maximum number of tokens a model can process in a single request (input + output combined). As of March 2026: GPT-4o supports 128K tokens (~96,000 words), Claude 3.5 Sonnet supports 200K tokens, and Gemini 2.0 supports up to 2M tokens. But bigger isn’t always better — performance degrades well before hitting the limit.

When GPT-4 launched in May 2023, input tokens cost $30 per million and output tokens cost $60 per million. By March 2026, OpenAI’s GPT-5 Nano offers input at $0.05 per million and output at $0.40 per million. That’s a 600x reduction in input cost for a model that’s arguably more capable than the original GPT-4 for most tasks.

Three forces drive the collapse: (1) Hardware improvements — each new GPU generation (H100 → H200 → B200) delivers 2–3x more inference throughput. (2) Algorithmic efficiency — techniques like speculative decoding, quantization, and distillation make models faster and cheaper to run. (3) Competition — DeepSeek, Mistral, and open-source models force prices down across the board.

GPT-5 Nano ($0.05/$0.40), GPT-4.1 Nano ($0.10/$0.40), GPT-4o mini ($0.15/$0.60), DeepSeek V3.2 ($0.27/$0.42). These models handle classification, extraction, summarization, and simple Q&A. They cover 60–70% of production workloads at negligible cost.

Claude 3.5 Haiku ($1.00/$5.00), GPT-5 ($1.25/$10.00), GPT-4.1 ($2.00/$8.00), GPT-5.4 ($2.50/$15.00), Claude 4.6 Sonnet ($3.00/$15.00), Claude 4.6 Opus ($5.00/$25.00). The workhorses for coding, analysis, and complex reasoning.

o1-pro ($150/$600) sits at the extreme end. These models use extended “thinking” chains for hard math, science, and coding problems. The price reflects the enormous compute required for reasoning tokens (covered in Chapter 3).

All API pricing is quoted as cost per million tokens ($/M). When you see “$2.50/$10.00” it means $2.50 per million input tokens and $10.00 per million output tokens. To calculate a single request cost: (input_tokens × input_price + output_tokens × output_price) / 1,000,000.

Training a frontier model is a massive one-time investment (GPT-4 reportedly cost $100M+ to train). But once trained, the model serves millions of users continuously. Inference — the process of generating responses to user queries — now accounts for roughly two-thirds of all AI compute spend industry-wide. Every API call, every chatbot response, every agent action is an inference cost.

Training costs are borne by the model provider (OpenAI, Anthropic, Google). Inference costs are borne by you, the developer or company using the API. As AI agents become more autonomous — making dozens of LLM calls per task — inference costs scale with usage in ways that training costs never did. Understanding token economics is understanding your operational cost structure.

Here’s the complete taxi meter analogy: Getting in the cab = loading your system prompt and context (input tokens). Telling the driver your destination = your user query (more input tokens). The drive itself = the model generating a response (output tokens, billed at a higher rate). Taking a detour = reasoning/thinking tokens you never see but still pay for (Chapter 3). Sitting in traffic = quadratic attention scaling on long contexts (Chapter 3).

Now that you understand what tokens are and how they’re counted, Chapter 2 dives into why input and output tokens are priced differently — the restaurant analogy. Reading the menu is cheap; having the chef cook your meal is expensive. The asymmetry between input and output pricing is the single most important concept in token economics.

Tokens are sub-word units (~0.75 words each) created by BPE. Format matters: JSON costs 2–3x more than prose. Token prices have collapsed 600x since 2023, but the range from budget to flagship is 3,000x. Inference now dominates AI spend. Every AI cost optimization starts with understanding this fundamental unit.