Large language models are trained to predict the next token. This makes them excellent at pattern matching, fluent text generation, and retrieving memorized knowledge. But reasoning — the ability to follow logical steps, perform multi-step computation, and arrive at correct conclusions through deduction — is fundamentally different from pattern matching. When you ask an LLM “What is 23 × 47?”, it doesn’t compute the answer. It predicts what tokens are likely to follow that question based on training data. For simple problems it has seen many times, this works. For novel problems requiring genuine computation, it fails. This is the reasoning gap: the difference between what LLMs appear to understand and what they can actually reason about.

Daniel Kahneman’s “Thinking, Fast and Slow” describes two modes of human cognition: System 1 — fast, automatic, intuitive. Recognizing faces, reading text, answering “2 + 2 = ?”. Requires no deliberate effort. System 2 — slow, deliberate, analytical. Solving “17 × 24”, planning a route, writing a proof. Requires focused attention and step-by-step reasoning. Standard LLMs are System 1 machines: they generate responses in a single forward pass, with no ability to “think harder” about difficult problems. They spend the same compute on “What color is the sky?” as on “Prove Fermat’s Last Theorem.” The revolution in reasoning AI is about giving LLMs System 2 capabilities: the ability to slow down, think step by step, explore multiple paths, and verify their work.

Not all reasoning is the same. LLMs struggle differently with each type: Mathematical reasoning — arithmetic, algebra, word problems. LLMs fail on novel computations but can solve problems similar to training data. Chain-of-thought helps significantly. Logical reasoning — deduction, induction, abduction. LLMs struggle with formal logic, especially negation and quantifiers (“all”, “some”, “none”). Commonsense reasoning — understanding physical world, social norms, temporal relationships. LLMs are surprisingly good at this because it’s heavily represented in training data. Causal reasoning — understanding cause and effect, counterfactuals. LLMs learn correlations, not causation. Planning — multi-step goal-directed behavior with state tracking. One of the weakest areas for LLMs.

Standard prompting asks the model to produce an answer directly: “Q: Roger has 5 tennis balls. He buys 2 more cans of 3. How many does he have now? A:” The model must jump from question to answer in a single step. For simple problems, this works. But for multi-step problems, the model must perform all intermediate computations implicitly within its forward pass. This is like asking a human to solve a complex math problem entirely in their head, without writing anything down. The problem: the model’s “working memory” is its hidden state, which has limited capacity. Complex reasoning requires more intermediate state than the model can maintain in a single forward pass. The solution: let the model “write down its work” by generating intermediate reasoning steps as text.

The field of LLM reasoning has evolved rapidly: Jan 2022 — Wei et al. publish “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” The foundational paper. Mar 2022 — Kojima et al. discover zero-shot CoT: “Let’s think step by step” works without examples. Mar 2022 — Wang et al. introduce self-consistency: sample multiple reasoning paths, take majority vote. May 2023 — Yao et al. publish Tree-of-Thought: BFS/DFS over reasoning paths. Sep 2024 — OpenAI releases o1: the first model trained specifically for reasoning with test-time compute scaling. Jan 2025 — DeepSeek releases R1: open-source reasoning model matching o1 performance. Jan 2025 — OpenAI releases o3-mini: 14x cheaper than o1 with better math performance.

AI capability can be improved along two dimensions: Training compute (traditional scaling) — more parameters, more data, more training time. This is the approach that gave us GPT-3 → GPT-4. Diminishing returns: doubling parameters doesn’t double capability. Test-time compute (reasoning scaling) — more computation at inference. Let the model think longer on harder problems. This is the approach behind o1/o3. The key insight from OpenAI’s research: for reasoning tasks, scaling test-time compute can be more efficient than scaling model size. A smaller model that thinks longer can outperform a larger model that answers immediately. This changes the economics of AI: instead of always needing bigger models, you can use smaller models with more inference compute for reasoning-heavy tasks.

This course covers the full spectrum of reasoning techniques: Chain-of-Thought (Ch 2) — the foundational technique. Generate intermediate reasoning steps. Zero-shot and few-shot variants. Self-consistency for robustness. Tree-of-Thought (Ch 3) — explore multiple reasoning paths using search algorithms (BFS, DFS, MCTS). Test-Time Compute (Ch 4) — the o1/o3 approach. Train models to reason with “thinking tokens.” Scale compute at inference. Verification (Ch 5) — process reward models (PRMs) that verify each reasoning step. Outcome reward models (ORMs). Tool Use (Ch 6) — augment reasoning with code interpreters, calculators, and retrieval. Benchmarks (Ch 7) — how we measure reasoning. GSM8K, MATH, ARC-AGI, GPQA. Future (Ch 8) — open-source reasoning models, hybrid approaches, and what’s next.

Reasoning is arguably the most important frontier in AI: AGI connection — many researchers believe that robust reasoning is the key missing piece for artificial general intelligence. If we can make AI truly reason, we unlock capabilities far beyond pattern matching. Practical impact — reasoning AI enables: reliable code generation (plan before coding), scientific discovery (hypothesis generation and testing), mathematical proof (formal verification), medical diagnosis (multi-step differential diagnosis), legal analysis (complex argument construction). Economic shift — reasoning models change the cost structure of AI. Instead of paying for the biggest model, you pay for the right amount of thinking. o3-mini costs 14x less than o1 but outperforms it on math. Democratization — DeepSeek-R1 (open source, Jan 2025) made reasoning capabilities available to everyone, not just OpenAI customers.