When you look at a photograph, you see objects, faces, scenes. When a computer looks at the same photograph, it sees a grid of numbers. Each pixel is represented by three values (red, green, blue), each ranging from 0 to 255. A standard 1080p image is 1920 × 1080 pixels × 3 color channels = over 6 million numbers. The challenge of computer vision is extracting meaning from this sea of numbers.

The same object looks completely different depending on angle, lighting, occlusion, scale, and background. A cat photographed from above, in shadow, partially hidden behind a sofa, looks nothing like a cat photographed from the side in bright light. Yet humans recognize both instantly. Teaching machines this robustness is what makes computer vision one of the hardest problems in AI — and one of the most commercially valuable.

The computer vision market reached approximately $15–22 billion in 2025 and is projected to exceed $58 billion by 2030. Quality assurance and inspection alone account for over 33% of revenue. Medical imaging is the fastest-growing segment at ~15% CAGR. The market is driven by manufacturing automation, autonomous vehicles, healthcare diagnostics, and retail analytics.

A CNN uses small filters (typically 3×3 or 5×5 pixels) that slide across the entire image. Each filter is trained to detect a specific pattern. One filter might detect vertical edges. Another detects horizontal edges. Another detects a specific texture. As the filter slides across the image, it produces a feature map — a new, smaller image that highlights where that pattern appears.

Early layers detect simple features: edges, gradients, color transitions.
Middle layers combine simple features into textures and shapes: circles, corners, grids.
Deep layers combine shapes into recognizable parts: eyes, wheels, windows.
Final layers combine parts into complete objects: faces, cars, buildings.

This hierarchy is learned automatically from data — no human tells the network what features to look for. It discovers them through training on millions of labeled images.

Between convolutional layers, pooling layers reduce the size of feature maps by keeping only the strongest signals. This serves two purposes: it makes the network computationally manageable (without pooling, the math would be prohibitively expensive), and it makes the network invariant to small shifts — a cat shifted 10 pixels to the right still activates the same features.

Image classification assigns a single label to an entire image: “This is a cat.” “This X-ray shows pneumonia.” “This product is defective.” It’s the simplest computer vision task and the one that launched the deep learning revolution when AlexNet won ImageNet in 2012 (Chapter 9). Today, image classification achieves superhuman accuracy on many benchmarks.

Medical imaging — Classify X-rays, CT scans, and retinal images for disease detection. AI-assisted radiologists achieve higher accuracy than either AI or humans alone.
Quality control — Classify manufactured parts as pass/fail. Detects defects invisible to the human eye at production line speed.
Document processing — Classify documents by type (invoice, receipt, contract) for automated routing and processing.
Agriculture — Classify crop health from drone or satellite imagery.

Training a CNN from scratch requires millions of labeled images. Transfer learning solves this: take a model pre-trained on a massive general dataset (like ImageNet’s 14 million images), then fine-tune it on your specific task with just hundreds or thousands of examples. The pre-trained model already knows how to detect edges, textures, and shapes — it just needs to learn your specific categories.

Classification tells you what is in an image. Object detection tells you what and where — drawing a bounding box around each detected object. A single image might contain multiple objects of different types: three pedestrians, two cars, one bicycle, one traffic light. The model identifies each one, classifies it, and locates it with pixel-level coordinates.

YOLO (You Only Look Once) revolutionized object detection by processing the entire image in a single pass rather than scanning it region by region. This made real-time detection possible — processing 30–60+ frames per second on modern hardware. Now in its 12th major version (YOLOv12, 2025), it integrates attention mechanisms from Transformers while maintaining the speed that made it famous.

Autonomous vehicles — Detect and track pedestrians, vehicles, signs, and lane markings in real time. Safety-critical: must work in rain, fog, darkness, and glare.
Retail analytics — Track customer movement through stores, detect shelf stockouts, monitor queue lengths.
Security & surveillance — Detect unauthorized access, abandoned objects, crowd density anomalies.
Warehouse automation — Identify and locate packages, pallets, and equipment for robotic picking and routing.

While object detection draws boxes, segmentation classifies every individual pixel in the image. It doesn’t just say “there’s a tumor here” — it outlines the exact boundary of the tumor, pixel by pixel. This is the most detailed level of visual understanding, and it’s essential when precision matters.

Semantic segmentation — Labels every pixel by category. All road pixels are “road,” all sky pixels are “sky,” all car pixels are “car.” But it doesn’t distinguish between individual cars.

Instance segmentation — Labels every pixel and distinguishes between individual objects. “This is car #1, this is car #2, this is car #3.” Each gets its own precise outline. This is the most computationally demanding but most informative level of visual understanding.

Medical imaging — Precisely delineate tumor boundaries for surgical planning and radiation therapy targeting. Accuracy directly impacts treatment outcomes.
Autonomous driving — Understand the exact drivable surface, distinguish sidewalk from road, identify lane boundaries at pixel precision.
Agriculture — Map crop health, weed distribution, and irrigation needs at the individual plant level from aerial imagery.
Manufacturing — Measure exact dimensions and surface areas of defects for quality grading.

Medical imaging is the fastest-growing segment of computer vision at ~15% CAGR. The volume of medical images is growing faster than the supply of radiologists. In the US, a radiologist reads an image every 3–4 seconds during a typical shift. AI can pre-screen, prioritize, and flag anomalies — ensuring the most critical cases get immediate attention.

Diabetic retinopathy — Google’s AI system matches or exceeds ophthalmologist accuracy in detecting the leading cause of blindness in working-age adults.
Breast cancer screening — AI reduces false negatives by 9.4% and false positives by 5.7% (Google Health study in Nature).
Lung nodule detection — AI identifies potential lung cancers in CT scans that radiologists miss, particularly small nodules under 6mm.
Pathology — AI analyzes tissue slides at a scale and consistency no human pathologist can match.

The evidence consistently shows that AI-assisted physicians outperform either AI or physicians alone. The model is not replacement but augmentation: AI handles the screening and flagging; the physician makes the diagnosis and treatment decision. This partnership model is critical for regulatory approval, liability, and patient trust.

Quality assurance and inspection accounts for over 33% of computer vision revenue — the largest single segment. The economics are compelling: human inspectors catch 80–90% of defects on a good day, with accuracy declining over a shift due to fatigue. AI vision systems maintain consistent accuracy 24/7, often detecting defects invisible to the human eye (microscopic cracks, sub-millimeter misalignments, subtle color variations).

Cameras mounted on production lines capture images of every product at high speed. The AI model classifies each item as pass/fail, identifies the type and location of defects, and triggers automated sorting. Advanced systems provide root cause analysis — correlating defect patterns with upstream process variables (temperature, pressure, material batch) to identify the source of quality issues before they escalate.

Infrastructure inspection — Drones with AI vision inspect bridges, power lines, pipelines, and wind turbines, replacing dangerous manual inspections.
Construction monitoring — Track progress, detect safety violations, and verify compliance against blueprints from site cameras.
Agriculture — Assess crop health, detect disease, and optimize harvesting from drone and satellite imagery.
Retail shelf monitoring — Detect out-of-stock items, verify planogram compliance, and monitor competitor pricing from shelf images.

“Is this X or Y?” → Image classification. Simplest, fastest, cheapest. Good for binary quality decisions, document routing, content moderation.

“What’s in this scene and where?” → Object detection. Needed when multiple objects must be identified and located. Autonomous driving, security, retail analytics.

“What’s the exact boundary?” → Segmentation. Required when pixel-level precision matters. Medical imaging, precision agriculture, autonomous navigation.

Edge vs. cloud — Real-time applications (autonomous vehicles, production lines) need on-device processing. Batch analysis (medical imaging review, satellite imagery) can use cloud.
Latency requirements — Self-driving cars need <50ms. Quality inspection needs <100ms. Document classification can tolerate seconds.
Accuracy vs. speed tradeoff — Faster models are less accurate. The business context determines which matters more.

Vision Transformers are challenging CNNs by applying the Transformer architecture (Chapter 13) to images. They achieve state-of-the-art results on many benchmarks and are converging with language models in multimodal systems (Chapter 17).

Foundation models for vision (like Meta’s SAM — Segment Anything Model) can segment any object in any image without task-specific training, dramatically reducing deployment time.