Commercial API pricing can become material at high request volume. Open-source deployments shift the cost model toward infrastructure you control and optimize directly.

Every API call goes through a third party. If they change pricing, deprecate a model, or have an outage, your product goes down. Open source models give you a stable, self-hosted foundation.

Sending customer data to an external API raises compliance issues (GDPR, HIPAA, SOC 2). Many enterprises cannot send sensitive data to third-party AI APIs. Open source lets you run entirely within your own infrastructure.

Open-weight model releases accelerated community experimentation and adapter-based customization. The ecosystem moved from a few flagship models to many task-specific variants in a short period.

Recent model generations expanded capability, context handling, and open deployment options across multiple model families. This widened practical use from hobby demos to production copilots, retrieval systems, and domain-specific assistants.

Chinchilla scaling laws (2022) showed models were undertrained. Techniques like LoRA (2021) made fine-tuning cheap. Quantization (GGML/GGUF) made large models run on laptops. Each breakthrough made the next one easier.

The model weights encode learned behavior and are published with model cards and metadata on distribution hubs such as Hugging Face. Those cards describe license terms, intended use, and known limitations that teams should review before adoption.

How weights are stored and transported. SafeTensors: standard for training and cloud GPU inference. GGUF: optimized for CPU and local inference, includes quantization metadata. PyTorch .bin: legacy format, being replaced by SafeTensors.

What actually loads and runs the model weights. llama.cpp: runs GGUF on any hardware. Ollama: wraps llama.cpp with a one-command interface. vLLM: high-throughput production serving. TensorRT-LLM: NVIDIA-optimized inference.

Application-layer frameworks abstract model calls and orchestration. LangChain, LlamaIndex, and DSPy each provide different strengths for workflows, retrieval, and programmatic optimization.

Developed by Hugging Face. Safe (no arbitrary code execution unlike pickle-based .pt files). Zero-copy memory mapping for fast loading. The standard for storing and sharing full-precision model weights. Used for training, cloud serving, and fine-tuning.

The original format — uses Python pickle serialization. Security risk: malicious .bin files can execute arbitrary code. Being phased out in favor of SafeTensors. Most models on HuggingFace now offer SafeTensors versions.

GPT-Generated Unified Format (created by llama.cpp's ggerganov). Single file containing quantized weights + model metadata + tokenizer. Optimized for CPU inference. Supports Q4, Q5, Q8 quantization levels. The standard for local AI with Ollama and llama.cpp.

Ollama: one-command ollama run llama3.2 — downloads, caches, and serves prebuilt model variants. Built on llama.cpp. LM Studio: GUI wrapper around llama.cpp with a model browser. llama.cpp directly: for custom configurations and integrations.

Ollama is common for local/developer iteration, while vLLM is commonly used when higher serving throughput is required. This split lets teams prototype quickly, then benchmark serving behavior before promoting workloads.

vLLM, TensorRT-LLM, and SGLang are widely used production inference options with different performance and integration tradeoffs. Engine selection usually depends on hardware stack, latency targets, and operational tooling preferences.

A widely used AI application framework that provides chains, agents, memory patterns, and tool integrations with observability support. Best for: multi-step workflows, agent orchestration, quick prototyping.

RAG-first framework with the deepest retrieval tooling. LlamaParse for complex document parsing, 9 vector DB integrations, advanced strategies (CRAG, Self-RAG, HyDE, RAPTOR). Best for: RAG applications, document intelligence, agentic workflows over structured data.

Stanford's framework that treats LLM programs as optimizable code. Instead of writing prompts by hand, you write modules that DSPy compiles into optimized prompts and few-shot examples. Best for: production-grade pipelines where prompt quality matters systematically.

A. Foundation: HF Hub, libraries, open models — the lay of the land
B. Local AI: GGUF, Ollama, llama.cpp, LM Studio — run AI on your hardware
C. Fine-Tuning: LoRA, QLoRA, Unsloth, Axolotl — adapt models to your data
D. Production: vLLM, TGI, SGLang — serve thousands of requests per second
E. Applications: LangChain, LlamaIndex, DSPy, end-to-end stack

By the end of this course, you'll understand every layer of the open source AI stack — from how models are stored and quantized, to running them locally, fine-tuning them on your data, serving them in production, and building applications on top of them.