Fine-tuning takes a pre-trained LLM (like Llama 3, Mistral, or GPT-4o) and continues training it on your own dataset. The model learns your specific patterns, terminology, style, and tasks. Think of it as taking a generally educated person and giving them specialized job training.

Pre-training: Train from scratch on trillions of tokens from the internet. Costs millions of dollars and takes months on thousands of GPUs. Creates a "base model" that can predict the next token but doesn't follow instructions.

Fine-tuning: Continue training on thousands to millions of task-specific examples. Costs $10 to $10,000 depending on model size and method. Takes hours to days on 1-8 GPUs. Creates a specialized model that excels at your use case.

1. Task specialization: Make the model excel at a specific task (code generation, medical Q&A, legal analysis, structured extraction).

2. Style and tone: Match your brand voice, writing style, or communication patterns consistently.

3. Domain knowledge: Teach specialized vocabulary and reasoning patterns (finance, healthcare, law).

4. Cost reduction: A fine-tuned 8B model can match GPT-4 quality on your specific task at 1/100th the inference cost.

5. Latency: Smaller fine-tuned models respond faster than large general-purpose ones.

6. Privacy: Run the model on your own infrastructure. No data leaves your environment.

What: Write instructions and examples in the prompt. No model weights change.
Best for: General tasks, rapid iteration, when you have a strong base model (GPT-4o, Claude).
Limits: Context window constrains how much you can teach. Inconsistent on complex tasks. Expensive at scale (long prompts = more tokens).

What: Retrieve relevant documents and inject them into the prompt at query time.
Best for: Grounding answers in up-to-date or proprietary data. Reducing hallucination on factual questions.
Limits: Doesn't change model behavior or style. Retrieval quality is the bottleneck. Adds latency.

What: Train the model on your data. Model weights change permanently.
Best for: Consistent behavior, specific output formats, domain expertise, cost optimization at scale.
Limits: Requires training data and compute. Can't easily update knowledge (use RAG for that). Risk of catastrophic forgetting.

Update all parameters in the model. Maximum flexibility but requires the most memory and compute. For a 7B model: ~56 GB GPU memory (fp16 weights + optimizer states + gradients). Typically needs 4-8 A100 GPUs.

When to use: Large high-quality dataset (100K+ examples), sufficient compute budget, need maximum quality.

Freeze the base model. Train small adapter matrices (typically 0.1-1% of total parameters). Dramatically reduces memory: fine-tune a 7B model on a single GPU with 16 GB VRAM.

When to use: Most use cases. Best quality-to-cost ratio. The default recommendation for 2024-2025.

Load the base model in 4-bit precision (NF4 quantization), then train LoRA adapters on top. Fine-tune a 70B model on a single 48 GB GPU (A100/A6000). Introduced by Dettmers et al. (2023).

When to use: Limited GPU memory, large models (30B-70B), prototyping.

Prefix Tuning: Prepend trainable virtual tokens to the input. Very few parameters but limited expressiveness.

Prompt Tuning: Similar to prefix tuning but only at the input layer. Google's approach for T5 models.

IA3 (Infused Adapter by Inhibiting and Amplifying Inner Activations): Learns rescaling vectors instead of matrices. Even fewer parameters than LoRA.

DoRA (Weight-Decomposed Low-Rank Adaptation): Decomposes weights into magnitude and direction, applies LoRA to direction only. Often outperforms standard LoRA.

Train on trillions of tokens from the internet. The model learns language, facts, reasoning, and code. Output: a base model (e.g., Llama 3 base). It can complete text but doesn't follow instructions. This stage costs millions of dollars and is done by labs (Meta, Google, Mistral, etc.).

Train on instruction/response pairs. The model learns to follow instructions, answer questions, and produce structured output. Output: an instruct model (e.g., Llama 3 Instruct). This is what most people mean by "fine-tuning." Costs $10-$10,000. This is the stage you'll do most often.

Train the model to prefer helpful, harmless, and honest responses using human preference data. Methods: RLHF (Reinforcement Learning from Human Feedback) or DPO (Direct Preference Optimization). Output: an aligned/chat model. Makes the model safer and more useful in conversation.

The LIMA paper (Zhou et al., 2023) showed that just 1,000 carefully curated examples can produce a model competitive with GPT-3.5 on many tasks. Microsoft's Phi models demonstrated that high-quality synthetic data can outperform much larger datasets of lower quality.

The lesson: 1,000 excellent examples beat 100,000 mediocre ones.

Minimum viable: 100-500 examples for style/format transfer.
Good baseline: 1,000-5,000 examples for task specialization.
Strong model: 10,000-50,000 examples for complex reasoning tasks.
Maximum quality: 50,000-500,000 examples for full fine-tuning on broad tasks.

More data helps, but with diminishing returns. The first 1,000 examples matter most.

Fine-tuning data is typically instruction/response pairs:

The model forgets general capabilities while learning your specific task. A model fine-tuned on legal text might lose its ability to write code or do math. Mitigation: use LoRA (base model stays frozen), keep learning rate low, mix in general-purpose data.

The model memorizes training examples instead of learning patterns. It performs perfectly on training data but poorly on new inputs. Mitigation: use a validation set, early stopping, regularization (weight decay, dropout), and don't train for too many epochs (1-3 is typical).

Garbage in, garbage out. If your training data has errors, inconsistencies, or biases, the model will learn them. Always review a random sample of your training data before training. One bad pattern repeated 100 times can dominate the model's behavior.

Fine-tuning can undo safety alignment. Research has shown that even a small number of harmful examples can remove safety guardrails from aligned models. Always evaluate safety after fine-tuning. Consider re-applying alignment (DPO) after SFT.

Model licenses matter. Llama 3 has a community license (free for most uses, restrictions for 700M+ monthly active users). Mistral models are Apache 2.0 (fully permissive). Phi-3 is MIT licensed. GPT-4o fine-tuning is API-only (OpenAI hosts it). Always check the license before fine-tuning and deploying.

1. Consistent output format: You need the model to always produce JSON, XML, or a specific structure that prompting can't reliably enforce.

2. Domain-specific behavior: Medical diagnosis, legal analysis, financial modeling — tasks requiring specialized reasoning patterns.

3. Cost optimization: You're spending $10K+/month on GPT-4o API calls and a fine-tuned 8B model could handle 80% of queries at 1/100th the cost.

4. Latency requirements: You need sub-100ms responses that only a small, local model can provide.

5. Privacy: Data cannot leave your infrastructure. You need a self-hosted model.

1. Prompting works: If few-shot prompting with GPT-4o gives you 95% accuracy, fine-tuning won't add much.

2. You need up-to-date knowledge: Fine-tuning doesn't update facts well. Use RAG instead.

3. Small dataset (<100 examples): Not enough signal for the model to learn meaningful patterns.

4. Rapidly changing requirements: If your task changes weekly, retraining is impractical. Use prompting.

5. No evaluation plan: If you can't measure whether fine-tuning improved the model, don't do it.