Most AI services are priced per token — roughly ¾ of a word. Every interaction has a cost: the tokens you send in (input) and the tokens the model generates (output). Output tokens cost 3–5× more than input tokens because they require sequential processing. A 1,000-word prompt with a 500-word response costs roughly 2,000 tokens total — but the output portion costs disproportionately more.

Budget tier — Gemini Flash-Lite: $0.075/M input tokens. DeepSeek-V3: $0.14/M. Suitable for high-volume, simple tasks.
Production tier — GPT-5.1: $1.25/M input. Claude Sonnet: $3.00/M. Gemini Pro: $1.25/M. The enterprise workhorse.
Frontier tier — Claude Opus: $5.00/M input. GPT-5.2 Pro: $21.00/M input, $168.00/M output. For complex reasoning where quality justifies the premium.

The spread is 10–100× between cheapest and most expensive. Model selection is the single largest cost lever.

AI costs are declining at an unprecedented rate. GPT-4 equivalent performance dropped from $20/M tokens (late 2022) to $0.40 (December 2025) — a 50× reduction in three years. This is faster than Moore’s Law. DeepSeek disrupted the market further with 90% lower pricing than Western competitors. The implication: what costs $100,000 today will cost $10,000 next year and $1,000 the year after.

When executives budget for AI, they typically account for API or licensing costs. But hidden costs account for 40–60% of total AI investment. The full cost structure:

Visible costs (40–60%)
• API/token costs or GPU infrastructure
• Software licenses and platform fees
• AI engineering salaries

Hidden costs (40–60%)
• Data preparation and curation
• Integration with existing systems
• Change management and training
• Monitoring and maintenance
• Human review and quality assurance
• Compliance and governance overhead

Large organizations processing 5–50 billion tokens monthly face $45,000–$1,000,000/month in API costs alone. But the total cost of an enterprise AI program — including talent, infrastructure, data preparation, and governance — is typically 3–5× the API cost. A $500K annual API budget implies a $1.5M–$2.5M total program cost.

56% of companies report no significant financial returns from AI investments — no revenue increase, no cost reduction. Only 12% of CEOs confirm AI delivers both cost and revenue benefits. MIT research found a 95% failure rate for enterprise GenAI projects. Only 6% of organizations qualify as “AI high performers” with measurable impact. Nearly two-thirds have not begun scaling AI across the enterprise.

Global AI investment is forecast to hit $2.52 trillion in 2026. If 56% of that generates no returns, over $1.4 trillion is being spent with no measurable outcome. This is not a technology failure — it’s a strategy and execution failure. The technology works. The organizational readiness to capture value from it does not.

Formal AI strategy — Companies with formal strategies report 80% success rates vs. 37% without. Strategy means defined use cases, clear KPIs, executive sponsorship, and governance frameworks.

Start narrow, prove value, then scale — High performers begin with 1–3 high-impact use cases, demonstrate measurable ROI, then expand systematically.

Invest in data infrastructure — 47% of organizations cite data infrastructure as a barrier. High performers fix their data before building AI on top of it.

Measure relentlessly — Track cost per task automated, time saved per employee, error rate reduction, and customer satisfaction impact.

Route tasks to the cheapest model that meets quality requirements. Simple classification tasks don’t need GPT-5 — Gemini Flash at $0.075/M tokens handles them fine. Complex reasoning tasks justify Claude Opus at $5.00/M. A smart routing layer that matches task complexity to model capability can reduce costs by 60–80% with minimal quality impact.

Shorter, more efficient prompts reduce token consumption. Optimized prompts use 30–50% fewer tokens than naive ones (Chapter 16). At enterprise scale, this translates directly to cost savings. A prompt that uses 500 tokens instead of 1,000 cuts your input cost in half across millions of requests.

Prompt caching saves up to 90% on repeated input patterns. If your system prompt is 2,000 tokens and every request includes it, caching that prefix eliminates the cost of re-processing it. Semantic caching goes further: if a similar question was asked recently, return the cached answer without calling the model at all. Reduces LLM costs by up to 68.8% in production workloads.

4. Quantization — Reduce model precision from 32-bit to 8-bit or 4-bit. Cuts operational costs 60–70% with minimal quality loss for most tasks.

5. Knowledge distillation — Train a small model to mimic a large one on your specific task. 90–95% quality at 25× lower cost (Chapter 15).

6. Batching — Group non-urgent requests and process them together during off-peak hours at lower rates.

7. Output constraints — Limit output length and use structured output formats. A JSON response with 5 fields costs far less than a 500-word free-text response.

Variable cost, zero infrastructure. You pay per token with no upfront investment. Costs scale linearly with usage. At low-to-medium volume (<10M tokens/day), APIs are almost always cheaper. A team processing 2M tokens/day pays ~$620/month. The simplicity is the value: no GPU procurement, no model serving, no operational overhead. The risk: costs can spike unpredictably with usage growth.

Fixed cost, full control. An NVIDIA H100 costs $25,000–$40,000 per card. An 8-GPU server: $200,000–$400,000. Cloud GPU rental: $2.85–$3.50/hour (H100), stabilized after a 64–75% decline. Self-hosting achieves 60–70% of cloud API costs at scale, but requires 50%+ GPU utilization to break even. Below that, you’re paying for idle capacity.

<10M tokens/day → APIs. The math is clear.
10–50M tokens/day → Hybrid. APIs for variable workloads, self-hosted for steady-state production tasks.
>50M tokens/day → Self-hosted for primary workloads. APIs for overflow and frontier model access.
Data sovereignty required → Self-hosted regardless of volume. The compliance requirement overrides the cost calculation.

Financial services — 4.2× ROI on GenAI investments. Fraud detection, risk assessment, compliance automation, and customer service.
Technology — AI coding tools deliver 15%+ developer velocity gains. Cursor alone generates $500M–$1B ARR from developer productivity.
Healthcare — Clinical documentation automation, diagnostic support, and drug discovery. Abridge and similar tools reduce physician documentation time by 50–70%.

IT operations automation — 50% cost reduction in documented cases.
Customer service — Klarna: $40M annual savings. Salesforce: 83% resolution rate with 1% escalation.
Content generation — 90% reduction in production time for marketing content.
Code generation — 40% developer productivity increase (JPMorgan Chase).

66% of organizations report productivity gains, but translating productivity into financial returns is the challenge. Employees save time, but if that time isn’t redirected to higher-value work, the savings don’t appear on the P&L. The organizations seeing real ROI are those that redesign workflows around AI, not those that simply add AI to existing workflows.

For organizations beginning their AI journey, a balanced allocation:

30% — Data & infrastructure
Data preparation, vector databases, integration middleware, cloud compute. This is the foundation everything else depends on.

25% — Talent & training
AI engineers, prompt engineers, workforce upskilling. AI engineering salaries grew 56% between 2023–2025. Budget accordingly.

20% — AI services & APIs
Token costs, platform licenses, SaaS subscriptions. The most visible but often not the largest cost category.

15% — Governance & operations
Monitoring, evaluation, compliance, security, human review processes. Under-investment here is the primary cause of AI project failure.

10% — Experimentation & innovation
Proof-of-concept projects, emerging technology evaluation, hackathons. This is your option value — the investment that discovers next year’s high-ROI use cases.

1. What is our cost per AI-assisted task? — Not just token costs, but total cost including human review, rework, and infrastructure.

2. What is our AI utilization rate? — Are we paying for capacity we’re not using? GPU utilization below 50% means you’re overpaying for self-hosted infrastructure.

3. Are we routing to the right model tier? — Simple tasks on premium models waste 10–100× the necessary cost.

4. What are our hidden costs? — Data preparation, integration, change management, compliance. If you can’t quantify these, you’re underestimating your AI investment by 40–60%.

5. Do we have a formal AI strategy? — 80% success rate with strategy vs. 37% without. This is the highest-leverage investment.

6. Are we measuring the right outcomes? — Track cost per task, time saved, error reduction, and customer impact — not just “AI adoption rate.”

7. When should we re-evaluate build vs. buy? — At least twice a year. The economics shift quarterly.

8. What is our AI cost trajectory? — Per-unit costs should decline. Total costs may increase as usage grows. Both trends are healthy.

9. Are we capturing indirect value? — 55% of long-term value is indirect: speed, quality, employee satisfaction, competitive positioning.

10. What would we stop doing? — The hardest question. Which current AI investments are not delivering and should be redirected?