Pick proprietary (OpenAI, Anthropic, Google, Cohere) when
- Speed to market is dominant. Their APIs ship in a day.
- Highest reasoning quality is required and you can pay for it.
- You don't have DevOps capacity for GPU pools, autoscaling, batching.
- Multimodal frontier (audio, video) — open models lag here meaningfully.
- Fine-tuning isn't needed or vendor offering is sufficient.
Pick open source (Llama, Qwen, Mistral, DeepSeek, Granite) when
- Data privacy forbids sending data to vendors (banking, healthcare, regulated, classified).
- Cost at scale matters — at 100M+ tokens/day, self-hosted is 10-100× cheaper.
- Customization is core — heavy fine-tuning, RLHF, DPO, LoRA stacks.
- Air-gapped environments (defense, regulated industrial).
- No vendor lock, model evolution under your control.
- Provenance required for audit (you control training data and the deployed weights).
The hybrid pattern (what most serious teams actually run)
- Cheap, simple tasks (classification, routing, structured extraction): self-hosted small open model (Llama 3.x 8B, Qwen 7B). Free at scale.
- Hard generation (the user-facing answer): proprietary frontier (GPT-4o, Claude, Gemini).
- Embedders & re-rankers: self-hosted open (BGE-M3, Cohere if vendor ok).
- Sensitive data tasks: stays on-prem with open-source.
- Public / low-stakes: vendor for speed.
The cost cliff math
Rough numbers, May 2026:
- GPT-4o: ~$5/Mtok input, $15/Mtok output.
- Claude Sonnet 4.x: ~$3 input, $15 output.
- Gemini 3.x Pro: ~$1.25 input, $5 output.
- Self-hosted Llama 4 70B on 8×H100: ~$0.05-0.20/Mtok all-in, given high utilization.
The crossover is not at low volume. It's at sustained high volume with high utilization. Below ~50M tokens/day, vendors are usually cheaper than your DevOps salary. Above 1B tokens/day, self-hosting saves millions.
The migration risk
If you build on a vendor and later need to leave, your prompts, fine-tunes, and behaviors won't transfer one-to-one. Plan for portability:
- Use abstractions like
LiteLLMfrom day one. - Keep an eval set that runs against multiple models.
- Don't over-fit prompt syntax to a single provider.
The vendor risk
Vendors change pricing, deprecate models, change rate limits, change ToS. If your business depends on it, your business depends on their stability. Open weights eliminate this risk entirely.