Frontier vs Open: How to Choose a Model

A practical decision framework for builders: when to reach for a frontier closed model, an open-weight model, or a fine-tune - based on cost, control, latency, and task type.

TL;DR: Use a frontier API when you need maximum capability out of the box, fast iteration, or multimodal reasoning. Use an open-weight model when you need data control, predictable inference cost at scale, or the ability to self-host. Fine-tune when you have a high-volume, repetitive task where consistent output format and domain tone matter more than general intelligence. Most production systems end up using all three.

The Three Families

Before the decision framework, a quick map of what you are actually choosing between.

Frontier closed models

These are models you access through an API. The weights are never shared. The provider runs inference on their infrastructure, and you pay per token. The current tier leaders:

Open-weight models

The weights are public. You can download, run, and modify them. You pay for compute, not tokens. The practical leaders right now:

Fine-tuned models

A fine-tune is not a separate family - it is a starting point (usually an open-weight model) that you continue training on your own data to encode domain-specific behavior, tone, output format, or decision boundaries. LoRA and QLoRA have made this accessible: a QLoRA run on a 13B model can finish for under $100 in raw GPU time. The expensive part is data curation and evaluation, not compute.

The Five Questions That Drive the Decision

Run through these in order. The first one that has a clear answer usually determines the choice.

1. Does your data leave your network?

Every frontier API call routes your prompt - including any user data in it - to a third-party server. For most creative and productivity tasks that is fine. For anything involving PHI, PII, legal documents, or proprietary IP under contract, it is often not. If your legal or compliance team says no data to external servers, you are running open-weight on your own infrastructure, full stop. No framework will override that constraint.

2. What is the task shape?

This is the most important technical question. Ask: is this task novel each time or the same pattern repeated?

A useful diagnostic: if you can write an eval for the task with clear right/wrong answers, it is probably a fine-tune candidate. If the eval requires human judgment every time, stick with frontier.

3. What is your call volume?

At low volume (under a few million tokens per month), the economics of a frontier API are hard to beat - no infra to maintain, no GPU to provision, no model to keep updated. At high volume, the math flips. Self-hosting an open-weight model eliminates per-token cost entirely and replaces it with fixed compute cost. The break-even point depends on your specific model choice and GPU costs, but the pattern is consistent: low volume favors APIs, high volume favors self-hosting.

There is also a middle option - running open-weight models through a managed inference provider like Groq (which delivers roughly 500+ tokens/sec on Llama 4 Scout using custom LPU hardware) or Together AI. You get open-weight pricing without managing your own cluster.

4. How much does latency matter?

For human-facing, real-time interactions - a chat interface, a voice agent, a live coding assistant - time-to-first-token matters. Frontier APIs vary widely here. Smaller models (Haiku 4.5, Gemini 3.1 Flash-Lite, GPT-4o mini) are faster than their larger siblings. Groq's LPU hardware delivers sub-300ms time-to-first-token on Llama 4 Scout, which is competitive with most frontier API response times.

For async pipelines - nightly batch jobs, document processing, data enrichment - latency does not matter and cost does. Use Anthropic's Batch API (50% off standard pricing), OpenAI's batch endpoint, or Gemini's batch mode. These are designed for this exact shape.

5. Do you need reproducibility and version lock?

Frontier API providers update their models. A provider controls when the swap happens. For regulated outputs, audit trails, or pipelines where output drift causes downstream bugs, open-weight models give you a frozen artifact you can pin and version-control. You run the same weights in the same quantization forever.

A Decision Matrix

Situation                            -> Reach for
--------------------------------------------------------------
Novel task, broad reasoning needed   -> Frontier API (Opus 4.8, GPT-5.5, Gemini 3.1 Pro)
Fast, cheap, capable everyday tasks  -> Mid-tier API (Sonnet 4.6, Gemini 3.5 Flash)
Need blazing throughput on a budget  -> Frontier Haiku / Flash-Lite or Groq+Llama
Data cannot leave your network       -> Self-hosted open-weight (Llama, Mistral)
Same pattern, millions of calls/mo   -> Fine-tuned small open-weight model
Output must be frozen, auditable     -> Self-hosted open-weight, pinned version
Exploring fast, costs uncertain      -> Frontier API, then optimize later

The Reasoning Model Exception

OpenAI's o-series and Anthropic's adaptive thinking (available on Opus 4.8, Sonnet 4.6, and Fable 5) are a distinct category. They run an internal chain-of-thought before producing output, which costs additional tokens you pay for at output rates. On o3, a single complex answer can burn 5,000 to 20,000 internal reasoning tokens on top of the visible response - making it significantly more expensive than the listed rate implies.

Reasoning models are worth the cost for tasks where correctness is expensive to verify and errors are costly - complex code generation, multi-step math, legal analysis, scientific reasoning. For tasks where "pretty good" is good enough, standard models at a tenth of the price make more sense. Do not default to a reasoning model just because it is available; be deliberate.

Fine-Tuning: When and How

Fine-tuning is the right tool for a specific problem: stable behavior that prompting cannot reliably produce. Good candidates include consistent output format across thousands of edge cases, domain terminology that general models handle inconsistently, and brand voice or compliance language that needs to be predictable at scale.

The 2026 consensus in production teams is a hybrid architecture: frontier models handle volatile, reasoning-heavy work; fine-tuned open-weight models handle high-volume repeating patterns behind a private gateway. This is not a compromise - it is the right shape for the problem.

To fine-tune practically today:

# Typical LoRA fine-tune workflow (Hugging Face + transformers)
# 1. Pick a base model (e.g., meta-llama/Llama-4-Scout-17B-16E-Instruct)
# 2. Prepare 500-5,000 input/output pairs in JSONL format
# 3. Run QLoRA training (fits on a single A100 or H100)
# 4. Merge and quantize the adapter for serving

pip install transformers peft datasets bitsandbytes
# See Hugging Face PEFT docs for the full training script

The actual training compute cost is low. The data curation and eval work is where the real investment goes - plan for it.

Cost Levers You Should Know

Before concluding a frontier API is too expensive for your use case, check these levers first:

Key Takeaways

Try this next: Now that you understand the model landscape, see Prompt Engineering Fundamentals to learn how to get the most out of whichever model you choose.

LearnlandscapeFrontier vs Open: How to Choose a Model
Guidelandscapecore8 min read

Frontier vs Open: How to Choose a Model

A practical decision framework for builders: when to reach for a frontier closed model, an open-weight model, or a fine-tune - based on cost, control, latency, and task type.

CREATORS OF TODAY.
@creatorsoftoday
319

TL;DR: Use a frontier API when you need maximum capability out of the box, fast iteration, or multimodal reasoning. Use an open-weight model when you need data control, predictable inference cost at scale, or the ability to self-host. Fine-tune when you have a high-volume, repetitive task where consistent output format and domain tone matter more than general intelligence. Most production systems end up using all three.

The Three Families

Before the decision framework, a quick map of what you are actually choosing between.

Frontier closed models

These are models you access through an API. The weights are never shared. The provider runs inference on their infrastructure, and you pay per token. The current tier leaders:

  • Anthropic Claude - Fable 5 at the top ($10/$50 per MTok input/output), Opus 4.8 for complex reasoning ($5/$25 per MTok), Sonnet 4.6 for everyday production ($3/$15 per MTok), Haiku 4.5 for high-volume lightweight tasks ($1/$5 per MTok). Fable 5, Opus 4.8, and Sonnet 4.6 carry a 1M token context window; Haiku 4.5 has a 200K context window.
  • OpenAI GPT series - GPT-5.5 is the current flagship (released April 2026) for broad capability work; the o-series reasoning models (o3, o4-mini) handle complex reasoning tasks. o3 costs roughly $2/$8 per MTok input/output - but reasoning models burn additional hidden tokens for internal chain-of-thought, so real bills are higher than the listed rate suggests.
  • Google Gemini - Gemini 3.5 Flash ($1.50/$9.00 per MTok) leads on speed and grounding; Gemini 3.1 Pro Preview ($2.00/$12.00 per MTok) is the deep-reasoning option. Gemini 3.1 Flash-Lite at $0.25/$1.50 per MTok is one of the cheapest capable APIs available.

Open-weight models

The weights are public. You can download, run, and modify them. You pay for compute, not tokens. The practical leaders right now:

  • Meta Llama 4 - Scout (17B active parameters, 16 experts, 10M token context, fits on one H100 with Int4 quantization) and Maverick (17B active, 128 experts, 400B total parameters). Both are free to download from Hugging Face and llama.com.
  • Mistral - Mistral Small 4 (6B active, 119B total, MoE architecture, Apache 2.0 license) and Mistral Large 3. Mistral publishes their weights openly and runs a commercial API on top.
  • Qwen, DeepSeek, Gemma - a deep bench of capable open models that cover the range from 3B to 70B+ parameters with permissive licenses.

Fine-tuned models

A fine-tune is not a separate family - it is a starting point (usually an open-weight model) that you continue training on your own data to encode domain-specific behavior, tone, output format, or decision boundaries. LoRA and QLoRA have made this accessible: a QLoRA run on a 13B model can finish for under $100 in raw GPU time. The expensive part is data curation and evaluation, not compute.

The Five Questions That Drive the Decision

Run through these in order. The first one that has a clear answer usually determines the choice.

1. Does your data leave your network?

Every frontier API call routes your prompt - including any user data in it - to a third-party server. For most creative and productivity tasks that is fine. For anything involving PHI, PII, legal documents, or proprietary IP under contract, it is often not. If your legal or compliance team says no data to external servers, you are running open-weight on your own infrastructure, full stop. No framework will override that constraint.

2. What is the task shape?

This is the most important technical question. Ask: is this task novel each time or the same pattern repeated?

  • Novel, open-ended, or reasoning-heavy - drafting a strategy memo from scratch, debugging a novel error, synthesizing a research topic, generating creative concepts. Frontier APIs win here. General intelligence trained on broad internet-scale data handles open-ended work better than a narrow specialist.
  • High-volume, consistent-pattern - classifying support tickets, extracting structured fields from invoices, moderating content, generating product descriptions that must match a brand voice. A fine-tuned smaller model frequently beats a frontier API on your specific task at a fraction of the per-call cost.

A useful diagnostic: if you can write an eval for the task with clear right/wrong answers, it is probably a fine-tune candidate. If the eval requires human judgment every time, stick with frontier.

3. What is your call volume?

At low volume (under a few million tokens per month), the economics of a frontier API are hard to beat - no infra to maintain, no GPU to provision, no model to keep updated. At high volume, the math flips. Self-hosting an open-weight model eliminates per-token cost entirely and replaces it with fixed compute cost. The break-even point depends on your specific model choice and GPU costs, but the pattern is consistent: low volume favors APIs, high volume favors self-hosting.

There is also a middle option - running open-weight models through a managed inference provider like Groq (which delivers roughly 500+ tokens/sec on Llama 4 Scout using custom LPU hardware) or Together AI. You get open-weight pricing without managing your own cluster.

4. How much does latency matter?

For human-facing, real-time interactions - a chat interface, a voice agent, a live coding assistant - time-to-first-token matters. Frontier APIs vary widely here. Smaller models (Haiku 4.5, Gemini 3.1 Flash-Lite, GPT-4o mini) are faster than their larger siblings. Groq's LPU hardware delivers sub-300ms time-to-first-token on Llama 4 Scout, which is competitive with most frontier API response times.

For async pipelines - nightly batch jobs, document processing, data enrichment - latency does not matter and cost does. Use Anthropic's Batch API (50% off standard pricing), OpenAI's batch endpoint, or Gemini's batch mode. These are designed for this exact shape.

5. Do you need reproducibility and version lock?

Frontier API providers update their models. A provider controls when the swap happens. For regulated outputs, audit trails, or pipelines where output drift causes downstream bugs, open-weight models give you a frozen artifact you can pin and version-control. You run the same weights in the same quantization forever.

A Decision Matrix

Situation                            -> Reach for
--------------------------------------------------------------
Novel task, broad reasoning needed   -> Frontier API (Opus 4.8, GPT-5.5, Gemini 3.1 Pro)
Fast, cheap, capable everyday tasks  -> Mid-tier API (Sonnet 4.6, Gemini 3.5 Flash)
Need blazing throughput on a budget  -> Frontier Haiku / Flash-Lite or Groq+Llama
Data cannot leave your network       -> Self-hosted open-weight (Llama, Mistral)
Same pattern, millions of calls/mo   -> Fine-tuned small open-weight model
Output must be frozen, auditable     -> Self-hosted open-weight, pinned version
Exploring fast, costs uncertain      -> Frontier API, then optimize later

The Reasoning Model Exception

OpenAI's o-series and Anthropic's adaptive thinking (available on Opus 4.8, Sonnet 4.6, and Fable 5) are a distinct category. They run an internal chain-of-thought before producing output, which costs additional tokens you pay for at output rates. On o3, a single complex answer can burn 5,000 to 20,000 internal reasoning tokens on top of the visible response - making it significantly more expensive than the listed rate implies.

Reasoning models are worth the cost for tasks where correctness is expensive to verify and errors are costly - complex code generation, multi-step math, legal analysis, scientific reasoning. For tasks where "pretty good" is good enough, standard models at a tenth of the price make more sense. Do not default to a reasoning model just because it is available; be deliberate.

Fine-Tuning: When and How

Fine-tuning is the right tool for a specific problem: stable behavior that prompting cannot reliably produce. Good candidates include consistent output format across thousands of edge cases, domain terminology that general models handle inconsistently, and brand voice or compliance language that needs to be predictable at scale.

The 2026 consensus in production teams is a hybrid architecture: frontier models handle volatile, reasoning-heavy work; fine-tuned open-weight models handle high-volume repeating patterns behind a private gateway. This is not a compromise - it is the right shape for the problem.

To fine-tune practically today:

# Typical LoRA fine-tune workflow (Hugging Face + transformers)
# 1. Pick a base model (e.g., meta-llama/Llama-4-Scout-17B-16E-Instruct)
# 2. Prepare 500-5,000 input/output pairs in JSONL format
# 3. Run QLoRA training (fits on a single A100 or H100)
# 4. Merge and quantize the adapter for serving

pip install transformers peft datasets bitsandbytes
# See Hugging Face PEFT docs for the full training script

The actual training compute cost is low. The data curation and eval work is where the real investment goes - plan for it.

Cost Levers You Should Know

Before concluding a frontier API is too expensive for your use case, check these levers first:

  • Prompt caching - Anthropic's prompt caching drops cached input tokens to 10% of standard price. If you repeat the same large system prompt or document across many calls, this alone can cut costs by 70-90% on those tokens.
  • Batch API - All major providers offer async batch endpoints at 50% off. If your workload tolerates 24-hour turnaround, this is free money.
  • Model tier - Gemini 3.1 Flash-Lite at $0.25/$1.50 per MTok and Claude Haiku 4.5 at $1.00/$5.00 per MTok are capable models for structured tasks. Defaulting to a top-tier model for every call is a common and expensive mistake.
  • Quantization - If self-hosting open-weight models, Int4 quantization typically cuts VRAM requirements by 4x with minimal quality loss on most tasks. Llama 4 Scout fits on a single H100 at Int4.

Key Takeaways

  • Data sovereignty is the first gate: if your data cannot leave your network, the decision is already made - go open-weight and self-hosted.
  • Task shape matters more than raw capability: novel open-ended tasks favor frontier APIs; high-volume repeating patterns favor fine-tuned smaller models.
  • Volume determines economics: frontier APIs win at low scale; self-hosted open-weight models win at high scale. The crossover depends on your specific model and GPU costs.
  • Reasoning models (o3, adaptive thinking on Opus 4.8 and Sonnet 4.6) are a separate tier with hidden token costs - use them deliberately for tasks where correctness is expensive to verify.
  • Exhaust prompt caching and batch API before concluding a frontier API is too expensive for your use case.
  • The best production systems use all three types: frontier for reasoning, open-weight for control, fine-tunes for volume.

Try this next: Now that you understand the model landscape, see Prompt Engineering Fundamentals to learn how to get the most out of whichever model you choose.

References & sources

Reviews

Only verified humans can leave reviews. It keeps every rating real.

Verify to review

No reviews yet. Be the first to share your take.