Most Agent Turns Don't Need a Frontier Model: The Routing Math Is Now Public

For the last year, the default posture for production agents has been “pick a strong model and pay for every turn.” The week ending August 15, 2026 put numbers behind an alternative that practitioners have been circling for months: route turn-by-turn, and let the frontier model touch only the minority of steps that actually need it. Three independent releases converge on the same distribution — roughly 90%+ of turns are handled fine by a mid-size model, and the cost delta is large enough that not routing is now the expensive choice.

The 7% number

LangChain benchmarked NVIDIA’s Switchyard routing library against their Deep Agents eval suite — 145 multi-step tasks — and reported that only 7% of turns required Claude Opus 4.8. A 30B model handled the remaining 93%. The composite result: 74% cost reduction against roughly six points of accuracy loss.

That 7% figure is the interesting one. It is not a claim about routing between agents (which the archive has covered as a recommendation problem); it is a claim about routing between models within a single agent’s execution loop. The turns that need a frontier model are the planning inflections, the ambiguous tool-result interpretations, the recovery-from-failure moments. Everything else — argument construction, result validation, subagent dispatch, formatting — is executed by cheaper weights.

NVIDIA’s own release of Nemotron 3.5 Lightning is engineered around exactly this thesis. It’s a 30B MoE with 3B active parameters, explicitly positioned for “the high-volume execution layer of long-running agents” — tool calls, result validation, subagent delegation. The pitch is not that it competes with frontier models; it’s that it’s the right substrate for the 93%.

The same distribution shows up in production cost work

Connor Heggie’s account of Unify cutting agent costs 95% in two weeks is the practitioner-side view of the same distribution. The specific levers he names — prompt cache hit-rate optimization, the 15-requests-per-second OpenAI cache ceiling, subagents-as-function-calls, keyed memory — are all techniques for making the cheap path cheaper and the expensive path rare. A 95% cost cut without a quality regression only works if the expensive model was doing very little of the actual reasoning that mattered.

A related note from the same conversation: the LLM judge must be a different model family than the model being judged. That constraint has evaluation implications, but it also has routing implications — the judge is another node in the routing graph, and its economics are governed by the same distribution.

Why the economics have finally tipped

Two things had to become true for intra-agent routing to work as a default. First, 30B-class open models had to become good enough at tool use that they don’t fail silently on the majority of turns handed to them. Nemotron 3.5 Lightning and Meta’s Muse Glimmer — both released in the week ending August 15, 2026, both 30B, both explicitly positioned for local agentic workflows including multi-step tool use and failure recovery — are the concrete evidence that this threshold has been crossed. Muse Glimmer runs under 20GB with 4-bit quantization; that changes the deployment topology for the cheap path entirely.

Second, the routing decision itself had to become cheap and reliable. Switchyard is the specific artifact here, but the broader pattern is that routers are becoming standard components in production agent runtimes rather than research curiosities. Anthropic’s Managed Agents post frames session, harness, and sandbox as swappable interfaces — a design that only pays off if the model behind the interface is also swappable per turn.

What this does to agent architecture

The practical consequence is that agent design now has two loops, not one. There’s the semantic loop the agent runs (plan, act, observe, revise), and there’s a routing loop underneath it that assigns each step to a model tier. Practitioners who’ve been treating model choice as a deployment-time constant are now treating it as a per-call decision.

A few implications worth being explicit about:

• Prompt caching becomes tier-specific. Unify’s cache work only pays off if the same tier keeps hitting the same prefixes. Routing that flips models mid-conversation destroys cache locality. The 15-requests-per-second cache ceiling Heggie mentions is a real constraint on how aggressively you can shard traffic.
• Evaluation has to be per-tier, not just end-to-end. If Opus handles 7% of turns and a 30B model handles 93%, an end-to-end eval can hide a regression in either. LangSmith’s Engine, which clusters production failures into issues and proposes fixes, is going to spend most of its output on the cheap tier simply because that’s where most of the turns are.
• The “different family” judge constraint compounds. If your executor is one 30B model, your planner is Opus, and your judge must be a different family than both, you now have three routing endpoints minimum, each with its own cost profile.
• Benchmarks measured on a single model no longer describe your system. Even Gemini 3.7 Flash’s AutomationBench and AnalystAgent results describe a single-model configuration; a routed system’s number will look nothing like it.

The six-point accuracy tax deserves scrutiny

The LangChain benchmark reports 74% cost reduction against ~6 points of accuracy loss. That trade is not automatically a win — it depends entirely on the shape of the failure. If the six points are randomly distributed across easy tasks, you probably fix them with better routing rules and get most of them back. If they’re concentrated on a specific failure mode (early hypothesis anchoring is a candidate), routing them to the frontier tier is straightforward.

What the six points almost certainly are not is uniform quality degradation. They are a specific set of turns where the router made the wrong call, and identifying which turns is the actual engineering work. This is where production trace mining meets routing: the failures aren’t uniformly distributed, and neither should the router’s tier assignments be.

The uncomfortable part

Intra-agent routing pushes the interesting engineering work down one level. It used to be enough to argue about which model to pick; now the argument is about the routing policy that picks between them per turn, the eval regime that scores each tier separately, and the cache topology that makes any of it economical. Cost-driven agent architecture stopped being an optimization concern and started being the primary structural decision — the 7% figure is not a curiosity, it’s a design constraint that reshapes what an agent runtime has to expose.

The teams still running one model for every turn in a production agent this quarter are paying, roughly, four times more than they need to for the same output. That is a large enough gap that it will not survive contact with a finance team once these benchmarks circulate.