The Model-Routing Rubric

I run a fleet of AI agents every day. The most expensive mistake I made early on had nothing to do with prompts or context windows. It was treating model choice as a technical preference when it’s actually a purchasing decision.

The frame that fixed it is simple. Every task that flows through my stack gets scored on three axes - cost, intelligence, and taste - and routed to the cheapest model that clears the bar. That’s the whole system. The discipline is in actually doing it.

The three axes

Cost isn’t the sticker price per token. It’s whatever is scarce for you - a subscription quota, an API budget, the hours in your own day. My frontier model is the most rationed thing I own, so a task it does that a cheaper model could have done isn’t a convenience. It’s a loss.

Intelligence is the axis everyone already optimizes, so I’ll only say this: most work doesn’t need the smartest model. A clear spec plus a mid-tier model beats a vague spec plus a frontier model, and it isn’t close.

Taste took me longest to price. Some work is mechanical - rename these files, write this migration, sweep these logs. Some work is judgment - what should this look like, what’s worth building at all, what would embarrass me if it shipped. Models differ more on taste than on intelligence, and taste is what you’re actually buying when you pay frontier prices. Once I scored it as its own axis, the routing table wrote itself.

Routing in practice

The frontier model gets only the work nothing else can do: orchestration, synthesis, design calls, anything a customer will see. Standard coding with a clear spec goes a tier down. High-volume mechanical work goes to the cheapest thing that won’t make a mess.

Intentions drift, so I run tripwires instead. If I catch a model grinding through more than a handful of similar files, or a hundred-plus lines of work someone already specced, that batch gets stopped and delegated down, then the diff gets reviewed. And I track one number in the other direction: what fraction of delegated work survives review. When it drops below roughly four in five, the routing is miscalibrated - the spec was too thin or the tier was too low - and the rubric gets tightened, not the delegation abandoned.

Where the scores come from

The seed of this system was a question I asked myself while building eval infrastructure: if I’m already running the same tasks across models to catch regressions, why am I still routing by folklore? Evals aren’t just a quality gate. Run them across models and you accumulate a scoreboard of who wins which class of task. That scoreboard is the routing table. You stop deciding “which model do I like” and start reading off “which model earns this task.”

Why I call it a P&L decision

Every company already knows this logic under a different name: make or buy. You don’t put your most expensive hire on data entry - not because they’d do it badly, but because their hours are the constraint the whole business is planned around.

Model routing is the same decision at a smaller grain, made hundreds of times a day. Judgment is the scarce input. Spend it where it compounds, buy everything else at the market rate, and audit the results like you’d audit any vendor.

The payoff isn’t the saved money, though it’s real. It’s that the expensive model finally has room to do the only work that was ever worth paying it for.

⁂