Where the Model Is Genuinely Load-Bearing

Once you’ve accepted that the model belongs where being wrong is cheap and that its job is to produce artifacts code executes, a useful question follows: where is the model actually load-bearing? Where does it do something no reasonable amount of code could? The honest answer, in most agent systems, is a short list — and being strict about that list is what separates a system you can trust and afford from one you can’t.

The short list

There are really only a few jobs where a language model is doing something genuinely irreplaceable:

• Turning a heterogeneous document into a structured mapping. Making sense of a format you’ve never seen well enough to line it up with your schema.
• Understanding free-text well enough to tell intents apart. Distinguishing a deferral from a question from a refusal in a human reply that follows no template.
• Drafting courteous, specific language — often across several languages — that’s correct in tone and detail, not boilerplate.
• Forming a hypothesis from sparse, contradictory signals — when something has gone quiet and you need a best-guess cause from evidence that doesn’t add up.

That’s roughly it. Notice what these have in common: each requires understanding unstructured, ambiguous, human input, and in each case a wrong answer is cheap — it becomes a review item, a redraft, a second look. None of them is a consequential, auditable decision.

Everything else is code, and it’s not close

The corollary is where the discipline actually bites. Ranking is code. A derived metric is arithmetic. Eligibility is rules. Identity or record scoring is a weighted feature function you can read, test, and explain. None of these should touch a model, and the temptation to let them is exactly the reversed-principle failure from the first post — reaching for the model on the very decisions that most need to be deterministic and auditable.

It’s worth being blunt about why. A model that ranks your results gives you a different order on a bad day and can’t tell you why this came above that. Arithmetic done by a model is arithmetic you can’t trust and can’t reproduce. A rule “decided” by a model is a rule with no version history and no owner. Every one of these is a solved problem in software, and handing it to a model trades a correct, cheap, explainable answer for a probabilistic, expensive, opaque one.

The one-week test

When a new candidate use for the model appears — and it always does, because the model is the fun part — apply a single test: could a competent engineer write this deterministically, given a week? If yes, it’s code. Not “could the model probably do it” — of course it could; the model can do almost anything badly. The question is whether the problem has enough structure that a person could capture it in rules and tests. If it does, that’s where it belongs, because deterministic code is cheaper to run, reproducible, auditable, and doesn’t quietly drift when a model version changes underneath it.

The model earns its seat in the few places where the answer to that test is a clear no — where the input is genuinely unstructured and the required judgment genuinely resists specification. Guard that list. Every job you let creep onto the model’s side of the line is a job that just got more expensive, less reproducible, and harder to explain — usually in exchange for nothing.

Cheap to Be Wrong — a six-part series on agent architecture

1. Put the Model Where Being Wrong Is Cheap
2. Use the Model to Write the Parser, Not to Be the Parser
3. Where the Model Is Genuinely Load-Bearing — you are here
4. An Agent Is a State Machine, Not a Loop
5. The Model Upgrade That Quietly Breaks You
6. The Model Is the Least Defensible Part of Your Agent