Deterministic Code Is the Secret to Smaller Models
Smaller models should not be asked to do work that code can do better.
That sounds obvious, but it is one of the most common mistakes in AI system design. Teams ask a model to route, validate, format, calculate, retrieve, score, approve, remember, and explain, all inside the same prompt. Then they blame the model when the result is inconsistent.
A better architecture separates judgement from machinery.
The model handles the language problem. Deterministic code handles the operational contract.
What code should own
In a governed agentic workflow, many steps should be predictable by design:
- input validation
- permission checks
- data retrieval
- formatting
- routing
- thresholds
- audit logging
- version control
- escalation triggers
- repeatable calculations
None of these should rely on a model’s mood, context window, or interpretation of a vague instruction. They should be built into the system.
That makes the model’s job smaller and clearer.
Why this helps smaller models
A smaller model has less general capacity than a frontier model. That is exactly why it needs a well-built operating environment.
If the surrounding code supplies the right context, constrains the available actions, validates the output, and routes exceptions to a human or larger model, the smaller model can concentrate on a narrower reasoning task.
This is how smaller systems become useful in production. They are not winning by being broadly smarter. They are winning because the architecture stops asking them to solve everything at once.
The adapter plus code pattern
Jonny’s model upskilling idea is not just “train a small model.” It is closer to:
- Use a capable model to teach the reasoning pattern.
- Capture the pattern in specifications, examples, and tests.
- Train or tune adapters for the repeatable judgement task.
- Wrap the smaller model in deterministic code.
- Route uncertainty, exceptions, and high-risk decisions appropriately.
The smaller model then becomes part of a system rather than the whole system.
That is the key distinction. Production AI is not a model. Production AI is an operating design.
The governance point
Deterministic code also makes governance easier. It creates clear places to enforce policy, log decisions, test conformance, and prove that the system did what it was supposed to do.
For regulated, client-facing, or high-volume work, that matters as much as output quality.
Smaller models do not punch above their weight alone. They do it when the system around them carries the weight they should never have been asked to carry.
Further context
This post supports Jonny Bowker’s work on upskilling AI and the wider spec-driven AI methodology.