The control pattern
Data asset
Instructions and context examples
Knobs
Example selection, Ordering, Diversity, Output schema, Tool instructions, Response constraints
Outcome
More consistent responses without retraining
Measurement
See how to measure this below
What it is
Prompt optimisation treats instructions, demonstrations and output formats as a configurable data layer. The prompt is not only prose; it is a compact policy that tells the model what evidence to use, which steps to perform, what to avoid, and how to represent the result.
Example selection is a powerful knob because models often imitate the structure and reasoning pattern demonstrated in context. Ordering can change which example dominates, diversity controls generalisation, and explicit schemas make responses easier to validate.
Prompt changes are faster and more reversible than fine-tuning, which makes them the right first move for new features and rapidly changing requirements. Once a prompt is stable and used at high volume, its approved input-output pairs can become distillation data for a smaller model.
The knobs in detail
Each row is one adjustable property of the data asset, and what moving it tends to do.
| Knob | What you adjust | Likely effect |
|---|---|---|
| Example selection | Which demonstrations appear in context | Models imitate the pattern they are shown |
| Ordering | Sequence of the demonstrations | Changes which example dominates |
| Diversity | How varied the examples are | Controls generalisation beyond the shown cases |
| Output schema | The required response structure | Makes responses machine-validatable |
| Tool instructions | When and how tools may be called | Shapes routing and escalation behaviour |
| Response constraints | Length, hedging, null handling | Prevents estimation where evidence is absent |
Applied: Stocks Assistant
A Stocks Assistant earnings prompt might require four blocks: verified metrics, changes from the comparable period, management outlook, and risks or uncertainties. It can instruct the model to distinguish reported facts from interpretation, attach evidence identifiers to numeric claims, and return null rather than estimate an absent value. Few-shot examples should cover more than famous technology companies : banks, insurers, retailers, industrial firms, real estate businesses and companies with unusual reporting conventions all belong in the set.
How to measure it
Evidence that the knob produced the intended behaviour, rather than shifting the problem elsewhere.
- Schema validity rate
- Factual accuracy and citation correctness
- Verbosity, latency and cost per response
- Consistency across repeated runs of the same input
Common mistakes
Optimising for polished language. Eloquence conceals weak evidence.
Treating prompts as ephemeral. Version them like software and test them against the same evaluation set.