The control pattern
Data asset
Generated training and test examples
Knobs
Generator model, Constraints, Coverage plan, Rarity, Perturbation level, Human review rate
Outcome
Coverage of important cases that real data does not supply reliably
Measurement
See how to measure this below
What it is
Synthetic data uses generated examples to fill gaps in training and evaluation. It is most useful when real examples are scarce, expensive to label, privacy-sensitive or badly imbalanced.
The generator model and its instructions are knobs, but so are the distribution of cases, the realism of document structure, the frequency of rare events, and the amount of deliberate corruption. Synthetic data should be designed from a coverage plan. Produced as an undifferentiated batch, it amplifies the generator's habits and creates the illusion of diversity without adding genuinely new situations.
Every synthetic record should carry provenance, generation settings, intended difficulty and validation status.
The knobs in detail
Each row is one adjustable property of the data asset, and what moving it tends to do.
| Knob | What you adjust | Likely effect |
|---|---|---|
| Generator model | Which model produces the examples | Caps realism and imprints its own habits |
| Constraints | Rules the generated case must satisfy | Keeps examples internally consistent |
| Coverage plan | Which scenarios are represented, and how often | Prevents illusory diversity |
| Rarity | Frequency of unusual events | Teaches the tail without swamping the centre |
| Perturbation level | Deliberate corruption and noise | Builds robustness to real-world mess |
| Human review rate | Share of records checked by a person | Focuses scarce review on high-risk labels |
Applied: Stocks Assistant
For Stocks Assistant, synthetic cases could model unusual earnings scenarios: a company withdraws guidance, changes reporting segments, announces a restatement, reports a large one-time impairment, changes currency presentation, or gives contradictory signals between prepared remarks and Q&A. Synthetic documents can also reproduce OCR problems, missing table borders, merged columns, low-resolution scans and ticker ambiguity. These are valuable for teaching extraction, uncertainty detection and escalation.
How to measure it
Evidence that the knob produced the intended behaviour, rather than shifting the problem elsewhere.
- Improvement measured on a separate set of real, point-in-time examples
- Automated checks on arithmetic, schema validity and absence of future information
- Reviewer agreement on high-risk labels
- Coverage achieved against the original plan
Common mistakes
Gains that appear only on synthetic tests. That is a sign the data is teaching artifacts rather than transferable skill.
Generating market outcomes and presenting them as real observations. Synthetic evidence has no place in return prediction.