The control pattern
Data asset
Representation in training, retrieval and evaluation data
Knobs
Group balance, Market coverage, Labels, Peer definitions, Counterfactual examples, Weighting
Outcome
More equitable and less systematically distorted behaviour
Measurement
See how to measure this below
What it is
Bias in a financial AI system often begins with uneven data representation. Large, heavily covered companies produce more news, analyst research, transcripts and clean structured data than small or international firms. If that abundance is mistaken for quality, the assistant may consistently favour popular securities or sectors.
Representation is therefore the data asset to change. The knobs include company size, sector, geography, reporting style, language, exchange, analyst coverage, and whether failed or delisted companies remain in historical data.
Fairness does not require identical scores across unlike companies. It requires comparable evidence, defensible peer definitions and consistent rules.
The knobs in detail
Each row is one adjustable property of the data asset, and what moving it tends to do.
| Knob | What you adjust | Likely effect |
|---|---|---|
| Group balance | Representation across segments | Stops abundance masquerading as quality |
| Market coverage | Size, geography, exchange, language | Controls where the system is reliable |
| Labels | How ground truth is defined | Encodes assumptions into every score |
| Peer definitions | Who a company is compared against | A misdefined peer group is a fairness failure |
| Counterfactual examples | Same fundamentals, different name or sector | Reveals unwanted preference |
| Weighting | Emphasis across segments in training | Corrects systematic under-representation |
Applied: Stocks Assistant
Stocks Assistant should evaluate extraction and scoring across market-cap bands, industries, business models and reporting conventions. Peer groups are themselves a fairness knob: comparing a bank's leverage with a software company's leverage produces a misleading score, as does applying one profitability threshold across capital-intensive and asset-light industries. Counterfactual tests can hold fundamentals constant while changing names, sector labels or popularity indicators. Historical backtests must control survivorship bias by including firms that merged, failed or left an index.
How to measure it
Evidence that the knob produced the intended behaviour, rather than shifting the problem elsewhere.
- Data coverage and missingness by segment
- Error rates and calibration by segment
- Ranking exposure across market-cap bands and sectors
- Counterfactual sensitivity to name and sector labels
Common mistakes
Turning data scarcity into a negative investment judgment. Show lower confidence instead.
Backtests that quietly exclude companies that failed or were delisted.