Core definition
A knob is a first-class, controllable input : a variable, data asset, configuration or policy : that measurably changes system behaviour or a business outcome. Unlike a metric, it can be deliberately set, tested, versioned and retuned.
The qualification bar
When a control is actually a knob
A signal you only observe is a metric. Raw configuration becomes a knob only once it is linked to behaviour, ownership, measurement and safe change.
Definition
Named intent, valid range, default, owner and change authority.
Measurement
Effective value is linked through telemetry to outputs and outcomes.
Safety
Staged rollout, known failure modes, limits, rollback and escalation.
Governance
Access control, version history, audit log and artifact lineage.
Three views
One concept, three complementary lenses
These are overlapping lenses, not competing lists. Every knob has an answer under all three.
| View | Question | Types |
|---|---|---|
| Control layer | What is being controlled? | World, Data, Behavior, Policy |
| Artifact family | Where is the control attached? | Data, Code/Runtime, AI/Model, Infrastructure, Interface, Process |
| Lifecycle role | What job does it perform? | Selection, Creation, Shaping, Control, Evaluation, Adaptation |
Artifact families
Where the knob lives
The most concrete inventory for product design: inspect the assets and code paths, then expose the high-leverage intervention points. Six families cover a production AI system : what data goes in, what code runs, what model reasons, what serves it, what the user sees, and who intervenes.
Data
What the system reasons over
Code / Runtime
What executes
AI / Model
What reasons
Infrastructure
What serves it
Interface
What the user sees and sets
Process / Human
Who intervenes
| Family | Knob type | Typical settings |
|---|---|---|
| Data | Scope & cohort | filters, segments, eligibility, time windows |
| Representation & aggregation | granularity, rolling windows, deduplication | |
| Integrity & quality gates | null, validity, freshness and drift thresholds | |
| Semantics & labels | definition of truth, success, attribution window | |
| Generation & augmentation | synthetic scenario mix, rarity, perturbation, provenance | |
| Code / Runtime | Feature flags & rollouts | binary/multivariate toggles, targeting, canaries |
| Algorithmic thresholds | weights, scoring coefficients, cutoffs | |
| Operational resilience | timeouts, retries, circuit breakers, fail mode | |
| AI / Model | Model selection & routing | versions, checkpoints, providers, fallbacks |
| Prompt & context | instructions, retrieval filters, chunks, context size | |
| Decoding & sampling | temperature, top-p/top-k, max tokens, stops | |
| Safety & policy guardrails | refusal, PII filters, tool permissions, review gates | |
| Infrastructure | Capacity & concurrency | replicas, autoscaling bounds, rate limits, quotas |
| Caching & precomputation | cache scope and TTL, batch vs. real-time, refresh cadence | |
| Placement & residency | region, tenancy, hardware tier, data residency | |
| Cost ceilings | per-request spend cap, model tier by plan, degradation order | |
| Interface | Disclosure & evidence | citations shown, confidence display, uncertainty language |
| Interaction defaults | default view, verbosity, alert cadence and channel | |
| User-exposed controls | risk tolerance, time horizon, filters, notification preferences | |
| Process / Human | Review triggers & sampling | queue criteria, sampling rate, dual-review threshold |
| Authority & change windows | approval roles, freeze periods, rollout gates | |
| Escalation & adjudication | routing to experts, response targets, tie-break rules |
On resisting sprawl: agent orchestration sits under AI/Model, and the semantic layer sits under Data as semantics and labels. Promote either to its own family only when a product has enough surface area there to warrant its own owner.
Control layers
What the knob shapes
World knobs
Encode the operating reality or business posture: risk tolerance, freshness requirements, confidence tolerance, recency bias and compliance stance.
Data knobs
Use data assets themselves as control inputs: golden sets, failure cases, corpus slices, edge cases, evaluation scenarios and synthetic examples.
Behavior knobs
Shape runtime output distributions through retrieval depth, ranking weights, prompts, schemas, generation settings and post-processing filters.
Policy knobs
Externalise security, privacy, auditability, business constraints, approval rules and autonomy boundaries as inspectable, versioned controls.
Lifecycle roles
What the knob does
The roles form a loop rather than a list. The first three build the material a system reasons over, the fourth governs live behaviour, and the last two decide whether the change worked and when to go round again. The final column links to the data-family topics that use each role.
| Role | Primary job | Representative examples | Topics |
|---|---|---|---|
| Selection | Choose the few high-information examples or regions that matter most. | sampling criteria, coverage slices, golden-set membership | |
| Creation | Generate bounded data to fill sparse, rare or risky gaps. | synthetic scenario constraints, rarity, difficulty, provenance | |
| Shaping | Change the form of what already exists, rather than which pieces or whether new ones are made. | granularity, aggregation windows, deduplication, normalisation, example weighting, label definitions | |
| Control | Adjust live behaviour without retraining, with versioning and rollback. | confidence thresholds, routing, refusals, escalation rules | |
| Evaluation | Set the bar a change must clear, and the slices it must clear it on. | metric definitions, evaluation slices, holdout boundaries, pass thresholds, guardrail bounds | |
| Adaptation | Decide when the artifact itself should change, not just its runtime settings. | drift thresholds, retraining triggers, refresh cadence, promotion and rollback criteria |
Why six rather than three: mapped against the artifact inventory, Selection, Creation and Control alone did not discriminate : most entries collapsed into Control, representation and semantics had no home at all, and the measurement the design rule demands had no controls to point at. Shaping, Evaluation and Adaptation close those gaps and let the loop return to its own starting point.
How the views overlap
The labels change; the control object is the same
A trusted-source retrieval filter is simultaneously an AI/Model artifact, a Behavior knob and a runtime Control knob. The lens you use depends on the design question in front of you.
| Control object | Control layer | Artifact family | Lifecycle role |
|---|---|---|---|
| Trusted-source retrieval filter | Behavior | AI / Model | Control |
| Curated earnings-call evaluation set | Data | Data | Selection |
| Per-request spend cap | World | Infrastructure | Control |
| Dual-review threshold for high-materiality claims | Policy | Process / Human | Evaluation |
| Drift alert that triggers a refresh | World | Data | Adaptation |
The library
This page is the map
Everything else is a territory within it. Each guide should say which part of the map it covers.
Field guide
Data as Levers
One territory in depth: twenty topics working the Data artifact family across all four control layers.
Open the guide →
Not yet written
Five families without equivalent depth
Code/Runtime, AI/Model, Infrastructure, Interface and Process. Infrastructure is where cost and latency are actually set; Interface is where an internal control becomes a feature a customer can see and adjust.
Reconciling the two uses of “data”: “Data knobs” is one of four control layers on this page, and a data asset sits behind every topic in the field guide. Both are correct : the guide takes the Data artifact family and works it across all four layers.