AI control plane · DataKnobs

Providers give you model knobs. DataKnobs gives you system knobs.

LLM providers expose parameters such as model choice, temperature, top-p, reasoning effort and token limits. DataKnobs operates one level higher: it makes the operating parameters of the entire AI system explicit, measurable, comparable and governable.

Model providers control intelligence. DataKnobs controls how that intelligence is allowed to operate.

See the difference View architecture

Provider layer OpenAI · Anthropic · Gemini · Local models Model, temperature, top-p, reasoning, token limits

↓

DataKnobs control plane System-level knobs Prompt · Context · Retrieval · Memory · Tools · Planning · Routing · Runtime · Governance · Evaluation

↓

Execution layer Agentic harness Assembles the selected operating point and runs the workflow

↓

Measurement loop Observability · Evaluation · Experimentation · Optimization Discover which settings actually improve quality, cost, latency and risk

Provider vs DataKnobs Knob families Value added Governance Optimization Architecture Three pillars

The distinction

DataKnobs is not another model-parameter panel

Temperature and top-p are useful, but they represent only a small part of the behavior of a production AI application. The larger opportunity is to control the system around the model.

LLM provider

Defines model-level controls

Model and model version
Temperature / top-p / sampling
Reasoning or thinking effort
Maximum output tokens
Provider-specific API features

DataKnobs

Defines the operating configuration of the whole AI system

Normalizes provider settings behind a stable control layer
Controls prompts, context, retrieval, memory, tools and planning
Controls routing, autonomy, budgets, approvals and escalation
Versions every operating point so runs are reproducible
Evaluates and experiments to identify which knobs matter
Governs which configurations are allowed for each risk tier

Positioning: DataKnobs does not create the underlying AI controls. It makes controls across models, prompts, data, tools, agents, policies and evaluation explicit, measurable, comparable and governable.

System knobs

Most important agent controls are not model parameters

A production agent can fail even when the model is excellent. DataKnobs elevates application and governance choices into first-class control dimensions.

ModelProvider, model family, reasoning capability, fallback.Provider + DataKnobs adapter

PromptSystem instructions, task prompt, prompt version, policy text.System knob

ContextContext sources, ordering, compression, context budget.System knob

RetrievalStrategy, top-K, similarity threshold, filters, reranker.System knob

MemoryEnabled state, horizon, summarization, persistence, expiration.System knob

ToolsAllowed tools, schemas, descriptions, credentials, permissions.System knob

PlanningPlanner type, decomposition strategy, reflection and retries.Harness knob

RoutingSpecialist selection, model routing, escalation thresholds.Harness knob

RuntimeMax steps, timeouts, token and dollar budgets, checkpoints.Harness knob

AutonomyRead, recommend, prepare, bounded execute, high-impact execute.Governance knob

PolicyBlocked actions, transaction limits, allowlists, approval rules.Governance knob

EvaluationJudge model, rubric, task metrics, safety and policy thresholds.Measurement knob

What DataKnobs adds

From scattered settings to an engineered operating system

The value is not the existence of knobs. The value is what DataKnobs lets the enterprise do with them.

Make hidden choices explicit

Move thresholds, retrieval settings, tool permissions, memory modes and escalation rules out of scattered application code and into a registered, versioned control plane.

Normalize across providers

Present stable business-level concepts while adapters translate supported controls to OpenAI, Anthropic, Gemini or local-model APIs.

Find which knobs matter

Measure whether the biggest improvement came from a new model, a better prompt, stronger retrieval, a clearer tool interface or a different planner.

Measure interactions

Detect combinations such as a planner that works well at top-K 8 but degrades at top-K 20. Orthogonal is a design goal, not a claim that interactions never exist.

Make runs reproducible

Record the full operating point: model, prompt, tools, retrieval, policy and evaluator: so an enterprise can explain why yesterday's behavior differs from today's.

Govern configurations

Treat autonomy, approvals, permissions, spending limits and tool access as configurable policy rather than hard-coded application behavior.

Optimize the operating point

Search for the lowest-cost configuration that satisfies quality, latency, safety and policy constraints instead of blindly selecting the largest model.

Key insight

A model upgrade is only one experiment. A tool-description change, retrieval strategy or governance policy may have a larger impact on the final outcome.

Enterprise outcome

DataKnobs turns AI development from informal prompt tweaking into controlled system engineering: configure, execute, observe, evaluate, compare, govern and optimize.

Governance as configuration

The same model can receive very different authority

This is where DataKnobs has a stronger enterprise role than the model provider: the enterprise: not the LLM API: defines what the agent is permitted to do.

Marketing agent

Produces drafts. Human approval can be optional because the action is low-impact and reversible.

autonomy: draft
human_approval: optional

Customer-service agent

May execute low-value credits within an approved policy envelope and escalate above the threshold.

autonomy: bounded_execute
refund_limit: 100
approval_above: 100

Treasury agent

May analyze exposure and recommend an action while money movement remains prohibited until an authorized human approves it.

autonomy: recommend
money_movement: prohibited
human_approval: required

DataKnobs principle: control authority independently from intelligence. A more capable model should not automatically receive more permissions.

Evaluation + experimentation

Having 50 controls is not useful unless you know which ones change the outcome

DataKnobs connects knobs to evaluation and experimentation so teams can move from configuration management to evidence-based optimization.

Example: discover the real performance driver

Change	Illustrative success rate	Signal
Baseline	82%	Current operating point
Model upgrade	84%	Small gain
Prompt revision	86%	Moderate gain
Retrieval change	90%	Large gain
Tool-description change	94%	Largest gain

Illustrative values used to explain the method: not benchmark claims.

Optimization objective

Once knobs and metrics are formalized, DataKnobs can search across operating points subject to enterprise constraints.

goal:
  quality: ">= 92%"
  p95_latency: "<= 4s"
  cost_per_task: "<= $0.04"
  policy_violations: 0

search_over:
  - model
  - retrieval_top_k
  - planner
  - toolset
  - routing

Configure

→

Execute

→

Observe

→

Evaluate

→

Compare

→

Govern

→

Optimize

Reference architecture

A stable control layer above fast-changing providers

Models may change quickly. Enterprise concepts such as quality, cost, latency, data access, authority and approval change much more slowly. DataKnobs becomes the stable abstraction between the two.

Model providersOpenAI · Anthropic · Gemini · local / open models

→

DataKnobs knob registry + control planeProvider adapters · prompts · context · retrieval · tools · memory · routing · runtime · autonomy · policies · evaluators

→

Agent harnessApplies the selected operating point to the agent workflow

ObservabilityTrace the complete run configuration, tools, outcomes and resource use.

Evaluation + experimentationCompare variants, estimate individual and interaction effects, run champion/challenger.

Governance + optimizationAllow only approved operating points and select configurations that satisfy enterprise constraints.

Concrete configuration

From provider-specific API calls to a system-level operating point

Fragmented application logicBefore

if confidence < .80:
    escalate()

if amount > 100:
    ask_human()

top_k = 8
max_steps = 12
use_memory = True

Registered DataKnobs operating pointAfter

reasoning:
  depth: high
sampling:
  creativity: low
retrieval:
  top_k: 8
  rerank: true
runtime:
  max_steps: 12
governance:
  autonomy: recommend
  approval_required_for:
    - money_movement
    - external_write

Important: the provider adapter can translate supported high-level settings into provider-specific API parameters, while DataKnobs continues to own system-level behavior and governance that the provider cannot know.

The DataKnobs model

KREATE builds it. KONTROLS bounds it. KNOBS improves it.

The control-plane idea becomes a natural bridge across the three DataKnobs pillars.

KREATE

Build the harness, agents, prompts, retrieval, memory, tools, workflows, adapters and evaluation harness.

KONTROLS

Define which configurations, data, tools, actions, spending levels and autonomy levels are permitted for each risk tier.

KNOBS

Expose important system choices as measurable, versioned, adjustable dimensions and discover which ones actually move the outcome.

Providers expose parameters. DataKnobs engineers the operating system around them.

That is the differentiation: a unified control plane for models, prompts, context, tools, agents, policies, evaluation and experimentation.