AI governance is changing.
A few years ago, governance could often be handled through model documentation, privacy review, bias testing and approval before deployment.
That is no longer sufficient.
Modern enterprise AI can retrieve sensitive data, generate customer-facing content, select tools, invoke APIs, make recommendations, act autonomously, delegate work and change behavior when models or prompts change.
The governance problem is no longer:
Did we approve this model?
It is:
Can we continuously prove that the complete AI system is operating within approved business, legal, security and risk boundaries?
An AI governance framework is the operating system of policies, roles, controls, evidence, evaluations and escalation mechanisms that determines how an organization designs, approves, deploys, monitors and changes AI systems.
Governance should not exist only in policy documents. It must exist in the system.
What is AI governance?
AI governance is the set of organizational and technical mechanisms used to ensure AI systems are accountable, reliable, secure, compliant, transparent where required, appropriately supervised and aligned with business intent.
A mature governance system connects five questions:
1. What AI systems do we have?
2. What risks do they create?
3. What controls should apply?
4. Is the system operating within those controls?
5. What evidence proves it?
AI governance vs data governance
Data governance covers ownership, lineage, quality, access, retention and provenance.
AI governance additionally covers models, prompts, retrieval, tools, agents, outputs, decisions and actions.
The unit of governance should increasingly be the AI system, not just the model.
Seven-layer enterprise governance architecture
1. INVENTORY
↓
2. CLASSIFY RISK
↓
3. DEFINE CONTROLS
↓
4. EVALUATE
↓
5. APPROVE
↓
6. MONITOR
↓
7. IMPROVE / RESTRICT / RETIRE
Each layer should generate evidence.
Layer 1: inventory every AI system
Track system, business owner, technical owner, purpose, users, model/provider, data sources, tools, agents, risk tier, environment and status.
Inventory use cases, not only models. The same model can power low-risk marketing generation and high-risk financial recommendations.
Layer 2: classify risk
Use factors such as:
- business impact,
- data sensitivity,
- customer impact,
- regulatory exposure,
- autonomy,
- reversibility,
- financial impact,
- security impact.
Governance intensity should follow consequence.
Autonomy level is a first-class risk attribute because a recommendation, human-approved action and fully autonomous action can use the same model but have very different risk.
Layer 3: define control families
A practical taxonomy includes:
- data controls,
- model controls,
- retrieval/context controls,
- prompt controls,
- tool controls,
- agent controls,
- output controls,
- business controls,
- operational controls.
Data controls
Govern ownership, classification, lineage, quality, access, retention, provenance and permitted usage.
Model controls
Track provider, model, version, approved uses, evaluation, known limitations, region, data handling and deprecation.
Prompt controls
Treat important system prompts as versioned business logic. Track prompt ID, owner, approved model, evaluation result and production date.
Retrieval and context controls
Govern source eligibility, metadata, freshness, access, Top-K, reranking and citation requirements.
Tool controls
Define which agents can use which tools, allowed arguments, impact limits, approval requirements and logging.
Agent controls
Control maximum steps, cost, tool allowlist, delegation depth, memory policy, approval threshold, network access and timeout.
Output controls
Evaluate safety, privacy, factuality, formatting, disclosure, citation and policy before customer-facing release.
Business controls
Encode business rules outside the model, such as transaction limits or mandatory approval thresholds.
Layer 4: evaluate before deployment
Every release should answer:
- what was tested,
- which dataset was used,
- which threshold was required,
- whether the candidate passed,
- who approved the result.
Evaluation should reflect the task: LLM correctness and safety, RAG retrieval and groundedness, agent task completion and tool correctness.
Turn evaluation into an executable release gate.
Layer 5: approve according to risk
Low-risk systems may need technical-owner approval.
High-risk systems may require business, risk, security, legal or compliance review.
Avoid centralized committees approving every prompt edit. Use risk-based governance and pre-approved operating envelopes.
Layer 6: monitor production behavior
Monitor quality, safety, cost, latency, drift, policy violations, human overrides and business outcomes.
Traditional CPU and error monitoring is not enough. AI observability should track hallucinations, retrieval failures, tool use, policy violations, agent loops and cost per task.
Layer 7: continuously improve controls
Use:
DEPLOY
↓
OBSERVE
↓
EVALUATE
↓
INCIDENT / FEEDBACK
↓
ADJUST CONTROL
↓
RE-EVALUATE
Governance should be a feedback loop, not an annual form.
NIST AI RMF
The NIST AI Risk Management Framework organizes work around:
GOVERN
MAP
MEASURE
MANAGE
A practical mapping is:
- Govern → policy, roles and accountability,
- Map → use case, context, impact and stakeholders,
- Measure → testing, evaluation and monitoring,
- Manage → controls, mitigation and escalation.
Use it as a risk-management structure, not merely a checklist.
NIST Generative AI Profile
GenAI adds risks such as hallucination, prompt injection, synthetic content, third-party foundation models and opaque training data.
Use GenAI-specific guidance as an overlay on the broader governance operating model rather than creating a separate governance bureaucracy.
ISO/IEC 42001
ISO/IEC 42001 provides an AI management-system structure using Plan-Do-Check-Act.
A practical mapping is:
PLAN
Define policies, objectives and risk criteria.
DO
Build and deploy under controls.
CHECK
Evaluate, audit and monitor.
ACT
Correct, improve or restrict.
NIST and ISO are complementary: one organizes AI risk management; the other provides management-system discipline.
EU AI Act and evidence
Regulation increases the importance of governance evidence.
Build systems that can produce:
- AI system records,
- risk classification,
- ownership,
- model/version,
- data lineage,
- evaluation results,
- policy checks,
- human approvals,
- change history,
- incidents,
- monitoring.
Do not build only a compliance spreadsheet. Build controls that generate evidence continuously.
Governance by evidence, not questionnaire
Forms are useful, but they should not be the entire system.
Use:
System telemetry
+
Evaluation
+
Policy engine
+
Human approvals
↓
Continuous evidence
Agentic AI governance
Agents add questions around identity, tool access, delegation, memory, autonomy, spending limits and human approval.
An agent is not governed merely because its underlying model was approved.
Govern the architecture itself.
Govern agent identity
Every production agent should have an identity, owner, purpose, permissions and risk tier.
Avoid anonymous agents sharing broad credentials.
Govern agent autonomy
Use explicit autonomy levels ranging from information-only to recommendation, human-approved action, bounded autonomous action and broader autonomous operation.
Increasing autonomy should increase governance requirements.
Govern delegation
Track who delegated, task scope and expiration. Authority should generally shrink as it is delegated.
Govern memory
Control what can be remembered, provenance, retention, correction, deletion and sensitivity.
Third-party AI governance
Track providers, contracts, data use, retention, regions, security, model updates and incident commitments for foundation models, MCP servers, plugins, frameworks and external tools.
Model change management
A model or prompt change may require regression evaluation, security testing, policy review or customer notice even when application code is unchanged.
Governance and model routing
Governance should constrain the eligible model pool before routing optimization begins.
Governance and LLM-as-a-Judge
Automated evaluation can support governance, but the judge itself must be calibrated, versioned and monitored.
Governance and experimentation
Controlled experimentation can be safer than silently replacing production behavior.
Govern eligibility, exposure, metrics and rollback.
Risk-based release gates
Low risk may use automated tests.
Moderate risk may require owner approval.
High risk may require security/compliance review and limited rollout.
Human-in-the-loop is not one control
Define who reviews, at what point, with what evidence, what they are approving, whether they can reject and how approval is logged.
Policy as code
Convert important policies into executable logic where practical.
Examples:
- restricted data cannot use unapproved providers,
- refunds above a threshold require approval.
Policy-as-code moves governance from documentation into runtime enforcement.
Continuous controls
Some controls should run on every high-impact action rather than only during annual review.
AI governance metrics
Track:
- inventory coverage,
- ownership coverage,
- risk classification,
- release-gate pass rate,
- lineage coverage,
- unauthorized tool attempts,
- human escalation,
- failure rate,
- unresolved exceptions,
- rollback time,
- budget-policy violations,
- evidence completeness.
Track exceptions and their age to identify weak or unrealistic controls.
Governance maturity model
- Ad hoc.
- Documented.
- Standardized.
- Embedded.
- Adaptive.
The goal is not maximum bureaucracy. It is embedded governance.
Where KONTROLS fits
KONTROLS can operationalize policy through a loop of:
POLICY
↓
KONTROLS
↓
MONITOR
↓
ENFORCE
↓
AUDIT
Controls can span data, infrastructure, models, outputs and business processes.
Governance and KNOBS
Governance defines the allowed range for variables such as risk threshold, approval threshold, autonomy, cost, retrieval freshness, confidence and tool access.
KNOBS can optimize the operating point inside that governed range.
Practical operating model
Central governance owns policy and standards.
Business/domain owners own outcomes and risk acceptance.
Engineering owns implementation and evaluation.
Security/privacy own access and threat controls.
Legal/compliance own mandatory obligations.
Audit/risk provides independent assurance.
Do not centralize every decision. Define operating envelopes and escalate only when teams cross them.
90-day implementation roadmap
Days 1–30
Inventory production and customer-facing AI systems, owners, risk tiers, model registry and third-party providers.
Days 31–60
Define required controls by risk tier and map them to NIST, ISO and applicable regulation.
Days 61–90
Implement release gates, runtime policies, monitoring, evidence and exception workflows.
Start with a few high-impact systems.
Minimum viable AI governance
- AI system inventory.
- Accountable owner.
- Risk classification.
- Model/provider registry.
- Data-source documentation.
- Evaluation thresholds.
- Tool/action controls.
- Decision logging.
- Approval for high-impact actions.
- Incident and rollback procedures.
Common mistakes
- Governance by committee only.
- Governing models instead of systems.
- One process for all risk levels.
- Annual review only.
- No inventory.
- Policy without evidence.
- Ignoring third-party models.
- Treating agents like chatbots.
- Human approval without useful context.
- Treating compliance as the only objective.
Bottom line
A modern AI governance framework should be an operating system:
INVENTORY
↓
CLASSIFY
↓
CONTROL
↓
EVALUATE
↓
APPROVE
↓
MONITOR
↓
IMPROVE
Use risk frameworks to organize thinking, management-system standards to operationalize the organization, regulations to define mandatory requirements, and technical controls to make governance observable, enforceable and auditable.
Govern the complete AI system: not just the model.
Frequently asked questions
What is an AI governance framework?
The combination of roles, policies, controls, evaluation, monitoring and evidence used to govern AI systems throughout their lifecycle.
What are the four NIST AI RMF functions?
Govern, Map, Measure and Manage.
What is ISO/IEC 42001?
An international AI management-system standard for establishing, implementing, maintaining and continually improving organizational AI management.
KONTROLS
Turn governance policy into active controls across data, models, agents, outputs and business processes using monitoring, enforcement and audit evidence.
