Multi-Agent Orchestration: Patterns, State, and Evaluation

A multi-agent system coordinates two or more AI agents to complete a shared business task. Multi-agent orchestration is the control layer that assigns work, determines when an agent should act, exchanges task state, validates results, and decides when the workflow has finished or requires human intervention.

The architecture is valuable when specialized knowledge, independent parallel work, or independent review measurably improves an outcome. It is not automatically better than one well-designed agent. More agents can add model calls, transfers, state synchronization, latency, and new authorization boundaries. The right starting point is a single-agent baseline and a business-specific evaluation set: not a preferred diagram.

Quick answer: Choose a sequential pipeline for predictable stages, fan-out/fan-in for independent parallel work, supervisor–worker for dynamic delegation with central accountability, handoff when another specialist must take ownership, and hierarchical coordination only when the workflow genuinely spans multiple teams or domains. Whatever pattern you use, implement durable state, typed task contracts, authorization at each execution boundary, evaluation, and a budget per completed task.

What multi-agent orchestration actually controls

An orchestrator is not simply another chat model. In a production architecture, it controls the transition from a user request to authorized work:

User goal
   |
   v
Authenticated request + policy context
   |
   v
Orchestrator / deterministic workflow
   |--- route --> Research agent --------|
   |--- route --> Data-analysis agent ---|--> Validate outputs
   |--- route --> Policy agent ----------|         |
   |                                              v
   |                                    Approval / action gateway
   |                                              |
   v                                              v
Durable task state ------------------------> Verified outcome

The orchestrator decides what work is required, which agent has the right capability, what context that agent may receive, whether tasks can run concurrently, how errors are handled, and what constitutes completion. Business rules such as authorization and approval should be enforced by trusted services, not solely by the orchestrator's prompt.

This distinction matters because a multi-agent workflow combines three separate concerns: reasoning (what to do next), coordination (who does it and when), and execution control (whether the requested action is authorized and safe). The AI agent architecture guide explores the broader system layers; this page concentrates on coordination between them.

When is a multi-agent system better than one agent?

Consider four tests before adding a specialist:

1. Specialization: Does the specialist succeed on an important task slice that the baseline consistently misses?
2. Parallelism: Can genuinely independent tasks run simultaneously and reduce end-to-end latency?
3. Context isolation: Can distinct agents work with narrower, better-scoped instructions and evidence instead of one increasingly crowded context window?
4. Independent assessment: Does a separate reviewer detect material mistakes that are missed by deterministic validators and ordinary single-agent self-checks?

If none of these is demonstrated, keep the single-agent design and improve its tools or context. LangChain's current multi-agent guidance similarly cautions that a single agent with the right tools can often achieve comparable results and identifies context management and parallel work as principal reasons to specialize (LangChain documentation).

A workflow with three agents is not inherently more autonomous. It may merely be three expensive calls placed inside a deterministic pipeline. That can be a good design, but the team should describe it accurately.

Seven orchestration patterns and when to use them

Patterns differ mainly in who owns the task, who routes the next step, and how results are merged.

These are architectural categories, not mutually exclusive products. Microsoft's Agent Framework documents sequential, concurrent, handoff, group-chat, and manager-led orchestration in its workflow orchestrations guide, last updated August 25, 2026. LangChain describes subagents, handoffs, routers, and custom workflows. The names vary across frameworks; the coordination trade-offs remain the same.

Sequential pipeline: predictable steps

A supplier-onboarding workflow might extract company information, perform sanctions-screening checks, assemble an evidence packet, and generate an analyst summary. If each step has a fixed contract, a deterministic workflow is often easier to operate than asking a model to repeatedly decide what happens next. Use models inside steps only where semantic judgment is valuable.

Fan-out/fan-in: independent work in parallel

A research task can dispatch independent workers to regulatory filings, market data, and relevant legal cases. Their results return as structured artifacts, and a merger checks consistency before producing an answer. Parallelism helps only if the tasks have no hidden sequencing requirement and the slowest worker does not dominate the deadline.

Supervisor–worker: dynamic delegation

A coordinator may send a question to a research worker, notice that the evidence is incomplete, request a second analysis, and then synthesize an answer. It retains overall ownership. Use explicit maximum steps, budget caps, escalation policy, and typed worker outputs; otherwise, a supervisor can repeatedly call workers without improving the answer.

Handoff: transfer ownership to a specialist

In a support workflow, a general agent can hand a billing request to a payments agent that now owns the interaction. This differs from calling the payments agent as a tool, where the original agent keeps control. Microsoft explains the difference between these patterns in its handoff orchestration guidance.

Hierarchical, peer, and debate arrangements

Use hierarchy when a single supervisor cannot reasonably manage independent domains and their sub-workflows. Use peer coordination only if decentralized operation has a real resilience or organizational benefit; it requires stronger termination and reconciliation mechanisms. Use reviewer or debate agents as tests of evidence, not substitutes for authoritative data, deterministic checks, or human approval.

Design the task contract before choosing a framework

The most reusable multi-agent interface is a typed task contract. It specifies the goal, available evidence, expected output, policy context, deadline, and traceable identity. Here is an illustrative application-level JSON contract, not an A2A protocol schema:

{
  "task_id": "case-4821:legal-review:v1",
  "workflow_id": "case-4821",
  "assigned_capability": "company-litigation-review",
  "goal": "Identify material litigation involving the supplier",
  "input_refs": ["gs://approved-evidence/case-4821/filings.json"],
  "output_schema": "litigation_assessment_v2",
  "allowed_actions": ["read_case_data", "draft_assessment"],
  "deadline_utc": "2026-10-16T21:00:00Z",
  "max_cost_usd": 0.50,
  "idempotency_key": "case-4821-legal-v1",
  "requires_approval_for": ["publish_external"]
}

The trusted gateway must derive tenant identity and authorization from the authenticated request. Do not let one model grant itself permissions by editing allowed_actions or inventing a different tenant ID. For a more detailed execution boundary, see secure agent tool and function integration.

Each response should carry machine-readable fields such as status, artifact_ref, schema_version, evidence_refs, error_code, and completion_timestamp. Treat generated natural-language explanations as human-facing annotations rather than authoritative workflow state.

Separate shared workflow state from each agent's memory

A frequent design mistake is forwarding the entire conversation to every new specialist. This increases irrelevant context, complicates permissions, and can spread untrusted or sensitive material between agents.

Instead, maintain a durable workflow state store with scoped references to immutable or versioned artifacts:

• Workflow record: request owner, policy decision, step graph, deadline, and status.
• Task record: assignee, attempt count, idempotency key, deadline, and last checkpoint.
• Evidence store: source documents, timestamps, classification, and lineage.
• Agent-local context: only the facts and tools required for one agent's current step.
• Action ledger: approved, attempted, completed, failed, and compensated side effects.

This is not the same as long-term agent memory. AI agent memory design covers retained facts across sessions. For each execution step, agent context engineering determines which current facts should be made available to the model.

Use optimistic version checks or transactions for concurrent updates. Where two agents may update the same assessment, record separate artifacts and merge them through a controlled step. Never rely on the order in which responses happen to arrive.

Build recovery around durable execution

A production workflow must survive timeouts, process crashes, duplicate deliveries, and partially completed actions. Use the following controls:

• Checkpoints: record task state after meaningful transitions so an interrupted run can resume.
• Idempotency: attach a stable key to side-effecting actions and deduplicate retries in the receiving service.
• Bounded retries: retry transient failures with backoff; do not retry policy denials or malformed requests indefinitely.
• Timeouts and cancellation: propagate deadlines through child tasks and release resources when a workflow is abandoned.
• Compensation: if a sequence has irreversible effects, define human-approved remediation rather than assuming a database rollback will undo external actions.
• Dead-letter handling: route exhausted or ambiguous tasks to a queue for investigation.

For failure scenarios and operational SLOs, see AI agent reliability engineering. A multi-agent system is reliable only if its end-to-end task can recover, not merely if each worker can be restarted.

A2A versus MCP: different integration boundaries

Multi-agent orchestration often uses more than one communication mechanism:

• Function calling and MCP: connect an agent to application tools, databases, retrieval, and services.
• Agent2Agent (A2A): connect independent agents or agent-hosting services across a task-oriented boundary.
• Local workflow edges: coordinate components within the same runtime without requiring a cross-service protocol.

A2A exposes concepts including Agent Cards, Tasks, Messages, Parts, and Artifacts, allowing independent agents to discover capabilities and exchange stateful work. The official A2A core concepts document describes the distinction. A2A does not replace application authorization, policy enforcement, or a workflow's responsibility for the final outcome.

Choose a cross-service protocol when there is a real interoperability requirement. Do not introduce protocol complexity for two functions running in the same process. For related protocol trade-offs, see the MCP vs A2A comparison.

Security boundaries must follow the task, not the prompt

Delegation creates trust boundaries. A retrieved document, specialist agent, or remote service may return useful information while also containing prompt-injection text or data that the receiving agent is not entitled to see.

Enforce capability allowlists, tenant-scoped retrieval, least-privilege credentials, sanitized outputs, and authorization at the execution service. Restrict whether workers can invoke other workers and cap delegation depth. Require explicit approval for sensitive actions such as payments, outbound communications, changes to customer records, and production deployment. Microsoft's current workflow documentation supports pausing certain tools for human approval through a request/response mechanism (Human-in-the-Loop workflows).

The agent security guide covers injection and tool abuse in depth; human-in-the-loop agent design explains binding an approval to a specific action and its parameters. Enterprise risk ownership should align with the broader Govern, Map, Measure, Manage functions in the NIST AI Risk Management Framework.

Measure whether orchestration earns its cost

Do not compare systems only on answer quality. Evaluate the same workload across a single-agent baseline, a deterministic pipeline, and the candidate multi-agent design. Hold task definitions and success criteria fixed.

Cost per successful task is especially important: total model + tool + infrastructure cost ÷ verified successful tasks. A lower cost per invocation does not help if failures and retries increase. Conversely, a higher-cost multi-agent solution can be justified if it substantially improves verified outcomes in high-value workflows.

Published percentages about a supposedly universal “multi-agent overhead” or “most common production topology” should not be treated as constants. Cost varies with task difficulty, context size, model mix, coordination strategy, concurrency, caching, and required quality. Benchmark it on your workload rather than copying an unsourced percentage into architecture decisions.

Evaluate both the final state and the trace: which agent was selected, what evidence it received, how often tasks bounced between workers, which constraints were enforced, and where recovery occurred. AI agent testing and evaluation explains outcome, trajectory, and component-level grading; agent observability connects those findings to production traces.

Example: an enterprise supplier-risk investigation

Consider a procurement analyst asking: “Is this new supplier suitable for a three-year contract?” A responsible system should not invent a single confidence score and approve the supplier on its own. Instead:

1. A coordinator receives an authenticated request and determines which checks are permitted.
2. A financial-data worker loads approved financial statements and emits a structured financial assessment with citations.
3. A legal-case worker examines relevant court-case data products and flags unresolved material cases, with source dates.
4. A policy worker checks the company's risk rules and identifies mandatory human review conditions.
5. An aggregator compares the evidence, surfaces disagreement and missing data, and drafts a recommendation.
6. A human approver accepts, requests more evidence, or rejects the proposed next step; only an authorized downstream system records the decision.

The financial, litigation, and policy checks may execute in parallel if their input data is independent. The final recommendation must wait until required checks finish or a clear partial-result policy applies. Every result includes provenance, version, and freshness.

This example illustrates why reusable data products matter: the quality of orchestration is limited by the trustworthiness of each downstream source and its contractual meaning. Data lineage and permissions should travel through the workflow rather than being reconstructed from the final prose.

A 30-day implementation plan

Week 1 : Baseline: Choose one real task, define outcome and risk thresholds, collect representative cases, and record the performance of a single agent and a deterministic workflow.

Week 2 : One added specialist: Introduce only the specialist with the strongest demonstrated potential. Define typed input/output contracts, authorized tools, durable state, and a maximum delegation budget.

Week 3 : Operate safely: Add cross-agent traces, timeouts, checkpoints, idempotency, authorization at action boundaries, and human approvals. Test duplicate deliveries, worker crashes, and stale evidence.

Week 4 : Compare and release: Run the same evaluation set across architectures. Promote the multi-agent workflow only if it meets mandatory controls and improves meaningful outcome, latency, or cost metrics. Canary the release, then turn production failures into regression cases.

Where DataKnobs KREATE fits

For organizations building data-rich workflows, DataKnobs KREATE provides a commercial starting point for creating AI agents and reusable data products. Pair agent-building capabilities with independently enforced policy controls and an evaluation discipline. A useful pilot is a read-only, multi-agent investigation workflow over trusted internal data products before expanding to consequential actions.

Frequently Asked Questions

What is the difference between a multi-agent system and multi-agent orchestration?

A multi-agent system is the collection of agents and shared services. Multi-agent orchestration is the process and control layer that routes work, coordinates state, handles failures, enforces transitions, and determines completion. A system can have several agents but weak orchestration: and consequently poor reliability.

Should I use a supervisor agent or a deterministic workflow?

Prefer a deterministic workflow when the steps and transitions are known. Use a supervisor when task decomposition and routing genuinely require contextual decisions. Hybrid designs are common: deterministic policy gates and state machines around agent-controlled reasoning tasks.

When does fan-out/fan-in reduce latency?

It helps when child tasks are truly independent, can execute concurrently, and their combined completion time: including aggregation: is less than sequential execution. Shared rate limits, the slowest worker, and synchronization overhead can erase the benefit.

Do MCP and A2A replace an orchestrator?

No. MCP provides interfaces to tools and context; A2A enables communication between independent agents. Neither decides the enterprise's release policy, workflow completion rules, action authorization, or recovery strategy on its own.

How do I prevent agents from passing sensitive information to one another?

Use scoped artifact references, classification labels, authorized retrieval, and trusted execution gateways. Avoid blindly copying full chat history between agents. Derive tenant identity and permissions from authenticated systems rather than model-generated messages.

How can I tell whether multi-agent AI is worth the additional cost?

Compare against a strong single-agent baseline on representative tasks. Track verified task success, critical failures, P95 latency, total cost per successful task, and failure recovery. Add agents only when the gains are material to the business objective and withstand repeated evaluation.

Conclusion

Effective multi-agent orchestration is less about assembling a large team of models and more about assigning ownership, narrowing context, controlling actions, preserving state, and verifying outcomes. Start with one agent. Add specialists only when they solve a measured problem. Keep side effects behind trusted gateways, and use repeatable evaluation to decide whether the extra coordination is justified.

The next practical step is to select a single enterprise workflow and test a supervisor or parallel pattern against its simpler baseline: then promote the design that produces the best verified business outcome within its risk, latency, and cost limits.

Sources

• LangChain: Multi-agent design and pattern selection
• Microsoft Agent Framework: Workflow orchestrations, updated August 25, 2026
• Microsoft Agent Framework: Handoff orchestration
• Microsoft Agent Framework: Human-in-the-loop workflows
• Agent2Agent protocol: Core concepts
• NIST: AI Risk Management Framework Playbook