AI Agent Context Engineering: Build Reliable Context Pipelines

AI agent context engineering is the design of the information an agent receives at each model call: instructions, the current task, permitted tools, retrieved evidence, task state, and selectively recalled memory. It is not the practice of putting everything into a large prompt. It is the discipline of providing the smallest sufficient, authorized, current, and verifiable set of information for the next decision.

That distinction matters in enterprise systems. An agent answering a policy question may need an approved policy document and its effective date. The same agent initiating a refund needs live transaction state, user authorization, allowed tools, and a business rule. Both tasks may use the same model, but they require different context.

A dependable context pipeline should answer five questions before inference: What does the agent need now? Where does each fact come from? Who is allowed to see it? Is it current? How will we know the selection helped?

Context engineering vs prompt engineering, RAG, and memory

Prompt engineering is concerned with the clarity and format of instructions. Context engineering determines what information is assembled and exposed to the model throughout a workflow. Retrieval-augmented generation (RAG) supplies evidence from an external knowledge source; memory retains selected information across steps or sessions. These are components of context engineering, not competing replacements.

The distinction becomes clearer in long-running workflows: a prompt can remain unchanged while the context must be rebuilt after every tool call. For deeper storage strategies, see AI agent memory design and the RAG architecture guide.

Why bigger context windows do not solve the problem

More available tokens are helpful, but they do not guarantee more correct decisions. Context can include stale policies, duplicate passages, contradictory records, tool results from an unrelated user, and instructions embedded in untrusted documents. Putting all of that into a larger window can raise cost and make the agent less reliable.

Anthropic's September 2025 engineering guide describes context as finite and emphasizes selecting a compact set of high-signal tokens. LangChain's July 2025 guide organizes the main techniques into write, select, compress, and isolate. These approaches address both quality and resource constraints. See the Anthropic engineering guide and LangChain's context-engineering framework.

For enterprise systems, the main failure modes are concrete:

• Missing context: the relevant exception clause was never retrieved.
• Stale context: an old policy overrides a more recent effective version.
• Conflicting context: two systems define “active account” differently.
• Context overload: redundant tool output crowds out authoritative evidence.
• Cross-scope leakage: records from another user or tenant enter the package.
• Context injection: an untrusted document attempts to become an instruction.

The objective is not maximum context utilization. It is maximum task success subject to authorization, freshness, latency, and cost.

Reference architecture: the context assembly pipeline

Treat context assembly as a distinct application component rather than a string-concatenation helper. A practical architecture is:

User request + verified identity
             |
             v
    Intent and risk classifier
             |
             v
    Context policy / permissions
             |
       +-----+--------------------+
       |                          |
       v                          v
Current task state         Authorized retrieval
       |                  (SQL / APIs / RAG / Graph)
       |                          |
       +------------+-------------+
                    v
           Rank and deduplicate
                    |
           Validate freshness
                    |
            Compress / isolate
                    |
             Context package
                    |
           Model / agent step
                    |
            Trace and evaluate

An important boundary appears near the top: authorization and source eligibility are decided before retrieval results are eligible for inclusion, not delegated to the language model after retrieval. Tool execution and high-impact actions have their own policy checks even when the context looks correct.

1. Keep the context sources explicit

Label context by origin and purpose:

• System instructions: task boundaries and output contracts owned by the application.
• User intent: the current request and confirmed constraints.
• Task state: verified current workflow status, pending actions, deadlines.
• Knowledge evidence: retrieved documents, records, authoritative definitions.
• Memory: useful prior facts with provenance and retention rules.
• Tool metadata and results: available capabilities and observed execution outcomes.

Do not let retrieved text silently impersonate higher-priority instructions. Document passages are evidence, not policy authority over the agent's behavior.

2. Store provenance with each evidence item

For an enterprise agent, the context assembler should know more than the passage body. Record a source ID, version, effective time, retrieval time, authorization scope, and the specific task or claim the passage supports. This is especially important for financial, legal, compliance, and customer-account workflows.

3. Build different packages for different steps

A research step may need broad retrieval and tool discovery. A final answer step may need only the accepted evidence and conclusion. An action step needs live state, exact arguments, authorization, and confirmation rules. Replaying the complete history in all three stages wastes context and increases risk.

Apply the four operations: write, select, compress, isolate

The framework is useful when implemented as explicit engineering decisions rather than buzzwords.

Write: retain useful state outside the prompt

Persist task progress, source references, confirmed constraints, and important observations in a structured store or checkpoint. Do not persist unreviewed guesses as authoritative facts. Use stable IDs so that the agent can re-fetch the underlying record.

For example, save supplier_id=8421, review_period=2026-Q3, and policy_version=18 rather than a long generated narrative that may be difficult to verify later.

Select: retrieve only what this step requires

A selection policy can combine keyword search, embeddings, metadata filters, SQL, and graph traversal. Apply tenant, role, jurisdiction, effective date, and source status before ranking evidence. Select by task relevance and authority, not similarity alone.

A semantic near-match can still be wrong: a refund policy for US customers may resemble a UK policy but contain different deadlines. Source metadata must disambiguate the result.

Compress: preserve decisions and evidence, not noise

Compress repeated history and verbose tool responses, but keep material exceptions, numerical values, citations, contradictions, and unresolved questions. Prefer structured extracts over free-form summaries for high-impact fields. Store a reference to the full result so a reviewer can inspect the original.

Google's Agent Development Kit includes context-compaction features that summarize older interactions; this is a helpful implementation pattern, but a summary is still generated text and may omit or distort details. Preserve access to authoritative source state. See Google ADK's context-compression documentation.

Isolate: separate scopes and concerns

Maintain boundaries between users, tenants, subagents, tasks, and trust levels. Store large tool outputs outside the model context and expose only relevant validated fields. For multi-agent workflows, pass scoped work products and provenance rather than the entire private history of one agent to another.

This complements multi-agent system design, but context engineering focuses on what each participant can see, not on the full orchestration strategy.

Build a context budget that follows the task

A context budget is a planning rule, not a universal percentage. Reserve space for core instructions, the current request, live state, evidence, tool schemas, and expected output. Leave headroom for tool results and later turns. Do not fill the model's maximum window merely because capacity is available.

For illustration, suppose an application targets 20,000 input tokens for a complex review step:

These are example allocations, not industry benchmarks. A transaction agent may allocate more to live state and much less to long documents; a research agent may do the opposite. The correct mix should come from evaluation data.

Useful budget knobs include max_evidence_tokens, max_tool_result_tokens, retrieval_top_k, rerank_cutoff, history_turns, summary_trigger, and reserved_output_tokens. Keep the configuration versioned and observable.

Retrieval: start simple, then add structure

A robust retrieval pipeline generally begins with source selection and authorization, then combines lexical and semantic search where useful, reranks candidates, deduplicates overlapping chunks, and retains citations.

For enterprise use cases, an effective sequence is:

1. Identify the domain, task, and allowed source collection.
2. Apply tenant, role, region, effective-date, and document-status filters.
3. Retrieve candidates with keyword, vector, SQL, or API methods.
4. Rerank by relevance and source authority.
5. Detect duplicates, stale records, and conflicting statements.
6. Assemble evidence with stable source references.
7. Check whether evidence sufficiency is adequate; otherwise retrieve more or escalate.

Do not use a vector database as the only source of truth for precise live values such as an account balance, product inventory, or transaction status. Query the authoritative operational system when state matters.

When does GraphRAG help?

Graph traversal becomes useful when the answer depends on relationships: customer → product → entitlement → policy exception, or supplier → subsidiary → legal case → region. In those cases, combine relationship-aware retrieval with textual evidence. It adds complexity and should be justified by evaluation, not assumed to beat standard RAG for every question.

Related specialist guide: vector databases.

Memory must have expiry, provenance, and correction

Persistent memory and immediate context are different things. A preference may be safe to recall for weeks; an order status can change in minutes. Define which facts are durable, which must be refreshed, and which should never be remembered.

A memory item needs at least an identity and scope, source, timestamp, confidence or review status, and a deletion/correction mechanism. For sensitive settings, require explicit retention and access policies. If new evidence conflicts with recalled memory, current authoritative data should win and the discrepancy should be logged.

In particular, memory generated from prior agent responses is not automatically trustworthy. Errors can become persistent if a summary or inference is repeatedly read back as fact.

Tool results are data, not instructions

Agents receive external text from search engines, documents, messages, webpages, databases, and MCP servers. That material can contain malicious instructions. Treat retrieved content and tool output as untrusted input even when the source is helpful for factual grounding.

Recommended controls include:

• preserve clear trust boundaries among system instructions, user requests, and external data;
• avoid placing secrets in prompts or tool descriptions;
• restrict tool exposure by task and principal;
• validate tool arguments and authorization outside the LLM;
• strip or quarantine suspicious control text from retrieved material;
• use human approval for consequential actions;
• test indirect prompt injection through source documents and tool results.

OWASP's AI security guidance explicitly identifies cross-user retrieval leakage, untrusted MCP content, and context poisoning as security concerns. See the OWASP LLM security guidance and the AI agent security guide.

A practical context-package contract

The runtime needs an auditable data structure rather than an opaque prompt blob. The following is an illustrative application-level schema, not a standard required by any model provider:

{
  "context_version": "support-v12",
  "task": "Explain refund eligibility",
  "principal": {"tenant_id": "t17", "role": "support_agent"},
  "policy_scope": {"region": "US", "effective_at": "2026-10-10"},
  "evidence": [
    {
      "source_id": "returns-policy-v18",
      "title": "Returns Policy",
      "valid_from": "2026-10-01",
      "retrieved_at": "2026-10-10T09:00:00Z",
      "relevance": 0.93,
      "authorization": "allowed"
    }
  ],
  "live_state_refs": ["order:8742"],
  "max_evidence_tokens": 4000,
  "unresolved_questions": [],
  "trace_id": "trace-example-123"
}

The context assembler can then build the model-visible request from validated fields. The full source text can be fetched only for authorized references. Keep provenance in the trace so it is possible to reconstruct a decision without logging unnecessary personal content.

Enterprise example: a supplier-risk investigation

Consider an agent answering: “Is Supplier Northstar approved for the new EU contract?” A naïve system retrieves the nearest text and produces a persuasive answer. A governed system first identifies the supplier entity and jurisdiction, then loads verified account state, current approval policy, outstanding sanctions checks, and any relevant legal records.

Its context may include:

If the due-diligence status is missing or contradictory, the correct response is not an invented approval. The agent should mark the decision as pending and request the required check. Context engineering enables that behavior by representing uncertainty and absence explicitly.

This pattern pairs naturally with reusable AI-ready data products and an agent's governed tool integration.

Evaluate context quality, not just answer quality

A large amount of context may correlate with a fluent answer without improving the underlying task. Evaluation should compare context configurations on the same test cases using frozen source snapshots where possible.

Measure several levels:

A rigorous test corpus should include near-duplicate policies, superseded documents, cross-tenant lookalikes, missing evidence, long tool responses, and adversarial text. Test both false inclusions (irrelevant or prohibited data appears) and false omissions (necessary facts are missing).

Run experiments rather than tuning by intuition

Compare variants such as top_k=5 versus top_k=10, lexical-plus-vector retrieval versus vector-only, or raw versus compressed tool results. Keep model, task set, and sources fixed while changing one major dimension. Promote a configuration only if quality and safety gates hold and the latency/cost trade-off is acceptable.

This is where a context system becomes an optimization platform. See AI agent evaluation for broader outcome and trace grading.

Observe every context decision in production

For each agent step, capture context version, eligible collections, selected source IDs, filter outcomes, token counts by category, compression applied, retrieval latency, freshness checks, policy denials, and model or agent result. Avoid retaining full sensitive source text unless necessary and authorized.

An operator should be able to answer: “Why did the agent include this document and exclude the updated policy?” The decision trace should connect retrieval and context selection to the final output and its supporting evidence.

Use AI agent observability to track those traces and tie them to regression results.

Make context choices measurable with KNOBS

Retrieval breadth, reranking cutoff, memory lookback, compaction frequency, source freshness, evidence requirements, and tool visibility are adjustable operating parameters. KNOBS is the natural DataKnobs capability for comparing these settings against task success, groundedness, latency, cost, and risk. KONTROLS establishes the mandatory permissions and policy boundaries that no optimization setting may override.

Explore DataKnobs KNOBS for agent optimization to turn context selection from scattered hard-coded choices into versioned experiments and operational controls.

Implementation roadmap for enterprise teams

First 30 days : establish a baseline. Inventory source systems and agent workflows. Trace what the model currently sees. Define authorization boundaries and an initial gold dataset of real tasks and failures. Start with a deterministic context assembler and simple authorized retrieval.

Days 31–60 : improve relevance and reliability. Add source metadata, hybrid retrieval, reranking, freshness checks, task-specific token budgets, and structured compression. Introduce negative test cases for leakage, stale policy, and malicious tool outputs. Compare configurations against the baseline.

Days 61–90 : operationalize. Add context-version release gates, production tracing, dataset feedback from incidents, controlled knobs, and rollback. Extend to memory and multi-agent isolation only where those capabilities clearly improve outcomes.

The best initial architecture is rarely the most complex one. Build the smallest pipeline that gets authoritative context to the right step, prove it works, and expand only when evaluation shows a benefit.

Frequently Asked Questions

What is context engineering for AI agents?

It is the systematic selection, assembly, validation, compression, and isolation of instructions, evidence, memory, state, and tool information provided to an agent at each model step. Its objective is to improve task reliability while controlling cost, privacy, and freshness.

How is context engineering different from prompt engineering?

Prompt engineering improves how instructions are written. Context engineering governs the entire information package around those instructions, including retrieved evidence, live state, memory, tools, and subsequent updates during a workflow.

Is context engineering the same as RAG?

No. RAG is one common mechanism for supplying external knowledge. Context engineering determines when to retrieve, what sources are allowed, which passages deserve tokens, what state to include, and how to handle contradictions or stale information.

Should AI agents use the largest available context window?

Not automatically. Larger windows provide capacity, but irrelevant, stale, or contradictory content can raise cost and hurt reliability. Use a task-based budget and evaluate the effects of additional context on verified outcomes.

How do you prevent sensitive data leaking through agent context?

Apply identity and tenant authorization before retrieval, filter sources at the data layer, avoid cross-tenant shared context, minimize prompt logging, and enforce action permissions outside the model. Include adversarial and cross-scope tests in the evaluation suite.

What metrics matter most for context engineering?

Track evidence recall, unauthorized inclusion, freshness, groundedness, task completion, token usage, latency, and cost per successful task. Compare performance by workload, risk tier, source, and configuration version rather than relying on a single overall score.

Conclusion

Reliable agents are not built by filling the prompt window. They are built by controlling the flow of authorized, current, relevant information across each step of a real task. Use write, select, compress, and isolate as practical operations. Keep authoritative data and permissions outside the model, preserve provenance, evaluate on real outcomes, and treat context settings as versioned knobs that can be measured and improved.

Sources

• Anthropic: Effective context engineering for AI agents (September 29, 2025)
• LangChain: Context Engineering (July 2, 2025)
• Google Agent Development Kit: Context compression
• OWASP: Guidance on cross-user retrieval and untrusted agent context