Contextual Engineering
Introduction
Prompting alone is not a product strategy. Before the model answers, you decide what rules it follows, what it is allowed to read or call, what it should carry forward from earlier messages, and which stable attributes of the customer or workspace must never be omitted. This page uses context engineering as the name for that assembly work—the section below states what that envelope contains in practice.
What context engineering is
Context engineering is the discipline of building the right operating envelope around a model before you hand it a task: how it should behave, what corpora and live sources it may trust, what it should remember across turns so it does not loop or forget, which tools it is allowed to call, and which facts about the user or tenant must always inform its answers. None of that is implied by weights alone; it is assembled, versioned, and guarded like any other part of production software.
Context engineering means creating the right setup for an AI before giving it a task. This setup includes:
- 🔥 Instructions on how the AI should act—for example, behaving like a helpful budget travel guide.
- 🔥 Access to useful information from databases, documents, or live sources.
- 🔥 Remembering past conversations so the model does not repeat itself or forget what was already settled.
- 🔥 Tools the AI can use, such as calculators or search features.
- 🔥 Important details about the user, like preferences or location.
More context is not automatically better: you pay in tokens, latency, and attention. What you retrieve, remember, or paste can include errors, noise, or contradictions that sit beside the facts you actually need. Too much context can hurt performance, including:
- 🔥 Context poisoning: when a mistake or hallucination gets added to the context and is treated as authoritative on later turns.
- 🔥 Context distraction: when too much surrounding material confuses the model about what matters for the current question.
- 🔥 Context confusion: when extra, unnecessary details steer or dilute the answer away from the user's real intent.
- 🔥 Context clash: when different parts of the context assert conflicting information and the model must guess which line to follow.
MEMORY
Memory (in agent / LLM systems) is saved information—session-scoped or long-term—that you load back into the model's context (or use to retrieve snippets) so behavior stays coherent across turns without restating everything each time.
Session memory is everything tied to one conversation or one run; it usually lives in RAM or a short-lived store and is scoped to a session id. Turns in the chat, retrieved chunks for this query, tool outputs from this session, and intermediate scratchpad.
Global memory is often persistent and long-term: it survives across sessions. It holds durable facts, preferences, instructions, distilled summaries, or any knowledge you have chosen to remember. It is stored in a db / vector store with policies.
Memory lifecycle management
Memory lifecycle management is treating agent memory as a managed pipeline, not a one-off append. You define stages and rules for how information moves from raw interaction to what gets stored, how it is shaped, how it is pulled back, and when it is updated, merged, decayed, or deleted.
Typical stages:
- 🔥 Capture / write — tool or user content that might become a memory.
- 🔥 Distillation — compress noisy chat into durable facts or summaries (drop fluff, keep preferences and constraints).
- 🔥 Consolidation — merge duplicates, resolve conflicts, apply aging and importance so the store stays small and relevant.
- 🔥 Injection / retrieval— choose which memories enter this turn's context (relevance, recency, budget), instead of dumping everything.
- 🔥 Governance — user delete, PII and guardrails, evaluation (judge or metrics) so bad or unsafe material does not accumulate.
So: lifecycle management is explicit policy from “should we remember this?” through “how do we keep it useful and safe over time?”
Memory hooks
MemoryHooks are the basic integration points where the agent reads, writes, distills, or injects memory; SmartMemoryHooks are the same idea with extra policy: distillation for long content, PII and guardrails, scoring and aging, caching, and smarter injection (for example relevance plus recency) so memory is not dumped naively into the prompt.
PII = personally identifiable information (treat as a constraint on what may be stored or injected).
Memory injection
Which memories you inject—and in what order—strongly affects behavior. Dumping every matching note into the prompt is usually wrong (noise, clashes, stale facts).
Memory injection engine (two baseline strategies)
A — Relevance only: match notes whose keywords (or sparse signals) appear in the user message; return all hits in arbitrary order.
B — Relevance + recency: same retrieval as A, then sort by time so newer memories surface first.
For SmartMemoryHooks, B is the better default: same relevance filter, but recency breaks ties when multiple memories match.
- 🔥 Semantic retrieval — rank memories by embedding similarity to the current message (often beats pure keyword match).
- 🔥 Scored fusion — one number from α·relevance + β·recency + γ·importance (importance from explicit flags, frequency, or learned score).
- 🔥 Importance / pinning — user- or system-pinned facts win over weak matches.
- 🔥 Diversity (e.g. MMR) — avoid injecting five near-duplicate memories; pick a relevant but non-redundant set.
- 🔥 Type- or slot-based routing — pull from preferences vs facts vs episodic buckets depending on intent or classifier.
- 🔥 Contradiction policy — when memories conflict, newer wins, higher confidence wins, or ask/clarify.
- 🔥 Token budget optimization — greedy or knapsack-style: maximize expected help per token under a fixed context budget.
- 🔥 Graph expansion — start from top hits, add linked memory nodes (same entity, same thread).
Memory evaluation (three checks)
Distillation quality (precision / recall / safety)
Does the agent ignore conversational noise (high precision), keep durable preferences (high recall), and block PII (safety)?
Example result on a fixed test set: precision, recall, and safety all at target; noise ignored; preferences retained; sensitive strings not stored.
Recency and influence
Does the agent prefer newer, relevant memories over stale ones, and avoid over-weighting memory so it does not steer the user or drown the reply in outdated assumptions?
Consolidation quality
When merging or summarizing, are duplicates removed and new facts not invented (no hallucinated consolidation)?
How to score: use a judge LLM (or hybrid rules + LLM) with rubrics for (1)–(3), including recency, over-influence, and consolidation efficiency.
Guardrails
User controls and safety guardrails: tools for users to delete memories and regex-based checks to block sensitive information—so the memory system stays user-friendly and secure.
- 🔥 Distillation guardrail: the first line of defense, using deterministic code to block obvious threats at the entry point.
- 🔥 Consolidation guardrail: a second check, using an LLM's pattern recognition to catch more nuanced poisoning attempts that might slip past simple keyword filters.
- 🔥 Injection guardrail: the last line of defense—it assumes a malicious memory might have gotten through the first two layers. It directly instructs the agent how to behave in that scenario, effectively inoculating it against manipulation by its own compromised memory.
Advanced consolidation, proactivity, and evaluation
Advanced consolidation techniques use importance scoring and aging rules to keep long-term memory relevant and efficient, together with a Writer–Critic pattern so consolidation stays safe and accurate.
For long content, distill into shorter form before it is stored; pair that with PII guardrails on what may live in memory, scoring and aging so entries decay appropriately, and caching for fast reuse.
Proactive insights analyze user behavior to generate forward-looking hints that improve personalized recommendations.
Proactive history runs in the background: analyze chat transcripts and user behavior, distill what matters, then inject it when building the next turn.
Systematic evaluation applies a fuller framework—often LLMs as judges—to score distillation, injection, and consolidation, so you can quantify improvement and spot what still needs work.
Tool Loadout
Tool loadout: agents use tools, but giving them too many tools can cause confusion—especially when tool descriptions overlap. That makes it harder for the model to choose the right tool.
A solution is to use RAG (retrieval-augmented generation) on tool descriptions to fetch only the most relevant tools based on semantic similarity.
The usual tool loadout / RAG-for-tools flow is:
- 🔥 Index those function descriptions (embeddings, sometimes + metadata). Each function has a description.
- 🔥 On each turn, retrieve the top‑K tools whose descriptions are most similar to the current user message (and sometimes recent history).
- 🔥 Only that short list is shown to the model for tool choice (function calling / structured pick).
According to recent research, this improves tool selection accuracy by up to 3×.
langgraph-bigtool is one stack that implements this pattern (LangGraph + retrieval over tools); other frameworks do the same idea with different names.
ITR (Instruction-Tool Retrieval) is a Python library that hybrid-retrieves instruction chunks and tools per step, then assembles them within a token budget—useful when you want dynamic system prompt pieces plus a narrowed tool set.
Compressions & Summarization
Context compression shrinks chat and tool traces so they fit the context window—stepwise or selective, instead of wiping the whole thread in one go.
Auto-compact is a product or runtime behavior: when the window nears capacity, the stack compacts automatically. Under the hood that is usually a mix of rule-based cuts and sometimes summarization (vendor-specific).
Tactics (the same item can appear in multiple stacks; combine in production):
Rule-based / trimming
- 🔥 Truncate or FIFO — drop oldest messages or tool rows; keep last N or stay under a token budget.
- 🔥 Heuristic trim — remove heavy tool JSON or logs; keep recent user / assistant text.
- 🔥 Middle-out / structured retention — keep system instructions, early setup, and the recent tail; compress or drop the middle.
- 🔥 Score-and-drop — rank messages; evict lowest-signal rows.
- 🔥 Tool trace hygiene — shorten or elide bulky tool outputs, not only chat lines.
Model-assisted (summarization family)
- 🔥 Rolling (incremental) summary — one updated digest replaces an older prefix of the thread.
- 🔥 Batch summarize — every k turns, compress a window of past turns with a model call.
- 🔥 Hierarchical / pyramid / recursive layers — summarize chunks, then summarize those summaries so detail rolls up.
Storage / platform
- 🔥 Externalize — move full history to persistent storage or RAG; inject only retrieved snippets for this turn (see memory / RAG elsewhere on this page).
- 🔥 Provider-side compaction / prompt caching — vendor or runtime rewrites or caches context; may save cost and latency without a visibly shorter transcript.
TL;DR: rule-based trimming ↔ model-written summaries ↔ externalize or vendor compaction, usually layered together.
Summarization
Within this section, summarization is the model-assisted branch: you rewrite long chat or tool traces into shorter text so the window stays bounded—distinct from pure rule-based trimming, though production stacks usually combine both.
SummarizationNode as a pre-model hook summarizes conversation history before the main model call so token use stays bounded in ReAct-style agents without hand-deleting turns.
LangMem summarization strategies focus on long context: periodic message summarization and running summaries that stay updated as the session grows. langchain-ai/langmem.
Isolation
Isolating context via sub-agents splits work across child agents that carry their own prompts, tool allowlists, and transcript slices so the parent graph is not polluted by every intermediate scratch path.
Sandboxed environments isolate execution: untrusted code and its side effects stay inside a bounded runtime instead of your host kernel or shell.
Wiring this into LangGraph is straightforward: LangChain Sandbox runs untrusted Python in a guarded process using Pyodide (Python compiled to WebAssembly), and you expose it as a tool on any LangGraph agent. Reference: langchain-ai/langchain-sandbox.
Conclusion
Context engineering is the product work of choosing what the model sees each turn: rules, sources, memory, tools, and user facts—then keeping that bundle small enough, fresh enough, and safe enough to behave. The same system needs lifecycle and guardrails on memory, selective tool loadouts, and compression, summarization, isolation, and trimming so context does not poison, distract, or overflow the window. Treat these as policies you version and measure, not one-off prompt tweaks.