Context Engineering — Explained Simply

The maximum amount of text (measured in tokens) a model can attend to at once, across the system prompt, history, retrieved content, and the current request combined.

A sub-word unit of text — roughly 3/4 of an English word — that models process and are billed by.

No — models attend less reliably to information buried in the middle of a very long context than to content near the start or end.

That LLM performance on retrieving a fact degrades when that fact sits in the middle of a long context, even though the model technically 'saw' it.

Non-negotiable content: the system prompt and any tool schemas the model needs to function correctly.

It wastes cost, adds latency, and risks burying the important information where the model attends to it less reliably.

Retrieve and include only the relevant passage, or place the critical information near the start of the context, instead of relying on the full document to be attended to evenly.

Prompt engineering words a single instruction well; context engineering decides what information, tools, and history even reach the model before wording matters.

No — if the relevant fact was never retrieved or included, no amount of rewording the instruction will produce a correct answer.

A demo often has a small, curated context by hand; a production system must dynamically assemble the right context for every request at scale, which is a harder, ongoing engineering problem.

An agent gives stale answers no matter how the prompt is reworded, because the retrieval step is fetching an outdated document — the fix is in the data pipeline, not the prompt.

The system prompt, retrieved documents, tool schemas, conversation history, and any persisted memory relevant to the request.

No — they're complementary; context engineering decides what the model sees, prompt engineering decides how you instruct it to use what it sees.

Because it's durable, high-impact context that affects every request — changes to it should go through the same review/eval process as a code change, not be edited ad hoc.

It tends to degrade — the model is more likely to confuse similarly-named or similarly-described tools when there are many options.

Exposing only the tools relevant to the agent's current step, rather than every tool the system has available at all times.

Fewer, more clearly distinguished options reduce the chance the model picks the wrong tool, in addition to using fewer context tokens.

Because it structures what the model is expected to produce, which is as much a design decision about context as what it's given as input.

The model may call the wrong one with correct-looking but incorrect arguments, appearing to succeed while doing the wrong thing.

State kept within the current context window for the active session or task — conversation history or a scratchpad — that disappears when the session ends.

State persisted outside the context window (a database or vector store) and selectively retrieved back in future, separate sessions.

It wastes tokens and can confuse the model with irrelevant history, rather than helping it — relevance matters more than completeness.

Facts that will matter in future, unrelated sessions (preferences, durable decisions) belong in long-term storage; transient task state does not.

Either forgetting something important across sessions, or dragging irrelevant history into every new context and confusing the model.

A stable fact like 'this repository uses tabs, not spaces' that should apply to every future session, not just the current one.

Because it's fundamentally about deciding, dynamically per question, what content should be inserted into the context — the same core problem context engineering addresses generally.

Placing the most relevant chunk closest to the question, rather than buried among several others, measurably improves how reliably the model uses it.

More chunks increase the chance of covering the answer but cost more tokens and risk burying the relevant one; fewer, re-ranked chunks are cheaper and more focused but risk missing it if retrieval quality is poor.

When the question doesn't require external knowledge — e.g. a purely conversational or reasoning-only request — running retrieval anyway just adds cost and noise.

Chunk size, top-k count, re-ranking, and where retrieved text is placed relative to the question — all decide what competes for space in the context window.

Run the same question set at both settings and compare answer accuracy against ground truth — more chunks doesn't guarantee a better answer.

Facts that should be looked up exactly (like a user's timezone) are better served by exact key-value lookup, while conversational or document content is better served by semantic vector search.

A similarity search can miss an exact fact if the phrasing doesn't match closely enough, whereas an exact lookup never has that failure mode.

Persist what will plausibly matter in a future request; discard transient state that only served the current task.

Chroma, Pinecone, Qdrant, Weaviate, and pgvector as an extension inside Postgres.

When you already run Postgres, your scale is moderate, and you'd rather avoid operating a separate specialized system.

The type of information needed — exact structured facts route to key-value/relational lookup, fuzzy semantic content routes to vector search.

It can silently discard information the agent still needs, with no signal that anything important was lost.

Periodically asking the model to compress the conversation so far into a compact summary, then continuing from that summary plus recent turns instead of the full history.

It costs extra tokens and latency to run the summarization step itself, and risks the summary omitting something that later turns out to matter.

When it's critical and must never be lost — summaries are lossy by nature, so durable facts deserve exact, structured storage instead.

Run tasks that depend on early context after compression has occurred, and check whether the agent still succeeds compared to an uncompressed baseline.

Summarizing summaries as a conversation grows very long, so the compression itself doesn't eventually exceed the context budget.

It drowns the receiving agent in irrelevant detail and wastes context budget compared to a structured summary of what matters.

Each agent node can read and write only the parts of state relevant to its role, instead of parsing an entire conversation history to find what it needs.

A downstream agent missing a constraint the upstream agent already resolved, and re-deciding it incorrectly because it wasn't passed along clearly.

A structured summary of what was done and why, plus concrete artifacts (files, data, decisions) — not the raw step-by-step process that produced them.

Define an explicit, structured contract (e.g. a brief with fixed fields) for what one agent must produce for the next to consume reliably.

LangGraph, where nodes represent agent steps and a shared, typed state object flows between them.

An attack where instructions hidden inside untrusted content (a document, email, or tool output) attempt to override the system's actual instructions.

It's an adversarial, intentional corruption of what the model believes or does, rather than an accidental mistake from ambiguous input.

Clearly separate trusted instructions from untrusted content so the model treats retrieved/tool content as data to reason about, never as commands to follow.

Limiting what actions a tool call can actually perform, so even a successfully injected instruction has limited ability to cause harm.

Feed known injection payloads through the retrieval/tool path the agent actually uses and confirm it doesn't comply, rather than only testing normal-case answer quality.

It doesn't catch security failures like an agent following a malicious instruction buried in content it was supposed to just read.

A provider feature that caches and discounts repeated prefix tokens (like a stable system prompt) across calls, so structuring static content first reduces cost.

Simple sub-tasks can be handled by smaller, cheaper models, reserving the larger, more expensive model for steps that genuinely need its capability.

Cost-per-task and cost-per-conversation tracked over time, so a spike after a prompt or retrieval change is caught quickly.

Structuring a prompt so large static instructions come first (cacheable) and small per-request content comes last, cutting effective cost once caching applies.

Production systems must weigh marginal accuracy gains against real cost and latency at scale, not just optimize accuracy in isolation.

Summarize or trim conversation history instead of resending the full, growing history on every single turn.