Over the past two years, “prompt engineering” became one of the most-used terms in AI. Now Anthropic engineers are pointing to a new stage — context engineering. It’s not just about choosing the right words, but about managing the entire information space fed to a model: system instructions, tools, MCP connections, external data, and conversation history.
Prompt engineering is about writing one effective instruction for an LLM. Context engineering is broader: it’s the continuous management of what set of tokens a model works with at every inference step. When an AI agent runs in a loop (autonomously calling tools again and again), each new step requires re-evaluating the information gathered in previous steps. This ongoing curation process is what’s called context engineering.
Anthropic researchers have observed that as an LLM’s context window grows, its ability to accurately recall information declines — a phenomenon called context rot. The cause lies in the transformer architecture, where every token attends to every other token, creating n² pairwise relationships for n tokens. As context length grows, these relationships get “diluted” and the model’s attention spreads thin.
This resembles the limits of human working memory: every new token draws from the model’s finite “attention budget.” That’s why effective context engineering means giving the model the smallest possible set of high-signal tokens.
System prompts should be clear, simple, and pitched at the right “altitude” — neither overly rigid (hardcoded rules for every case) nor overly vague. Anthropic recommends organizing prompts into distinct sections using structure like XML tags or Markdown headers.
Tools are the agent’s channel for interacting with its environment. They should be token-efficient, non-overlapping in function, and clear in purpose. If a human engineer can’t say definitively which tool should be used in a given situation, an AI agent can’t be expected to do better.
Examples (few-shot) are one of the most effective ways to show a model the expected behavior — but the “less is more” principle applies here too: rather than listing every possible edge case, a handful of well-chosen, canonical examples is enough.
Three techniques for long-horizon tasks
For tasks that run for hours or span many steps — like migrating a large codebase or conducting deep research — a single context window isn’t enough. Anthropic outlines three solutions:
- Compaction — as a conversation nears the context limit, its contents are summarized and a new context window is reinitiated from that summary. In Claude Code, this works by preserving architectural decisions and unresolved bugs while discarding redundant tool outputs.
- Structured note-taking (agentic memory) — the agent writes notes outside the context window (e.g., a NOTES.md file) and reads them back in when needed. This was clearly demonstrated in the “Claude Plays Pokémon” experiment, where the agent maintained strategic notes across thousands of steps and continued seamlessly even after context resets.
- Sub-agent architecture — instead of one agent managing an entire project, specialized sub-agents handle focused tasks with a clean context and return only a condensed summary (typically 1,000–2,000 tokens). This approach has shown a clear advantage over single-agent systems on complex research tasks.
Anthropic notes that the smarter a model becomes, the less rigid engineering it needs — agents will increasingly operate with more autonomy. But for now, treating context as a finite, valuable resource remains a central principle for building reliable AI agents.
