The State of Context Engineering
The Job is Knowing When to Be Deterministic & When to Be Probabilistic
This week, I gave a short talk at AGNTCon titled, “Context Fails & Fixes,” that functioned as a sort of “State of Context Engineering.” How did context engineering emerge, where has it gone, and where might it go in the future? As model makers target new capabilities, they inadvertently create challenges we, as users and harness builders, must overcome.
Below are the slides and a lightly edited transcript.
Hi, everyone. Thanks for joining me today. We’re going to walk through context fails and fixes.
I have a book coming out in a bit, The Context Engineering Handbook. When I first talked to the team at O’Reilly, back in June 2025, they asked me a very reasonable question: “How do we know this won’t become obsolete in a few weeks?” The trouble with AI textbooks is that many age extraordinarily quickly.
I said, I don’t know, but I’m optimistic. Giving clear instructions about what to do and what not to do, and ensuring those instructions are consistent and not noisy, will forever be a challenge with AI, no matter how good it gets.
We have coworkers, partners, and other people we know and interact with who are sentient. They pass the Turing test, they are “AGI,” and yet: we still have a hard time conveying instructions, ideas, and concepts among each other.
Context engineering will continue to be hard. But it will be hard in unique ways, because models aren’t people. You can make assumptions about how models think or interpret things, but it helps to understand how they’re built, optimized, and trained. Their unique attributes and construction create a unique set of failure modes we need to reckon with.
So let’s review how we got here.
2024: The Promise of Long Contexts
Let’s flash back to the end of 2024. If you remember, this was the height of the “RAG is dead” moment. The hope was: if we can just make the context window large enough, we can put the entire internet in it. We don’t have to be selective. We don’t have to build agentic systems. We can fit it all in there, and AI will be solved.
Model builders started aiming at long context capabilities and we had a big leap from ~128,000-token context window models to million-token context window models, starting with Gemini 1.5.
If you look at the marketing material for Gemini 1.5 Pro, it was mostly about the million-token context window. They were even teasing a 10-million-token context window model they had in the lab, that would come out soon.
But they got beaten to the punch. Llama 4 Scout came out with a 10-million-token context window, and well, it wasn’t quite what we hoped. Inference was a challenge, and Scout never really caught on, and we never got another Llama. But we still got a million tokens. That’s pretty good. We’ll take it.
So this was the brief at the time: just put everything in the context and let the model sort it out.
More Context, More Problems
But with more context, we had room for more problems. Long contexts are not a panacea. I identified four key failure modes:
- Context poisoning: when a hallucination or error is repeatedly referenced.
- Context distraction: when the model over-focuses on a long context and neglects base knowledge.
- Context confusion: when superfluous information produces low-quality responses.
- Context clash: when new information and tools conflict with other information in the context.
Let me give you my favorite example of context poisoning. In the original Pokémon Red, you give a guard lemonade. In the 2004 remake, Pokémon FireRed, they changed it to iced tea. In the Gemini 2.5 technical report, researchers described how the model became convinced it needed to get tea, even though it was playing the older game. The tea task for FireRed was in its weights, so it poisoned its context. It spent hours trying to find or give the guard tea.
Another failure mode is context distraction. As your context gets longer, how does your behavior change? Recently, I stumbled on a paper that wonderfully illustrates this: Do as I Say, Not as I Do, by Carolina Camassa and Derek Shiller. They instructed a model to translate every question into Russian, then provided examples where the assistant answered in French. It turns out this affects every model; as your context grows, things fall apart.
Gemini 2.5 ran into this too. While it supports 1M+ token context, the paper observed that as the context grew significantly beyond 100k tokens, the agent showed a tendency toward favoring repeating actions from its vast history rather than synthesizing novel plans.
Tactics for Context Engineering
Thankfully, we’ve developed a pretty good toolbox. We have six tactics:
- Context Augmentation: RAG, skills, and other ways to bring in relevant information.
- Tool Loadout: selecting which tools and skills to expose.
- Context Quarantine: subagents and peer agents with blank histories.
- Context Pruning: stripping out tool results you no longer need.
- Context Offloading: storing information in the environment.
- Context Summarization: compaction.
These are the tools, and we’ve increasingly moved them into the harness. An agent is essentially a while loop. We can implement these fixes by passing history to a smaller decision model to prune, summarize, or select tools before the main model acts.
2026: The Promise of Long Horizons
We’ve moved these context fixes into the harness. What do we want next? We want agentic models that can drive our harnesses. We’re now talking about agents running for days, swarms of agents, and models that drive harnesses over much longer periods.
But this approach creates new failure modes. Two that we’ve found are:
- Fighting the weights: when the model doesn’t do what you ask because you’re working against what it was trained to do.
- Context inertia: models hyper-focus on a hypothesis, subtask, or goal in their context and keep pursuing it after they should stop.
Anthropic’s Memento and Microsoft’s Fork-and-Flush research both highlight that agents can become fixated on an incorrect idea. The fix is simple: periodically clear the context while keeping the artifacts produced in the environment. Every time it clears the context, you see another jump in performance.
This is the advice I keep giving now: Have you tried resetting your context? Sometimes that really is the fix.
The Future
Every advance in model capability changes what we delegate to the model and what systems must enforce. My prediction for 2027 is that as we focus more on memory, we also need to think about strategic forgetting. Over time, overly precise memories become conflicted and messy, and nobody has yet figured out how to handle this.
Our output is the product of the context and the weights. When building intelligent applications, harnesses, and workflows, our job is to choose when to be deterministic (what to put in code) and when to be probabilistic (when to let the model drive).
