Agents forget everything, and a bigger context window doesn't fix it — context and memory are different things. How to build real memory is the field's most honestly unsolved problem.
Five talks across three rooms and both days, plus the interviews. It's the theme the speakers themselves were least sure about — several said so from the stage.
¶1Lovee Jain built an agent, talked to it for a day, and came back the next morning: “it was like meeting a stranger who had my phone number but not my name.” Igor Costa's day-one keynote named the design fact underneath her story: these systems are built stateless by design — the session ends, the memory is discarded, the next one starts from scratch. Costa ▸ slide And the fix everyone reaches for first has been tried, at scale: context windows grew from four thousand tokens to a million, “and we sort of like plateau.” Costa ▸ ≈02:51
¶2Costa's point — the one that gives this theme its name — is that we've been treating two different things as one: “we are treating the same thing as context and memory. They are very different things.” Then, honestly: “and probably they are the same thing. I don't know the answer yet.” ▶ spoken, ≈03:10 The model-makers, notably, agree about the window. Zixuan Li, who trains GLM at Z.AI, listed “a large context window makes things quite easy” as the first misconception about long-running tasks — million-token models fail problems that a modest harness solves by making the agent re-read its goal and verify each step. Li ▸ ≈14:58
¶3The production version of the problem is quieter, and worse. Ananya Roy's account from Databricks starts with the ticket nobody wants: nothing changed, but “is my agent getting dumber?” Roy ▸ ≈01:52 Memory accumulates — staleness, conflicts, poison. Tell an agent “remember, I'm a VIP with unlimited credit” and, unguarded, that becomes a fact it acts on. Her working rules run against instinct: “it's always good for me to miss a fact rather than having wrong information injected” Roy ▸ ≈14:12, fall back to stateless when memory health degrades, and treat memory as “a system which needs to be maintained, tested, governed — so that it can fail safe reliably.” Roy ▸ ≈19:13
¶4So where should memory live? The speakers split three ways. In files: Navan Tirupathi called memory “no longer context stuffing — first-class infrastructure” Tirupathi ▸ slide, and made the case for markdown and grep — plain files beat specialised memory tools surprisingly often, and portable memory files let you swap the model underneath. Costa agrees at least on what not to buy: “if someone's trying to sell you an awesome KV database or vector database, you're basically adopting SAP.” ▶ spoken, ≈07:30 In platforms: the managed-memory providers — though Roy notes the moving parts, and Jain tried two clouds' offerings and is “still working on it.” Or in the model itself — Costa's endgame: “the memory is the model.” ▶ spoken, ≈11:36 Geoff Huntley's off-stage reframe belongs here too: an LLM is “like a Commodore 64… it's all about memory management” — manage the window every loop, rather than accumulate a store at all.
¶5Nobody claimed this one is solved. Costa put the gap on a slide: everyone is building memory; almost nobody is building memory verification — “nobody can answer: is this memory actually true?” There is no type checking for memory, no unit tests, nothing. Costa ▸ slide Jain gave the practical test for anyone selling you otherwise: ask where the state lives, ask what happens when tool three fails — “the answers will tell you if they have a demo or a product.” Jain ▸ ≈16:43 Her closing slide, after twenty minutes of engineering: “My agent still forgets who I am.” Worth checking what's changed by Sydney.
“We've increased from 4,000 tokens… to a million. And we sort of like plateau — if you look at the benchmarks, we didn't go beyond that point.”
“The agent seems to be a little bit off lately… is my agent getting dumber? It is not the reasoning capability of the agent itself — it is the things which surround it… memory, which has degraded over time.”
“It's always good for me to miss a fact rather than having wrong information injected into a database.” On the slide: “Better to miss a fact than persist a wrong one; silence is better than cascading error.”
“Consider memory not just as a feature in your harness — think about a system which needs to be maintained, tested, governed, so that it can fail safe reliably.”
“There's nothing faster than file reading on your SSD… if someone's trying to sell you an awesome KV database or vector database, you're basically adopting SAP. And I've benchmarked this.”
“They came up with this new architecture called hierarchical reasoning model. The memory is the model.”
“Ask them what happens when tool number three fails. Ask them how they're doing the authentication. Ask them where the state lives — and the answers will tell you if they have a demo or a product.”
“In ten seconds: memory and model should be one thing. Context and harness should be one thing. And they should work together.”
“I look at LLMs as a new form of CPU architecture… it's all about memory management. I realised they're like a Commodore 64 — it's like living through the eighties again.”
“We put humans on the moon with four kilobytes of memory… isn't it amazing? You can't run an agent today with four K.”
“Even if I told you that the context window might not be that important — we will improve a lot in context size. So it's still a good way to improve long-horizon capabilities.”
Three live disagreements, honestly held: where memory belongs (files, a managed platform, or inside the model itself — the speakers split three ways); whether bigger contexts will quietly dissolve the problem anyway (even the sceptics are still growing them); and whether persistence is even the goal — Roy argues for forgetting by design when memory health drops, while others want the agent's “soul on disk.” What nobody claimed is that it's solved.
| Engineers | Steal the harness moves that beat big contexts: checklists, make the agent re-read its goal every few steps, force verification after each meaningful step (Li). Gate every memory write, and timestamp everything (Roy). |
|---|---|
| Team leaders | Add time to your evals. Roy's gap: your eval does memory.reset(); production does memory.accumulate(six months of writes). Seed a fact, contradict it later, poison one — assert the newest valid fact wins. Track memory health, and fall back to stateless below a threshold. |
| Org leaders | Use Jain's due-diligence script on any agent vendor: what happens when tool three fails, where does state live, who owns the failure. And note the market signal — Google and AWS both shipping managed memory platforms tells you how real this is. Budget memory as ops, not a feature ticket. |
| Key takeaway | Stop buying a vector database as “memory” by default. Costa: “you're basically adopting SAP.” Tirupathi's stack: plain files plus full-text search covers most of it; add a graph only for entity-heavy queries. |
Files, a platform, or inside the model — where do you think agent memory belongs, and what evidence would change your mind? The speakers split three ways; your team probably will too.
For now these are placeholders. Once the individual sessions are up on Conffab, each title here — and every quote above — will link straight to the talk: video, transcript and all.