Agent Memory
Why I told a room of agent builders that memory is the SSD

Agents get worse the longer they run. They get less accurate, slower, less private, and more expensive. That was my talk at Agent Memory Night on October 8.
We hosted the night at the AWS Builder Loft in San Francisco as an a16z #SFTechWeek event. The hosts were The Multimodal Society, which I founded, GMI Cloud, and my team at FalkorDB (graph DB for agents). 652 people signed up, which put us in the top 6% of all SF Tech Week events.
We set one rule for the speakers. Each talk had to answer one real builder complaint. Vasilije Markovic, the founder of Cognee, went deep on how to structure memory. Kyle Cabigon of TwelveLabs and Roan Weigert of GMI Cloud spoke too.
You can watch my talk here. This is the short version.
I changed one word in Karpathy's analogy
Andrej Karpathy has an analogy I love. The model is the CPU. The context window is the RAM. Memory is the disk.
The first two land perfectly. I changed the third. For an agent, a disk is too slow. Memory should be the SSD. A graph stores the facts and how they connect, so the agent reads less and gets more right.
The four things that break
I put one number on each problem.
| What breaks | Flat vector or full history | Graph memory |
|---|---|---|
| Accuracy | 76% on LongMemEval | 89% on LongMemEval |
| Latency | 10 to 17 s per turn after 100 messages | 0.12 s of query time for all 25 questions |
| Privacy | At least 7 in 10 queries return another user's memory | One graph per user, no shared index |
| Cost | 20M tokens for one inbox | 4.4M tokens |
The accuracy numbers come from Hu et al., "Does Memory Need Graphs?" (ACL 2026).
The privacy number comes from the MemLeak study (Mudgal et al., NeurIPS 2026). A lot of teams keep all users in one store and filter by user ID. With pooled same-team retrieval, at least 7 in 10 queries brought back someone else's memory. Nobody had to attack anything.
The latency and cost numbers are mine. Nobody waits 17 seconds for an agent. And 20 million tokens is $200 at list price for one inbox, or $2M for 10,000 users.
Why I built my own benchmark
I wanted numbers anyone could check, so I open-sourced agent-memory-bench. It gives one agent six memory setups: full history, compaction, a memory tool, vector, SQLite, and graph. The model, prompts, and tools stay the same. Only memory changes.
Every engine gets its best setup and prompt caching stays on for all of them.
What I would build
- Split memory into two loops. A slow background loop pulls out facts with a small LLM call, which cost $0.31 for 200 messages. A fast loop sends one query per turn and loads only what the agent needs.
- Close old facts. Tom hands the Payments on-call shift to Emma. If you only append, the agent holds both and has to guess. End Tom's shift, add Emma's, and "who's on call now?" has one answer.
- Retrieve in one query. Use vector or full-text search to find a starting point, follow the relationships, and keep only what is still true.
- Give every user their own graph. Then you have no shared index to filter and nothing to leak. Deleting a user means dropping one graph.
- Watch the write side. Merge duplicate names into one node before you save facts.
Closing a fact takes one small query:
MATCH (:Entity {name: 'Payments'})
-[r:ON_CALL]->(:Person {name: 'Tom'})
WHERE r.valid_to IS NULL
SET r.valid_to = timestamp()
Thanks to AWS for the room, to GMI Cloud and The Multimodal Society for co-hosting, and to everyone who spent a Thursday night talking about agent memory.
The written version of the talk, with every source and the full recipe, is on the company blog: Agent memory for long-running agents.