Liz Zhang

Agent Memory

Same model, double the score: what the Company Brain track showed me

By Liz Zhang · · 4 min read

← Back to blog
Same model, double the score: what the Company Brain track showed me

Every company has a brain. It lives in tickets, pull requests, docs, chat threads, and a few people's heads. Ask an AI assistant who decided something, why, or whether it still applies. It finds the paragraph that looks most similar and hopes for the best.

We tested a better way at Graph Hacks, the online hackathon my team at FalkorDB (graph DB for agents) ran. The event drew 7,899 developers and 1,213 projects. 458 teams took the Company Brain track.

One result stands out from that track, so I will start there.

The result

inherit-brain, by Zeyu Sun, starts from a simple situation. A new maintainer inherits a product and asks who wrote the fixes for bugs that customers later reopened.

Answering that means joining people to pull requests, pull requests to bugs, and bugs to dated reopen events. Retrieval named four other engineers and scored zero on it. The graph join returned the answer in 13 ms, with the source record for every hop.

Zeyu then froze the graphs and tested on 1,203 questions the system had never seen. All four approaches used the same model.

ApproachScoreMedian input tokens
Graph tools80.010,117
Retrieval tools39.246,451
Whole product in one prompt26.0125,947
Hybrid RAG19.25,176

Three things in that table matter to me.

The model did not change. Only the way it finds facts changed, and the score more than doubled.

The graph sent about a fifth of the tokens that retrieval sent. Better answers cost less here.

The whole product in one prompt used 12 times the tokens of the graph and scored about a third as well. So a bigger context window would not fix this.

The setup is worth copying too. Zeyu loaded Salesforce AI Research's HERB dataset into 30 product graphs, with 56,708 nodes and 164,698 edges. Each edge keeps the IDs of the records it came from. The agent picks from nine fixed tools, and each tool runs a parameterized read-only query. The server rejects writes even if a tool's query is wrong.

Four more ideas from the track

Lineage, by Anmol Gupta. Python's design decisions sit across 26 years of proposals and forum threads, so old ideas keep coming back. Lineage puts 745 PEPs, 1,085 rejected ideas, 1,748 threads, and 2,175 people into one graph. Ask whether Python already decided something and you get the ruling, who made it, whether it still stands, and who to ask. Every company has its own version of this problem.

Ripple, by Sahil Mane and Harsh Gupta. Ripple records each answer its assistant gives, linked to the exact version of the fact it used. When a policy changes, one traversal finds everyone still holding the old version. Then it goes back and tells them.

Blackbox, by Divyam Singh and Manish Modak. Blackbox joins 4,235 NTSB accident investigations and 225,627 FAA maintenance reports by aircraft and by location on the airframe. For Southwest Flight 812, it shows cracks on the same stringer line rising from 9 to 18 to 31 aircraft before the 2011 rupture.

Staleproof, by Raj Karia. When a function changes, Staleproof follows the calls to every doc paragraph and code example that the change could make wrong. In one test across three repos, it reached 120 affected doc sections, 46 of them outside the repo that changed.

What I would do first

Pick one question your team keeps asking that crosses three systems. Load those systems into one graph, joined on shared IDs. Write the question as one query, and give your agent that query as a fixed, read-only tool.

If your agents also need to remember, read what 647 teams taught me about agent memory.

The full recap on the company blog has the queries and demos: Company brain for AI agents.