Qwen Cloud Hackathon · Track 1, Agent Memory

Agent memory that stays true as your life changes.

The field builds memory as retrieval over a growing log of turns, and that log silently breaks the moment a fact changes. Tenet reframes agent memory as a self-consistent, bi-temporal belief state: every fact carries two clocks, event time and transaction time, so a new value supersedes the old one instead of sitting beside it. The payoff is temporal correctness with zero LLM calls on the read path, and a belief state you can actually open and read.

By Anas Elghoudane · independent researcher · built end to end on Qwen Cloud

▶ Live demo 📄 Paper (PDF) Play with the memory graph

The Sator square. TENET is its center, reading the same forward and backward, in row and in column. So does this memory: recall(as_of=…) replays your beliefs in either direction of time.

97.0

MemoryAgentBench FactConsolidation, single-hop SubEM, 95% CI [94.8, 98.3], above the published gpt-4o-tier pooled score (94.8)

100→50%

naive-RAG's current-value accuracy as one fact is updated 2 to 12 times. Tenet holds 100% throughout

~11ms

flat read latency from 1,000 to 100,000 facts, zero LLM calls in the read path

4 OFF

mechanisms we built, measured, and shipped disabled because the data said no benefit

One belief, one current value. Seattle is current as of 2025-12-31; Boston is retired, not deleted. Full interactive version below.

The problem

Retrieval memory silently breaks when facts change

Most agent memory today is retrieval over a log: every message an agent has ever seen stays equally retrievable forever, and a top-k search hands the reader whichever passages are closest to the query, old or new. That works until a fact is updated more times than the retrieval budget can hold. Once the number of stale versions of a fact exceeds k, the current value can fall out of the retrieved set entirely, and the agent answers with a version of the truth that stopped being true.

We measured this directly. A single fact, updated 2 to 12 times across a distractor-laden history, retrieval budget k=6, 15 distractors, 12 independent principals per point:

Current-value accuracy as one fact is updated N times

naive-RAG vs Tenet · k=6 retrieval budget · BENCHMARK.md §3

Tenet naive-RAG

Tenet holds 100% at every point; naive-RAG loses fifty points, falling from 100% to 50% once stale versions outnumber the retrieval window. Bi-temporal supersession keeps exactly one current value per key regardless of how many times it changed, so the failure mode simply cannot occur. This is not a reader problem: the same collapse reproduces whether the reader is a light gpt-4o protocol reader, a local 7B backbone, or Tenet's own product-tier qwen3.7-plus reader (§3, §6.2b). It is a structural property of a fixed retrieval budget under churn, not reader weakness. A harder, paraphrased version of this stress test complicates the story honestly; see Results and the blog for the full account, including where we falsified our own pre-registered claim before fixing it.

Methodology

Five mechanisms, write time to read time

None of this works without a data model built for it. Every fact Tenet stores carries a stable subject::attribute key and two independent clocks: event time (valid_at / invalid_at, when it was true in the world) and transaction time (created_at / expired_at, when the system learned it). Five mechanisms sit on top of that model, three at write time and two at read time.

Keyed distillation (write time)

One LLM call turns a raw message into atomic, keyed facts (user::residence → "Seattle") instead of a chunk of text. The key is load-bearing: a value change scores 0.988 cosine similarity against its own prior value, higher than a plain rephrasing of the same fact (0.79). Embeddings alone cannot tell "changed" from "restated", which is why extraction has to happen at write time, not just retrieval at read time.

measured: 0.988 vs 0.79 cosine · BLOG.md

Bi-temporal supersession (write time)

A later message with the same key supersedes the current belief: the old row's invalid_at / expired_at are stamped closed, and the new value is inserted current. Nothing is overwritten and nothing is deleted, so the full history stays queryable at any past instant.

55% → 100% current-value accuracy once wired · BENCHMARK.md §4

Surprise-gated writes (write time)

An observation the store already predicts (cosine ≥ 0.97 to an existing belief) carries no new information and is never stored, a predictive-coding write filter. RAG stores everything by default; Tenet does not.

15% of turns dropped as redundant, zero accuracy loss · BENCHMARK.md §4

Belief-evidence consistency (read time)

A raw turn that echoes an already-superseded value for a key that already has a current fact in the pool is confirmed stale and is retired from recall, scoped narrowly to that key's own value chain so the threshold can go lower than a global echo filter safely could.

cosine 0.70 default: 100% stale-echo recall at 7% false-positive cost · BENCHMARK.md §9.1

Belief-anchored expansion + navigate() (read time)

Spare context budget is spent on query-relevant raw evidence from the sessions the belief state already surfaced, closing the gap to one-shot accuracy parity with RAG at equal-or-fewer tokens. An adaptive-depth navigate() knob deepens multi-hop recall while a relevance-gain gate keeps firing, LLM-free either way.

parity 57.5% = 57.5% at fewer tokens · BENCHMARK.md §2 navigate(): no measured accuracy benefit at tested tiers · §6.2a

Interactive

Watch the belief state, not a feed

This is a real Tenet belief board rendered as a graph: one entity, four keyed attributes, and each attribute's own supersession chain over event time. Drag the slider to move as_of. The graph re-renders exactly as recall(as_of=t) would: a solid node is current at that instant, a struck, faded node was superseded before it, and a dashed ghost was not yet true.

Belief board · the user · residence, job title, car, gym

current as of the slider date superseded before the slider date not yet true as of the slider date

recall(as_of=…)

2025-10-08

Beliefs use the same four attributes ChurnBench stresses (residence, job title, car, gym), rendered here on a small, hand-built ledger for illustration. The as-of resolution rule matches the real store: latest valid_at ≤ the query date among facts already true at that instant, zero API calls, zero network. This is exactly what recall() and navigate() do server-side over thousands of facts.

Live demo

Watch memory stay true

A real assistant session, simulated in your browser with the exact semantics of the shipped engine: ingest pays the one LLM call at write time; every recall, time-travel and navigate below is the LLM-free read path. Step through it, then hit the deployed instance for the real thing.

session

belief state · get_all()

Same story as scripts/demo_agent.py: keyed distillation at write time, supersession instead of overwrite (struck values stay queryable), and reads that never call a model.

deploy · alibaba cloud

Deploy it yourself

The Function Compute deployment that served this demo during judging has been retired. The one-command deploy that produced it is documented end to end.

mcp · claude desktop & IDEs

MCP server

Drop Tenet into any MCP client: learn / recall / time_travel / doubts as native tools.

terminal · zero api key

60-second local demo

python examples/00_zero_key_demo.py: the whole LLM-free read path, fully offline.

A real recorded session of the shipped assistant: facts change, the belief state supersedes them, time-travel recalls what was true before.

Evidence

Every number below is sourced, not vibes

Every result on this page ships with a raw evidence artifact and a one-command reproduce line in the repo. Scoring is each benchmark's own official metric, applied verbatim. Every reported accuracy carries a Wilson 95% confidence interval. Where a strong baseline beats Tenet, that is reported too, not hidden.

MemoryAgentBench · FactConsolidation (arXiv:2507.05257)

Serial-numbered facts with counterfactual updates; the question requires the current value. Official SubEM metric, official reader prompt copied verbatim, local qwen2.5:7b reader, zero-LLM deterministic ingestion, all 800 questions, zero exclusions.

Pooled SubEM, 6K to 262K context

BENCHMARK.md §6 · raw evidence factcon_results.json

Tenet naive-RAG (same backbone)

cell	naive-RAG	Tenet	published mini-SOTA	published gpt-4o
SH pooled (6K–262K)	47.8	97.0 [94.8, 98.3]	78.0	94.8
MH pooled (6K–262K)	4.5	45.8 [40.9, 50.6]	30.2	51.5

Single-hop lands above even the published gpt-4o-tier pooled score, on a local 7B reader. Multi-hop is 1.5× the published mini-tier SOTA (the CI excludes it) and still short of the gpt-4o tier, reported plainly. Both numbers follow a 2026-07-19 ingestion-keyer fix our own miss-file audit exposed (pre-fix: 86.5 / 30.0, both runs preserved in the artifact).

Knowledge churn: two regimes, one honest ledger

The templated churn primitive above is pre-registered to structurally favor Tenet. A harder, parametric stress test, ChurnBench, generalizes it to paraphrased, multi-attribute updates. There Tenet's first measurement failed its own pre-registered ship gate; a default-on read-time consistency fix recovered most of it. Full story in the blog below; the numbers:

U (updates per fact)	TENET (fixed)	RAG	Mem0-style (idealized)
2	98.0 [89.5, 99.6]	96.0 [86.5, 98.9]	90.0 [78.6, 95.7]
8	92.0 [81.2, 96.8]	100.0 [92.9, 100.0]	100.0 [92.9, 100.0]
32	82.0–98.0^*	30.0 [19.1, 43.8]	100.0 [92.9, 100.0]

^*Run-dependent across reader non-determinism and which fix components are stacked; churn half-life is 32 either way. BENCHMARK.md §9, §9.1.

Honest ceiling: Tenet ties the idealized delete-outright Mem0-style arm at U=32, it does not beat it on raw accuracy. That idealized arm is generous to Mem0, though: it deletes superseded memories outright, so nothing stale can ever leak. The real mem0ai package does not behave that way.

arm (live head-to-head)	current-value acc.	stale-leak rate	reads
Tenet	100.0 [88.6, 100.0]	0.0%	LLM-free
naive-RAG	100.0 [88.6, 100.0]	0.0%	LLM-free
real `mem0ai` 2.0.12	73.3 [55.6, 85.8]	26.7%	LLM per add

McNemar Tenet-vs-Mem0: 8 wins, 0 losses, p=0.0078. The real package answers with a superseded value more than a quarter of the time; Tenet never does. COMPARISON.md §A.2, raw evidence mem0_h2h_result.json.

Reads that stay flat as the store grows

facts in store	recall() end-to-end
1,000	9.0 ms
10,000	10.1 ms
100,000	10.8 ms

Latency is flat, dominated by the local embedder's fixed per-query cost, not store size; before a resident-index fix it was 1,054.9 ms at 100k, a roughly 98× win. Ingest throughput moved the same way after the fix: 16,353 facts/s on a cold store, 17,695 facts/s at 10,000 pre-existing facts, and 7,441 facts/s at 100,000, a real, reported residual dip the team has not fully re-attributed, versus 13.3 facts/s at 100,000 before the fix. No LLM call sits anywhere on this path. SCALE.md, raw evidence scale_results.json.

Second axis: accurate retrieval over long contexts

MemoryAgentBench's other core competency, 22 contexts of 197K–534K tokens, four sub-benchmarks, official per-sub-benchmark metrics, matched gpt-4o-mini reader: Tenet averages 59.3, second of all published systems (Mem0 32.6, Zep 37.5), and beats every published memory framework on EventQA (70.7 vs 67.6, CI excludes). RULER multi-hop is the honest loss against HippoRAG-v2's graph chaining (45 vs 66). BENCHMARK.md §7.

Four mechanisms shipped default-OFF

Every one of these was built, measured on real benchmarks, and disabled because the measurement said no benefit or an outright regression, exactly the discipline a field with a reproduction problem needs. COMPARISON.md.

TENET_AGG_READER

CAR-style read-time max(serial) aggregation. Clean null: FC-MH 15.0 → 15.0, LoCoMo 29.0 → 28.0. Redundant with a store that is already conflict-free at write time.

TENET_RETRACT

A tombstone / delete op. Measured regression on PersonaMem's retraction subset, 67.7 → 50.8 (CIs nearly disjoint). Removing context the reader needs to recognize a retraction happened.

TENET_RAW_RECALL

Raw-turn-favored recall mode. Measured regression on LoCoMo, the one verbatim regime Tenet loses: overall 28.0 → 24.0, temporal 9.5 → 0.0.

TENET_USAGE_RECALL

ReMe-style usage-scenario retrieval fusion. Clean null on ChurnBench: 97.3% pooled both ways, redistributed by one question, noise at this n.

A fifth, write-time consolidation (TENET_CONSOLIDATE, porting Mem0's delete-outright trick), was tested later and also measured no benefit over the default read-time machinery. BENCHMARK.md §9.2.

From the lab notebook

The blog

Your memory benchmark is grading on a broken curve

Anas Elghoudane · Global AI Hackathon with Qwen Cloud, Track 1: MemoryAgent · ~6 min

Every "AI memory" leaderboard number you have seen, Mem0's 92.5 on LoCoMo included, is downstream of a benchmark whose ground truth almost nobody checks. We started this project the normal way: read the field, pick a benchmark, beat a baseline. Partway through we read the one artifact almost nobody reads, the benchmark's own ground-truth data, and the number underneath the number did not hold up.

An independent audit of LoCoMo's ground truth found 99 of 1,540 QA pairs, 6.4%, have score-corrupting errors, wrong dates, wrong answers, category mislabels, which caps the benchmark's own theoretical ceiling at 93.6%. A perfect memory system, graded against LoCoMo's actual labels, cannot score above that ceiling. It gets worse on the grading side: LoCoMo's official gpt-4o-mini judge, tested against deliberately wrong but topically adjacent answers, accepted 62.8% of them as correct. A judge that lenient inflates every system it grades, and it inflates verbose, topically-broad retrieval more than terse, precise answers, which is the shape of a systemic bias, not noise. Three separate write-ups, over a year apart, converge on the same conclusion.

The honest response is not to stop citing LoCoMo. It is to stop trusting a single LLM-judged benchmark as the only evidence, and build evaluation where a broken curve is not possible in the first place.

Two properties make a benchmark hard to grade-inflate: the correctness check is a program, not a language model with its own biases, and the uncertainty on the resulting number is reported, not implied. So we leaned on deterministic evaluation: exact-match against the one currently-true value, no judge, nothing to lean on. The churn primitive above is one such test. MemoryAgentBench's FactConsolidation, with its official substring-match metric and Wilson confidence intervals on every score, is another. And to rule out "you just picked a better reader," we reimplemented four published memory mechanisms, CAR, Mem0-style consolidation, HippoRAG-v2-style OpenIE + PageRank, MemAgent-style rolling overwrite, as arms of the same harness: same reader, same embedder, same questions, same metric. Tenet led every arm on both axes, and the categories where it loses are published next to the ones where it wins.

Full post with every citation: docs/BLOG.md. Audit sources: dial481/locomo-audit and the independent Zep write-up on Mem0's LoCoMo methodology.

We falsified our own claim before you could

Anas Elghoudane · on ChurnBench, ship gates, and flags that default to OFF · ~4 min

ChurnBench, the harder paraphrased churn stress test, was pre-registered with a ship gate: Tenet's churn half-life had to lead the best baseline by 2×, CI-separated. It did not. Tenet scored worst of every arm tested, including at U=2, barely any churn at all, worse than plain naive-RAG. We published that failure rather than quietly rerunning until it looked better.

Diagnosing it found the real bug: stale raw turns for already-superseded values were surviving a write-time echo filter tuned for near-verbatim matches, not paraphrases. A narrower, key-scoped read-time consistency rule fixed most of it, churn half-life rose from under 2 to 32, and we shipped the fix as the new default with the regression suite green. The partial part stayed partial: it ties an idealized delete-outright baseline, it does not beat it, and we say so.

A benchmark you only publish when you win is the same failure mode as LoCoMo's, just wearing your own logo.

The same discipline runs through the codebase: five mechanisms were built, measured on real benchmarks, and shipped default-OFF because the measurement said no benefit or an outright regression, including one that regressed retraction handling by seventeen points. Every one of those nulls is documented next to the wins, because a field with a reproduction problem needs negative results a lot more than it needs another headline.

The measured flags: COMPARISON.md · the full ChurnBench story: BENCHMARK.md §9.

Reported, not hidden

Honest limitations

Multi-session synthesis is protocol-dependent

On the light protocol reader, RAG still leads (43% vs 57%, up from 29% after belief-anchored expansion): these questions need evidence from several sessions, and expansion only deepens sessions the top-k already surfaced. On Tenet's own product-tier Qwen reader the category flips in Tenet's favor (75.0 vs 54.2). Both numbers are real; which one applies depends on reader strength, and we report both. BENCHMARK.md §8.

Temporal multi-hop is a reader ceiling, not a memory one

navigate() already retrieves the facts a multi-hop question needs across hops; retrieval is not the bottleneck. The loss is the reader composing those facts into a chained answer. A knowledge graph would not fix a reader-composition limit, and it would cost the graph-database infrastructure Tenet deliberately avoids, so we do not build one. COMPARISON.md.

Churn half-life ties, does not beat, an idealized baseline

The read-time fix lifts Tenet's ChurnBench half-life from under 2 to 32, run-dependent 82% to 100% at U=32, but an idealized delete-outright Mem0-style arm still holds a flat 100% there. The real edge is against the actual mem0ai package, which does accumulate stale copies and loses a live head-to-head 100% to 73.3%, not against the idealized arm on raw churn accuracy.

Distillation quality is a ceiling on everything downstream

Supersession, dynamics, and staleness doubts are only as reliable as the keys the write-time LLM call assigns; a mis-keyed fact either fails to supersede or wrongly clobbers something unrelated. Validated at hackathon, single-user scale, brute-force cosine over sqlite, with no load-testing at multi-tenant volume.

The paper

Read the full write-up

Eight pages: the bi-temporal data model, the five mechanisms from write time to read time, the complete benchmark ledger with Wilson confidence intervals, the ChurnBench falsification story, and the honest limitations. Every number traces to a raw evidence artifact committed in the repo.

📄 Paper (PDF) Markdown source Benchmark log vs Mem0 / Zep / Letta

Cite

@misc{tenet2026,
  title  = {Tenet: Agent Memory as a Self-Consistent Belief State},
  author = {Elghoudane, Anas},
  year   = {2026},
  note   = {Global AI Hackathon with Qwen Cloud, Track 1},
  url    = {https://github.com/Nas01010101/tenet}
}