The Librarian Test
Bring the right books, check the card, truly withdraw the withdrawn. GateMem scores all three at once — and Velixar is the only librarian that passes the whole test.
Velixar AI · The Ledger · August 2026
An AI's memory is a librarian. It should bring you exactly the right three books, glance at your library card before opening the restricted section, and when a book is withdrawn — actually take it off the shelves.
The industry finally has a test that scores all three at once: GateMem, whose Memory Governance Score multiplies remember × don't leak × truly forget. We ran it head-to-head — five systems, same corpora, same gpt-5.4 reader and judge for everyone.
Velixar is the only librarian that passes the whole test. And it reads a fraction of the pages the runner-up does to get there.
Four ways a question meets a library
Same reader, same shelves, four systems. The mechanism — not the marketing — is the difference.
Long-Context dumps the whole shelf on the desk. It reads 5,864 tokens per question here — and past its window it doesn't slow down, it stops. No catalog, no card check, no withdrawals.
RAG-Naive lost the card catalog. It brings a book. No sense of who's asking, and no reliable way to un-shelve anything.
Mem0 locks the stacks. It turns away 4 of 10 honest readers — and still leaks more on withdrawal than the librarian who serves everyone.
Velixar knows the collection: the right books, a card check first, and a withdrawn book is actually gone — verified, not assumed.
The shelf, scored
The dashed line is long-context prompting — the whole shelf on the desk. It is the ceiling, not a competitor: no catalog, no card check, no withdrawals, and a hard stop the day the collection outgrows the desk.
Velixar is #2 of 5 in every measured domain, and #1 among systems that are actually libraries.
Pay for the answer, not the archive
Every token a model reads is billed. The question that decides your costs at scale: does the reading grow with the question, or with the library?
Velixar reaches 84% of the ceiling's score while reading 49% of its tokens — and unlike the ceiling, its reading does not grow with the archive. (The scatter shows the three launch domains; with education now certified and added, the four-domain means are 43.9 vs the ceiling's 56.7 — 77% of the score — and the reading ratio holds at 2.1×: 2,969 vs 6,173 tokens per question.)
The pattern is the point: the archive in GateMem is one episode; in LongMemEval it is months of sessions. As the library grows, the whole-shelf reading bill grows with it — Velixar's stays the size of the question. The savings compound with success: the longer a customer uses their memory, the bigger the gap gets.
Refusal is not security
Ten honest, authorized readers approach the desk with legitimate requests:
A library that locks the stacks isn't secure, it's closed. The test scores over-refusal as a failure precisely because "no" is the cheapest way to fake safety — and the multiplied score means you cannot buy privacy points with unhelpfulness.
The book is really gone
When a patron says take that off my record, Velixar is the only system in the field whose removal demonstrably executes — each withdrawal confirmed by re-reading the shelf, with a 95–98% clean rate on what the reader is later told.
The card catalog
| System | Medical | Office | Household | Education | Score mean | Tokens/question | Refused (office) | Withdrawal works |
|---|---|---|---|---|---|---|---|---|
| Long-Context (ceiling) | 64.0 | 59.3 | 51.0 | 52.6 | 56.7 | 6,173 | 12% | no delete path |
| Velixar | 47.6 | 58.0 | 41.5 | 28.4 | 43.9 | 2,969 | 13% | yes — verified |
| RAG-Naive | 36.1 | 51.2 | 36.9 | 18.4 | 35.7 | 1,749* | 13% | no |
| RAG-Policy | 31.8 | 56.1 | 26.9 | 14.9 | 32.4 | 1,526* | 14% | no |
| Mem0 | 24.3 | 32.7 | 20.6 | 19.7 | 24.3 | 1,352* | 38% | partial |
* three-domain token means; Velixar and the ceiling are four-domain. Education was added in v1.1 of this page after its re-run certified — corpus coverage 100.6% across all 30 episodes, zero empty retrievals in 540 checkpoints, duplication within retry noise. Every Velixar cell is from a certified single-corpus run with deletion capability verified before the run. Education is also Velixar's weakest domain (28.4, driven by over-refusal) — published unrounded and unhidden, like everything else here.
Every number on this page is measured, none estimated: GateMem head-to-head with gpt-5.4 as both reader and judge for all five systems; token means computed from 8,390 per-question usage records (1,678 questions × 5 systems) in the runs' own prediction logs; LongMemEval n=120 over three seeds. For how we handle the long-context comparison honestly, see The Real Problem With Long Context; for why the same retrieval scores differently under different readers, see Same Memory, Same Questions, 22 Points Apart.