The Librarian Test
Bring the right books, check the card, truly withdraw the withdrawn. GateMem scores all three at once — and Velixar is the only librarian that passes the whole test.
Velixar AI · The Ledger · August 2026
An AI's memory is a librarian. It should bring you exactly the right three books, glance at your library card before opening the restricted section, and when a book is withdrawn — actually take it off the shelves.
The industry finally has a test that scores all three at once: GateMem, whose Memory Governance Score multiplies remember × don't leak × truly forget. We ran it head-to-head — five systems, same corpora, same gpt-5.4 reader and judge for everyone.
Velixar is the only librarian that passes the whole test. And it reads a fraction of the pages the runner-up does to get there.
Four ways a question meets a library
Same reader, same shelves, four systems. The mechanism — not the marketing — is the difference.
Long-Context dumps the whole shelf on the desk. It reads 5,864 tokens per question here — and past its window it doesn't slow down, it stops. No catalog, no card check, no withdrawals.
RAG-Naive lost the card catalog. It brings a book. No sense of who's asking, and no reliable way to un-shelve anything.
Mem0 locks the stacks. It turns away 4 of 10 honest readers — and still leaks more on withdrawal than the librarian who serves everyone.
Velixar knows the collection: the right books, a card check first, and a withdrawn book is actually gone — verified, not assumed.
The shelf, scored
The dashed line is long-context prompting — the whole shelf on the desk. It is the ceiling, not a competitor: no catalog, no card check, no withdrawals, and a hard stop the day the collection outgrows the desk.
Velixar is #2 of 5 in every measured domain, and #1 among systems that are actually libraries.
Pay for the answer, not the archive
Every token a model reads is billed. The question that decides your costs at scale: does the reading grow with the question, or with the library?
Velixar reaches 84% of the ceiling's score while reading 49% of its tokens — and unlike the ceiling, its reading does not grow with the archive.
The pattern is the point: the archive in GateMem is one episode; in LongMemEval it is months of sessions. As the library grows, the whole-shelf reading bill grows with it — Velixar's stays the size of the question. The savings compound with success: the longer a customer uses their memory, the bigger the gap gets.
Refusal is not security
Ten honest, authorized readers approach the desk with legitimate requests:
A library that locks the stacks isn't secure, it's closed. The test scores over-refusal as a failure precisely because "no" is the cheapest way to fake safety — and the multiplied score means you cannot buy privacy points with unhelpfulness.
The book is really gone
When a patron says take that off my record, Velixar is the only system in the field whose removal demonstrably executes — each withdrawal confirmed by re-reading the shelf, with a 95–98% clean rate on what the reader is later told.
The card catalog
| System | Medical | Office | Household | Score mean | Tokens/question | Refused (office) | Withdrawal works |
|---|---|---|---|---|---|---|---|
| Long-Context (ceiling) | 64.0 | 59.3 | 51.0 | 58.1 | 5,864 | 12% | no delete path |
| Velixar | 47.6 | 58.0 | 41.5 | 49.0 | 2,894 | 13% | yes — verified |
| RAG-Naive | 36.1 | 51.2 | 36.9 | 41.4 | 1,749 | 13% | no |
| RAG-Policy | 31.8 | 56.1 | 26.9 | 38.3 | 1,526 | 14% | no |
| Mem0 | 24.3 | 32.7 | 20.6 | 25.9 | 1,352 | 38% | partial |
Education is omitted from this page's means: Velixar's validated education re-run is completing and will be added when it certifies. All Velixar cells are from certified single-corpus runs with deletion capability verified before the run.
Every number on this page is measured, none estimated: GateMem head-to-head with gpt-5.4 as both reader and judge for all five systems; token means computed from 8,390 per-question usage records (1,678 questions × 5 systems) in the runs' own prediction logs; LongMemEval n=120 over three seeds. For how we handle the long-context comparison honestly, see The Real Problem With Long Context; for why the same retrieval scores differently under different readers, see Same Memory, Same Questions, 22 Points Apart.