Stanford releases MEMORILLA memory system using 16 tokens

Researchers at Stanford have developed MEMORILLA, a memory system that encodes document collections into reusable embeddings and writes 16 query-specific tokens for a frozen LLM. The approach significantly reduces token consumption and runs faster than full-context evaluations, though it struggles with exact details like names and verbatim text retrieval. Open article →