lhadzic377-netizen/TokiRecur: Recycled-prefill LLM toolkit for KV cache reuse

Toki Recur is an open-source, cross-platform KV cache reuse toolkit for local LLM inference that matches exact prefixes in front of llama-server. Benchmarks show significant improvements in time to first token and prefill throughput, with the largest gains observed on large mixture-of-experts models. Open article →