Progress on provable inference for large language models

Anthropic researchers have released FlashProve, an open-source cryptographic proof system designed to enable provable inference for large mixture-of-experts language models. The system uses zkSNARKs to let a model provider prove that a response was generated by a specific set of model weights without revealing the weights themselves. According to the authors, the prototype generates proofs at approximately 110 tokens per second on a single H100 GPU for a 400-billion-parameter model. Open article →