Progress on provable inference for LLMs
The gist
Anthropic released FlashProve, an open-source zkSNARK prototype proving Llama 4 Maverick responses at ~110 tokens/second on one H100.
Proofs cost 70-95x the GPU time of plain inference, so trust is compute-expensive.
How it unfolded
- 2024zkLLM proved a 13B model at about 2.5 tokens/second on one A100.
- date unknownVerInf, described as the only earlier proof of Llama 4 Maverick Anthropic knows of, reported 14.3 hours for 1,000 tokens on a DGX Spark, with a 93.6 GB proof that took almost a day to verify.
- Sep 30, 2026Anthropic's Frontier Safety Roadmap committed to a prototype of provable inference by this date; Anthropic says it met the goal through a separate but related project that signs inference traffic and recomputes inference on a trusted server.