Google releases TPUv7 agentic inference benchmarks

SemiAnalysis released benchmark results showing that Google TPUv7 delivers competitive performance per dollar compared to NVIDIA accelerators on long-context agentic inference workloads. According to the report, TPUv7 matches or exceeds newer NVIDIA chips on several FP8 configurations using open-source execution stacks like vLLM and SGLang, though NVIDIA's FP4 hardware retains advantages in specific throughput metrics. Open article →