NVIDIA claims HBM hits 1000 tok/s on AgentX

According to a new NVIDIA AgentX submission, HBM chips can achieve over 1000 tokens per second per user on giant 700B+ models using agent swarm-written software. Furthermore, the Rubin NVL72 system delivers 20.9x better performance per TCO than the GB300 NVL72 at ultrafast speeds of 325 tokens/s/user, and up to 9.11x better performance per TCO at 200 tokens/s/user. Open article →