Groq 3 LPX clocks 3,431 tokens per second

NVIDIA published a technical blog reporting the first third-party benchmark of its Groq 3 LPX inference accelerator, the interactive-serving part of the Vera Rubin platform. Artificial Analysis ran its 100K-context benchmark on Gemma 4 31B on Groq 3 LPX and measured 3,431 output tokens per second, a per-user interactivity figure NVIDIA frames as the enabler for long-running agentic sessions where context grows across hundreds of turns.

The post attributes the result to the LPX's deterministic execution model. The compiler sees the individual compute units across the 256 LP30 local processing units, the 128 GB of aggregate SRAM-based memory those chips hold, and the 96 chip-to-chip links per chip running at 112 Gbps each, then emits a cycle-accurate schedule before the workload starts. That removes real-time arbitration from data transfers: LPUs send on the clock cycle data is ready and consume it on the cycle it arrives. Links are point-to-point and each LPU can route as well as compute.

NVIDIA's argument is that tensor parallelism's order-of-magnitude speedups only survive at the very small batch sizes high interactivity demands if first-bit latency — link selection, endpoint synchronization, conflict arbitration — is driven to a minimum.

Caveats are substantial. This is a vendor blog following an earlier launch post, covering one model on an NVIDIA-hosted system. NVIDIA also claims Groq 3 LPX paired with Vera Rubin NVL72 will serve 2T+ parameter multiagent systems at long context; no third-party measurement is offered for that configuration.