NVIDIA says Groq 3 LPX is in full production, pairing it with Vera Rubin to speed agentic AI inference

NVIDIA on Monday said its Groq 3 LPX rack-scale system is now in full production, and that it is extending its Vera Rubin inference platform with hardware tuned for “decode latency,” a problem that becomes more noticeable as AI agents generate responses token-by-token. In a blog post focused on the needs of agentic systems, the company described the goal as keeping inference responsive while also sustaining high throughput across very large context windows.

The announcement centers on a specific performance claim. In an Artificial Analysis benchmark running Gemma 4 31B, an open-source agentic model, NVIDIA said the Vera Rubin rack-scale system equipped with Groq 3 LPX generated 3,400 output tokens per second for 100,000-token long-context use cases. NVIDIA also said the result was 4x faster than the nearest alternative platform in that benchmark scenario.

NVIDIA framed the move as a shift from treating AI infrastructure as a set of separate components to codesigning the full “AI factory.” As models move from training to serving intelligence and reasoning, the company argued that inference is becoming the new frontier, with agentic AI driving higher token counts, larger context windows, and more cross-system collaboration. It said these workloads require infrastructure optimized for throughput, responsiveness, and economics at scale.

On partner adoption, NVIDIA cited multiple deployments and customer plans. SpaceXAI, it said, announced that NVIDIA Vera CPUs will power its next generation of agentic AI. CoreWeave, meanwhile, said it has deployed into production Spectrum-X Multiplane, a networking approach intended to connect Vera Rubin racks with multiple parallel switches for a high-bandwidth, flat and lossless AI network design. NVIDIA also said Nebius is the first AI cloud to adopt NVIDIA Groq 3 LPX.

NVIDIA’s story also ties the inference stack to networking. The blog highlighted NVIDIA Spectrum-X Ethernet and its “Multiplane” and cross-datacenter extensions as mechanisms to keep large-scale data movement predictable, with fewer added latency and cost layers than alternatives that require a third network tier. NVIDIA described Spectrum-X Multiplane as splitting each server’s network connection into multiple independent “planes,” each running a lightweight two-tier network, and said the design can scale to 512,000 GPUs without the complexity of a third tier.

Beyond compute and Ethernet, the post pointed to NVIDIA’s broader infrastructure push for agentic era operations. It introduced a new networking pillar called NVIDIA Scale-In, described as an accelerated infrastructure services layer for managing and securing the AI factory. NVIDIA said Scale-In is powered by a BlueField-4 processor and NVIDIA DOCA software, and that it is intended to transform the traditional north-south access network into a unified accelerated domain while keeping infrastructure services independent of host compute resources.

In describing how Groq 3 LPX fits into Vera Rubin, NVIDIA said the system is engineered to accelerate token generation for agentic workloads where decode latency is critical. It said the combination works by using Vera Rubin GPUs for large-scale context processing while LPX accelerates latency-sensitive decode workloads. The company characterized the intended effect as faster, more predictable token generation that can improve interactivity and infrastructure efficiency.

For the underlying architecture, NVIDIA described Groq 3 LPX as combining the strengths of GPUs and LPUs, again positioning this as the product of extreme codesign rather than optimizing a single component in isolation. It said Rubin GPUs and LPUs jointly compute every layer of an AI model. At rack scale, NVIDIA described deployments that can include up to 256 LP30 accelerators connected through direct chip-to-chip links, presented as an inference engine optimized for deterministic performance across large fleets of units.

NVIDIA did not provide additional details in the blog post about broader commercial terms, availability windows for specific customers, or whether the 3,400 tokens-per-second figure is representative across model types and serving configurations. The performance result is tied to a named benchmark setup, so it remains unclear how the same uplift would translate across different agent architectures, different batch sizes, different customer network topologies, or different operational targets such as sustained multi-tenant concurrency. It also did not disclose the specific “nearest alternative platform” used for the 4x comparison.

Looking ahead, NVIDIA’s positioning suggests the next milestones to watch are deployments and independent verification of long-context, low-latency agent serving performance using Vera Rubin plus Groq 3 LPX, along with evidence that networking designs such as Spectrum-X Multiplane and Scale-In are delivering predictable end-to-end results at large scale. For investors and customers, the key question will be whether the company can sustain the token factory proposition in real production workloads where models, context sizes, and traffic patterns vary over time.