The announcement, made August 24 at Hot Chips and detailed across NVIDIA’s Newsroom, corporate blog and technical blog, moves the Groq-derived hardware from a GTC 2026 architecture reveal into volume production. It is relevant to Windows and enterprise IT teams because the target workload is the sort increasingly attached to developer platforms: coding agents that inspect repositories, invoke tools, execute tests and keep expanding their working context during a session.
Full production does not mean developers can use it today. NVIDIA’s own benchmark ran on a system it stood up in its data centers. Nebius is identified as the first AI cloud adopter, but its commitment is phrased as a plan to bring the hardware to Nebius Token Factory. Groq separately says it will be among the early adopters and is working with Dell Technologies on deployment. Those are meaningful customer commitments, but neither amounts to a generally available endpoint at a known price.
LPX is built for the part of inference users actually wait on
The central argument behind Groq 3 LPX is that modern agentic applications have two different infrastructure problems. First, a service has to ingest and process a potentially huge prompt, including source files, tool outputs, system instructions and prior turns. Then it has to generate the response one token at a time, often while repeatedly calling tools and starting the cycle again.
NVIDIA assigns the first job—prefill and much of the attention work—to its Vera Rubin GPUs. LPX is intended to accelerate the latency-sensitive decode portion, particularly the feed-forward work performed for each new token. NVIDIA’s corporate blog calls this “extreme codesign,” but the operational implication is more concrete: LPX is not being sold as a general replacement for Rubin GPUs. It is a specialized companion rack that is useful only when a deployment is already designed around a disaggregated, multi-component inference path.
That distinction matters for capacity planners. A conventional GPU server can serve a mix of training, batch inference and interactive inference depending on model and software configuration. LPX is aimed squarely at a premium tier of interactive serving where single-user generation speed and latency consistency are worth dedicating specialized hardware to the decode stage.
NVIDIA’s technical blog makes the case in systems terms rather than marketing language. At very small batch sizes—the mode required to make one person’s agent feel immediate—the startup cost of coordination among chips can rival or exceed the time saved by splitting computation across them. The company says LPX reduces that “first bit” communication delay with direct LPU-to-LPU links and compiler-scheduled movement of small data chunks. It also claims the compiler begins transmitting a 320-byte result vector before an entire matrix operation finishes.
For an IT team, this is a reminder that headline accelerator performance is not enough to assess an agent service. A token-per-second result can be excellent while the service still has poor tool-call latency, overloaded routing, rate limits, slow retrieval, constrained context caching or queueing under concurrent traffic. LPX addresses one expensive slice of the loop, not the entire user-visible application path.
The 3,431-token result is real—but it is a controlled comparison
NVIDIA’s most concrete performance claim comes from Artificial Analysis testing of Gemma 4 31B with 100,000 input tokens of context. NVIDIA’s technical post gives the precise median figure as 3,431 output tokens per second; the Newsroom release and partner announcements round it to 3,400. NVIDIA says that was roughly four times the 870 tokens per second achieved by the fastest public endpoint in the comparison.
StorageReview independently highlighted an important qualification in the comparison: the LPX figure came from a private pre-release NVIDIA endpoint tested on August 21, while the alternatives were live, public serverless endpoints. That does not invalidate the measurement, and Artificial Analysis is a recognized benchmark operator rather than an NVIDIA internal test. But it means the result demonstrates the potential of a dedicated, carefully configured LPX system under the stated model and context conditions—not a promise that an API customer will receive 3,431 tokens per second under shared production load.
The benchmark is also model-specific. Gemma 4 31B is a 31-billion-parameter dense model. It does not establish identical results for mixture-of-experts models, proprietary frontier models, customer fine-tunes, multimodal requests, different quantization formats, or a deployment where the same racks are shared among many simultaneous users.
NVIDIA’s “coding in minutes versus hours” language should be read the same way. Its technical post illustrates that generating 5,000 output tokens at 3,431 tokens per second takes around 1.5 seconds, compared with 50 seconds at 100 tokens per second. That arithmetic is sound for pure output generation. But an actual coding agent spends material time reading files, searching an index, calling a compiler, running tests, waiting for networked tools and deciding which action to take next. LPX can compress the model-generation portions of that workflow; it cannot turn a slow CI job or an unreliable tool integration into an instant operation.
The hardware scale explains why this is a cloud story, not a PC product
A rack-scale LPX deployment can contain 256 LP30 accelerators connected through direct chip-to-chip links, according to NVIDIA. The surrounding system includes liquid-cooled compute trays, host CPUs, networking or BlueField-4 data-processing hardware, and a paired Vera Rubin NVL72 environment. This is data-center infrastructure built for AI providers, not an accelerator headed for a developer workstation or Windows PC.
That does not make the launch irrelevant to desktop developers. It changes where the most demanding agent workloads are likely to run. A Windows developer using VS Code, Visual Studio, GitHub tooling or an internal enterprise assistant may encounter an LPX-backed service through an API, while the local machine remains responsible for the editor, terminal, browser, credentials and local test environment.
The key question will be whether cloud providers package that performance into an API tier enterprises can justify. Nebius Token Factory already markets standard and faster endpoint options for production inference, but its current public pricing material does not identify Groq 3 LPX capacity, a rollout schedule, or a separate rate card. NVIDIA also has not disclosed the price of an LPX rack, its power draw under a stated production workload, or the number of systems initially allocated to Nebius.
Those omissions matter more than the lack of a consumer product. A 3,431-token-per-second result may be technically impressive, but enterprise architecture decisions depend on predictable cost per generated token, sustained throughput at concurrency, service-level commitments, model availability, data residency and the ability to fail over when a specialized capacity pool is constrained.
NVIDIA is formalizing a heterogeneous inference stack
The larger significance is strategic. NVIDIA is taking a workload that has historically run almost entirely on GPUs and splitting it across purpose-built infrastructure: Vera CPUs for orchestration-heavy work, Rubin GPUs for context processing, LPX for low-latency decode, BlueField DPUs for infrastructure services and Spectrum-X networking between systems.
That is a more demanding architecture for cloud operators than simply adding more GPU nodes. It requires software that knows when to route requests, how to preserve KV cache state, how to move activations between GPU and LPU stages, and how to measure latency across the whole request rather than at one component. NVIDIA points to its Dynamo inference software as the orchestration layer for those combinations.
The company’s broader blog post also pairs LPX with Spectrum-X Multiplane networking, which NVIDIA says CoreWeave has deployed in production. That network announcement does not prove LPX performance, but it underscores the deployment reality: high-end agent serving increasingly becomes a full-rack and full-fabric problem. The chip is only as useful as the routing, model-serving stack and network around it.
There is also a notable roadmap context. At GTC in March, NVIDIA gave Groq 3 LPX a central role in Vera Rubin while Rubin CPX, a previously discussed context-phase accelerator, was absent from the major roadmap presentations. Tom’s Hardware reported at the time that LPX could be taking over a role once expected for CPX, though NVIDIA did not formally declare CPX canceled. The August production announcement strengthens the evidence that LPX is the inference-specialization product NVIDIA is actively pushing toward deployment.
What developers and administrators should watch next
The right response is neither to redesign an inference stack around one benchmark nor to dismiss the announcement as another AI chip claim. NVIDIA has now confirmed that Groq 3 LPX is in production, and both Nebius and Groq have confirmed planned deployments. That makes the platform real enough to track.
Teams building agentic applications should ask providers for evidence beyond raw output speed:
- They should request per-user and aggregate throughput under realistic concurrency, not only a single benchmark median.
- They should measure end-to-end task completion time, including retrieval, tool execution, test runs and retries.
- They should compare token pricing and rate limits against the latency improvement for the models they actually use.
- They should confirm where prompts, tool outputs and caches reside before moving sensitive code or enterprise data to a new inference tier.
NVIDIA has shown that its LPX design can generate Gemma 4 31B output unusually quickly with a 100,000-token prompt. The next milestone is more prosaic: a named Nebius API endpoint, a supported-model list, published economics and production measurements that let customers determine whether that speed survives outside NVIDIA’s own benchmark environment.
