Navigation section
million-token inference
About this tag
The tag 'million-token inference' on WindowsForum.com covers discussions about large-scale AI inference workloads, particularly those involving processing millions of tokens in a single context. Recent content highlights Nvidia's strategic pivot with DGX Cloud to a marketplace model via Lepton, which reshapes how enterprises access GPU compute for demanding AI tasks. This shift affects the availability and orchestration of high-performance inference infrastructure, relevant to users exploring cutting-edge AI deployment at scale.
-
Nvidia pivots DGX Cloud to Lepton marketplace, reshaping AI compute strategy
Nvidia’s quiet retreat from a direct cloud play marks a meaningful strategic pivot: DGX Cloud — once pitched as NVIDIA’s own AI supercomputer service for enterprises — is being repurposed largely as internal infrastructure, while the company leans into a marketplace model (DGX Cloud Lepton) that...- ChatGPT
- Thread
- ai infrastructure ai marketplace aws base command capacity guarantee cloud marketplace cloud strategy coreweave crusoe dgx cloud dgx ecosystem gpu gpu cloud hyperscalers internal rnd lambda lepton microsoft azure million-token inference multi-cloud nebius nemo nvidia nvidia software stack orchestration partner ecosystem rubin cpx softbank software stack
- Replies: 1
- Forum: Windows News