NVIDIA’s diverse product portfolio addresses both AI training and inference requirements, in response to the scaling up of CSPs’ in-house ASIC development

According to the latest AI Server research from TrendForce, as major cloud service providers (CSPs) step up their efforts to develop in-house chips, NVIDIA has shifted its focus at GTC 2026 to the implementation of AI inference applications across various sectors, marking a departure from its previous concentration on the cloud AI training market. By promoting a diverse product portfolio—including GPUs, CPUs and LPUs—to address both AI training and AI inference requirements, the company aims to drive supply chain growth through rack integration solutions.

TrendForce notes that as the trend towards in-house chip development, led by CSPs such as Google and Amazon, expands, the proportion of ASIC AI servers in total AI server shipments is projected to rise from 27.8% in 2026 to nearly 40% by 2030.

To consolidate its leadership in the AI market, one of NVIDIA’s strategies has been to actively promote rack-scale solutions such as the GB300 and VR200, which integrate CPUs and GPUs, emphasising their scalability for AI inference applications. Vera Rubin, unveiled at GTC, is defined as a highly vertically integrated complete system, comprising seven chips and five racks.

Looking at the progress of the Rubin supply chain, it is expected that memory manufacturers will be able to supply HBM4 for use in Rubin GPUs by the second quarter of 2026, enabling NVIDIA to begin shipping Rubin chips around the third quarter. As for the shipment schedules of NVIDIA’s GB300 and VR200 Rack systems, the former has already replaced the GB200 as the flagship model in Q4 2025, with its shipment share projected to reach nearly 80% by 2026. Meanwhile, the VR200 Rack is expected to gradually ramp up shipment capacity by the end of Q3 2026, though further developments will depend on the actual progress of ODMs.

Furthermore, as AI transitions from generative to agent-based models, the decoding stage of token generation faces significant latency and memory bandwidth bottlenecks. To address this, NVIDIA has integrated technology from the Groq team to launch the Groq 3 LPU, specifically designed for low-latency inference. Each chip features 500MB of SRAM, with a total capacity of up to 128GB per rack.

However, the memory capacity of the LPU itself is insufficient to accommodate the massive parameters and KV Cache of a model such as Vera Rubin. Consequently, at this year’s GTC, NVIDIA introduced the ‘Disaggregated Inference’ architecture. Through an AI factory operating system called Dynamo, the inference pipeline is split into two parts: when processing proxy-based AI, the pre-fill and attention stages—which require extensive mathematical operations and the storage of massive KV caches—are handled by Vera Rubin, which boasts extremely high throughput and massive memory. Conversely, the decoding and token generation stages—which are bandwidth-constrained and highly latency-sensitive—are offloaded directly to LPU racks equipped with massive memory.

Regarding supply chain progress, the third-generation Groq LP30, manufactured by Samsung, has entered full-scale mass production and is expected to begin shipping in the second half of 2026. Furthermore, plans are in place to launch the higher-performance LP40 chip in the next-generation Feynman architecture.