GPT-6 Astra Trained on 100,000 Blackwell GPUs. China's AI Compute Gap Is Showing
Jensen Huang dropped a number this morning: GPT-6 Astra was trained on more than 100,000 Grace Blackwell GPUs, wired into a high-speed cluster with NVLink72. He also said another batch of 400,000 GPUs is coming online — but didn't specify which chips or for whom.
Using Astra as a yardstick, China's leading model labs still have a visible compute gap. ByteDance is closest, with about 36,000 B200s plugged in this year via Malaysia. Those are Blackwell-generation chips, the same family as Astra's GB200s, but the compute is deployed overseas. When ByteDance publicly describes its in-country clusters, it's mostly still the China-specific Hopper versions, the H20 and H800.
Kimi was recently reported to have gotten about 20,000 Hopper GPUs through Alibaba. Bloomberg sources say they're H200s, but Alibaba denies that specific model. H200 belongs to the previous Hopper generation; B200 has already moved to Blackwell. GB200 goes a step further, pairing a Grace CPU with a Blackwell GPU, and NVL72 can put 72 of those GPUs into a single high-speed interconnect domain.
DeepSeek hasn't revealed V4's full training hardware. A leaked transcript from Liang Wenfeng's investor meeting says the company had roughly 20,000 H100-equivalent GPUs in May, most of which had just arrived, and that future procurement will be "basically all Nvidia." Huawei is reportedly providing DeepSeek with about 16,000 of its 950 chips. Liang estimates that domestic batch equals roughly 4,000 Nvidia B-series cards — enough to train the current model generation, and not much more.
But Blackwell isn't Nvidia's newest generation anymore. Vera Rubin is already in full production, and Nvidia estimates Rubin can cut the GPU count needed to train a large MoE model to a quarter of what Blackwell requires.
No Chinese company has publicly disclosed training a single model on 100,000 same-generation advanced GPUs. The visible gap between Chinese and American AI labs is no longer just about how many cards you own. It's about which generation you can get — and how many of those you can throw at one model at once.