DeepSeek V4 specs leak: 1.6 trillion parameters, 384 experts, and a 1M token context

Princeton PhD student Yifan Zhang just dropped the full technical specs for DeepSeek's upcoming V4 model on X. He'd teased "V4 next week" back on April 19th with a few architecture component names, but tonight he posted the complete parameter table and revealed a new 285-billion-parameter lightweight version called V4-Lite.

V4 is a 1.6-trillion-parameter beast. Its attention mechanism is DSA2, which combines DeepSeek's earlier DSA (DeepSeek Sparse Attention) from V3.2 with the NSA (Native Sparse Attention) scheme proposed in a paper earlier this year. It uses a head dimension of 512, paired with Sparse MQA and SWA (sliding window attention). The MoE (Mixture of Experts) layer packs 384 experts, activating just 6 at a time via a Fused MoE Mega-Kernel. Residual connections stick with Hyper-Connections.

On the training side, newly disclosed details include using the Muon optimizer—a matrix-level optimizer that applies Newton-Schulz orthogonalization to momentum updates. Pre-training context length was 32K tokens, and the reinforcement learning phase used GRPO with KL divergence correction. The final context length was extended to a whopping 1 million tokens. It's a text-only model.

Zhang isn't employed by DeepSeek, and the company hasn't officially commented on any of this information. The leak comes amid a flurry of activity in the large language model space, including Alibaba's new Qwen3.6 model that also uses a sparse MoE architecture to manage compute costs.