OpenAI engineer Clive Chan says DeepSeek's V4 technical report is world-class overall, but its hardware recommendations for chip makers are "surprisingly mediocre and even wrong" — a stark departure from V3's acclaimed section. V3's hardware chapter had a Q&A that became the hottest topic at the ISCA conference, with recommendations specific enough to reference industry interconnects still being standardized. V4's advice, by contrast, is much vaguer.
https://twitter.com/itsclivetime/status/2047549077341270167
Chan laid out his criticisms point by point. On power consumption: the report claims software optimizations can keep a chip's compute, memory, and communication fully loaded simultaneously, and advises chip makers to leave more power headroom. Chan argues the opposite — chip power is limited by physics, and leaving headroom means lowering clock speeds, reducing performance. On GPU data transfer: the report prefers a pull model (GPU actively reads data) over push (sender pushes data), arguing push has too much notification overhead. Chan counters that pull is actually slower, and the fix should be better network interface processing. (Though the two might be talking past each other — the report's concern is notification overhead, Chan's is transfer latency.) On activation functions: the report suggests replacing SwiGLU with a simpler function to reduce compute. Chan says that's unnecessary, pointing to Sonic MoE's success with SwiGLU at optimal performance. He suspects DeepSeek may have "intentionally weakened this chapter."
Specs for V4 leaked earlier this week, detailing a 1.6 trillion parameter model with 384 experts.