Zhipu AI, in collaboration with Yuxun Networks and Tsinghua University, has taken aim at one of the most annoying bottlenecks in large-scale AI inference: network congestion. Their solution is ZCube, a new networking architecture designed specifically for the increasingly common practice of PD (Prefill-Decode) separation.
When long-context models and PD separation become the norm, KV Cache transfers between nodes create wildly asymmetric traffic patterns. Traditional network designs like ROFT (Rail-Optimized Fat-Tree) struggle, leading to hot spots and link contention. ZCube ditches the Spine switch layer entirely, flattening the topology to a 2-hop network diameter. It also introduces a hybrid single/multi-rail access scheme to balance load across all switches.
The results from a production cluster running GLM-5.1 on a thousand GPUs are hard to ignore. Keeping the same GPUs, software stack, and applications, the ZCube setup cut switching and optical module costs by 33%, boosted average GPU throughput by 15%, and slashed the P99 time-to-first-token (TTFT) by 40.6%. Zhipu AI, the company behind the GLM series, has been expanding beyond infrastructure — it recently launched AutoClaw on iOS with dual-mode PC control.