Huawei and USTC break NVIDIA’s grip with 58% faster AI training on Ascend

In the race to build large Mixture-of-Experts (MoE) models on homegrown hardware, a joint team from the University of Science and Technology of China (USTC), Huawei, and Peking University has unveiled a new compilation and scheduling framework called HyperParallel-MoE. The framework is designed specifically for Huawei's Ascend A3 chips, which feature two types of cores: AIC for matrix multiplication and AIV for vector compute and communication. The problem is that traditional serial operator scheduling leaves one core type idle while the other works, leading to poor utilization. In a 256-node cluster running a 671B-parameter DeepSeek-style model, AIC utilization was only 67%, and 39% of expert routing communication latency sat on the critical path.

HyperParallel-MoE tackles this with three key changes. First, it introduces an AIV-driven unilateral write primitive that triggers computation as soon as a tile of data arrives, without waiting for the whole batch. Second, it uses dependency-aware tile task generation to unify communication and compute operators into a single abstraction. Third, a static scheduler pre-generates task sequences, allowing both core types to run in parallel within a single kernel, sharing intermediate results through the fast L2 cache instead of writing back to slow HBM memory.

In tests across 64 nodes with balanced routing, the MoE-FFN (the core expert compute module) saw a 36% reduction in latency, translating to a 58% boost in data processing speed (a 1.49x to 1.58x improvement). End-to-end single training step speed also improved by 8% to 9%. The results make clear that Ascend's real efficiency isn't just about hardware specs — it's about how well the compiler and runtime can keep those AIC and AIV cores busy.