Zyphra has released ZAYA1-8B-Diffusion-Preview, a mixture-of-experts (MoE) diffusion model converted from a standard autoregressive large language model. While the company markets it as the "first" to achieve this architectural transformation, the route was actually pioneered late last year by teams behind SDAR and LLaDA 2.0. ZAYA1's real claim to uniqueness is that it's the first diffusion language model trained entirely within AMD's hardware ecosystem.
Marketing aside, the model does validate the engineering efficiency gains of diffusion architecture. Traditional autoregressive models are constrained by token-by-token serial generation, where accumulating KV cache eventually hits a physical latency wall. As Kaiming He's team recently demonstrated with the pure diffusion model ELF, parallel denoising is the key to breaking that bottleneck. ZAYA1 adopts the TiDAR approach, skipping pretraining from scratch and simultaneously denoising 16 token candidates in a single forward pass — effectively turning a memory bandwidth bottleneck into a compute-bound problem.
Benchmarks show that with ZAYA1's custom CCA attention mechanism and a standard lossless sampler, it achieves a 4.6x receive-side speedup without degrading generation quality. Switching to a hybrid logit sampler pushes that acceleration to 7.7x, offering substantial cost reduction for large-scale, high-latency inference tasks.