Chipmaker Cerebras announced it has successfully run the trillion-parameter open-source model Kimi K2.6 in an enterprise test, achieving a generation speed of 981 tokens per second — 6.7 times faster than mainstream GPU cloud services.
By distributing the model's weights across multiple 12-inch silicon wafers and streaming activations between them, Cerebras eliminated the interconnect latency that plagues traditional board-level communication. Independent benchmarking firm Artificial Analysis measured the speed at 981 tokens/s, and on a long-context task with 10,000 input tokens and 500 output tokens, total response time dropped from 163.7 seconds on Kimi's official API to just 5.6 seconds — a 29x improvement.
Because a single wafer can't hold all the parameters of a trillion-parameter model, the system splits weights across multiple wafers and streams activations. Within a wafer, all layer-to-layer communication happens on an on-chip network fabric, offering over 200 times the physical bandwidth of NVLink in Nvidia's NVL72 architecture. Combined with distributed computing optimizations, the model runs with original 4-bit weights for low-loss storage, uses 16-bit floating point during computation to maintain precision, and employs custom operator kernels and speculative decoding to achieve real-time performance.