MiniMax, the Chinese AI company behind the M2 series of large language models, recently discovered a deeply weird bug: its model could answer detailed questions about Chinese pop star Ma Jiaqi, but it absolutely couldn't say his name. The investigation started with that single word and ballooned into a full vocabulary scan that revealed nearly 5% of all tokens had quietly degraded beyond use.
The root cause? It's a story about how modern LLMs are built — and how fragile that process can be. MiniMax's tokenizer, the component that chops text into atomic units the model understands, merged the two characters "Jiaqi" into a single token. During pretraining, the model saw plenty of internet text and learned that token just fine. But in the post-training phase — when the model is fine-tuned on conversational data — fewer than 5 samples in the entire dataset contained that token. Meanwhile, high-frequency tokens like tool_call markers and code symbols kept updating the surrounding vector space, steadily pushing "Jiaqi" into a dead zone. The model still knows who Ma Jiaqi is; it just lost the ability to output his name.
The findings come amid a flurry of AI model releases, including Zhipu AI's GLM-5V-Turbo, which recently topped benchmarks for code generation.
Armed with that insight, MiniMax's team ran a full scan of roughly 200,000 tokens in the model's vocabulary. They found that 4.9% of all tokens had significantly degraded. The hardest-hit language was Japanese: 29.7% of Japanese tokens were degraded, far worse than Korean (3.3%), Russian (3.7%), Chinese (3.9%), or English (3.5%). Also high on the list were classic SEO spam phrases like "Legendary private server" and "Painless abortion" — the same mechanism at work.
The Japanese degradation solved an old mystery. The model had occasionally mixed Russian or Korean characters into Japanese responses, and nobody could figure out why. The analysis showed that Japanese token parameters had drifted and become confused with tokens from other languages in the vector space. This caused Japanese tokens to fire incorrectly (language mixing) and also squeezed neighboring low-frequency Chinese tokens out of the normal probability range (token forgetting).
The fix was surprisingly simple: construct a synthetic dataset that covers the full vocabulary, then run the model through a basic repetition exercise on every token. The results were dramatic. The rate of Russian characters appearing in Japanese responses dropped from 47% to 1%. Cosine similarity — a measure of output parameter stability — went from a low of 0.329 to above 0.97 across the entire vocabulary.