A team from the National University of Singapore, Nanyang Technological University, and the Shanghai Artificial Intelligence Laboratory has released Mega-ASR, the first open-source robust speech recognition foundation model designed to tackle hallucinations, missing words, and blank outputs in real-world conditions. The model is powered by Qwen3-ASR 1.7B and delivers up to 30% better performance than Whisper, Gemini 3 Pro, and Seed-ASR in extremely challenging acoustic environments. The entire project, including code and model weights, is now available on GitHub under the Apache-2.0 license.
To train Mega-ASR, the researchers built a dataset called Voices-in-the-wild-2M, containing 2.4 million samples totaling 11,000 hours of audio. Using a simulation pipeline based on spectral physics, they synthesized seven atomic acoustic effects — reverberation, echo, additive noise, far-field, packet loss, bandwidth limitation, and clipping distortion — and combined them into 54 composite environmental scenarios. To ensure stable training, they filtered out samples with a word error rate above 70% and then calibrated the dataset's difficulty distribution with a physical plausibility check.
On the training front, Mega-ASR introduces Acoustic-to-Semantic Stepwise Supervised Fine-Tuning (A2S-SFT), a method that aligns audio features in stages to improve the model's ability to recover semantics under heavy interference. In the policy optimization phase, the model uses Dual-Granularity WER Gated Policy Optimization (DG-WGPO), a reinforcement learning approach. When input audio quality is good and word error rate is low, the system focuses on character-level acoustic detail reconstruction. When audio is severely degraded and word error rate is high, the mechanism shifts to sentence-level semantic reconstruction, significantly reducing the hallucinations and omissions common in large models.
To address a slight dip in recognition accuracy on clean audio, Mega-ASR includes a dynamic routing mechanism. A router automatically assesses audio quality and decides whether to activate LoRA fine-tuning weights, ensuring optimal performance in both clean and noisy environments.