Simultaneous interpretation is evolving from plain speech translation into a full-modal digital interpreter that can see the scene and clone a speaker's voice. On May 19, Alibaba’s Tongyi Lab officially announced Qwen3.5-LiveTranslate, a new real-time audio/video simultaneous interpretation model that upgrades live translation to over 3,500 language pairs and introduces real-time voice cloning, custom hotwords, and visual understanding for the first time.
The new model is built on the Qwen3.5-Omni architecture and currently supports understanding and writing in 60 languages, with speech output in 29 languages.
Unlike traditional interpretation software that only processes audio, this model incorporates real-time visual context to resolve semantic ambiguities. For example, when a video shows a specific type of mask, the system can use visual features to accurately distinguish between a medical mask and a party mask in English, compensating for missing audio cues.
To eliminate transcription errors caused by noise and accents, the model also introduces a dynamic hotword injection mechanism. Users can specify particular names, brands, or industry terms directly in the translation stream, forcing the correct translation and preventing proper nouns from drifting during interpretation.
For cross-language interpretation, the model supports real-time voice cloning, reproducing the original speaker’s timbre and intonation during the interpretation stream.
Currently, the new model is available on the Qwen Omni experience platform, and the API will later be launched on Alibaba Cloud’s Bailian platform.