Meituan's LongCat team has open-sourced version 1.5 of its digital avatar generation framework, LongCat-Video-Avatar, with a complete overhaul of audio extraction and video generation algorithms. The framework is built for industrial-grade spatiotemporal stability and lightning-fast inference, aiming to close the gap between research prototypes and real-world deployment.
The team swapped out the old Wav2Vec2 encoder for Whisper-large-v3, which brings major improvements to lip-sync accuracy and lip dynamics. Whisper-large-v3's acoustic representations also make cross-lingual and multilingual lip generation far more robust.
The model uses GRPO reinforcement learning alignment to reduce common artifacts like hand deformation and jarring frame drops, while improving identity consistency across longer videos. For extended clips, the framework employs multi-segment rolling inference, using previous video segments to build global temporal context and keep the character's identity coherent.
On the inference side, DMD2-based few-step distillation compresses the denoising iteration down to just 8 steps, striking a balance between efficiency and image fidelity.
In testing, the team used 508 image-audio pairs. A crowdsourced evaluation involved 770 raters who made 13,240 judgments, plus 10 experts who scored physical plausibility, temporal stability, visual harmony, and identity consistency. The official benchmarks show comparisons against HeyGen, Kling Avatar 2.0, and OmniHuman-1.5, with claimed gains in temporal stability, identity consistency, and natural lip synchronization. Beyond photorealistic portraits, the framework generalizes to anime and animal styles, and natively supports mono and multi-channel audio to drive multi-person conversations and hand-object interactions.
Model weights are released under the MIT license. The project page's ethics statement clarifies that generated content shown on the site is for academic use only and not for commercial purposes. Actual commercial use will require separate checks on weights, code, assets, and output boundaries.