Alibaba's Qwen team has open-sourced a new text-to-image (T2I) benchmark called Qwen-Image-Bench, designed to rigorously evaluate how well large models turn text descriptions into images. Alongside it, they released Q-Judger, a unified visual judge model trained on Qwen3.6-27B. The benchmark simulates a professional creative workflow, testing five major dimensions — image quality, aesthetics, text-image alignment, and two new categories: real-world fidelity and creative generation. That breaks down into 23 sub-capabilities and 56 granular metrics.
Qwen-Image-Bench contains 1,000 bilingual prompts (English and Chinese), split evenly between short and long descriptions, each averaging over four dimensions simultaneously. For fine-grained evaluation, Q-Judger was trained on over 130,000 bilingual expert-labeled pairs, with blind reviews and triple-checking overseen by 80 professional reviewers from art schools. The model outputs structured scores across all 56 metrics, matching human expert ratings with 92% accuracy.
In the first round, 18 mainstream image generation models were tested. GPT Image 2 took the top spot with a composite score of 64.69, also finishing first in every single dimension. Nano Banana 2.0 scored 59.82, GPT Image 1.5 scored 59.65, and Nano Banana Pro scored 59.45, rounding out the top four. Alibaba's own Qwen Image 2.0 Pro ranked fifth with 57.84, while GLM Image brought up the rear at 48.19. The results highlight that real-world fidelity and creative generation are the key differentiators separating the leaders from the pack. The evaluation also exposed shared industry pain points: even the best models struggle with rendering human hand bones, depicting physical laws like gravity and lighting, and handling object interpenetration — scoring below 44 on these specific metrics.