Alibaba launches Qwen4-Max with training, attention optimizations
Alibaba Group Holding Ltd. today introduced Qwen4-Max, its newest and most capable large language model. The algorithm not only generates higher-quality output than its predecessor but also costs less. Those improvements are the result of a new architecture, Qwen4, that made the LLM’s training run more efficient. Additionally, Alibaba has equipped the model with an […] The post Alibaba launches…
Alibaba unveiled Qwen4-Max, its most advanced large language model, boasting higher-quality outputs and lower costs compared to its predecessor. The model's enhanced performance stems from a new architecture, Qwen4, which streamlines training. Alibaba has also upgraded the model's attention mechanism, incorporating two key customizations: Gated DeltaNet and QSA.
These innovations lower hardware requirements and reduce unnecessary data processing, further cutting infrastructure costs. Qwen4-Max's backbone consists of 2.4 trillion parameters, with only 71 billion actively engaged at any given time. It is complemented by 328 billion N-gram parameters that store results of repetitive calculations, eliminating the need for recalculation.
The model's layers process data sequentially, with gradients refining outputs from the final layer. A new residual feature in Qwen4-Max allows gradients to bypass certain layers, reducing training errors and costs. Alibaba's benchmark tests show Qwen4-Max outperforming Qwen3.8-Max by 35% on knowledge work tasks and 49% on coding challenges.
The model is accessible via Alibaba's cloud platform and will eventually be open-sourced alongside smaller models and the Qwen-Live-Delivery-Plugin, which allows for faster iterative improvements on long-horizon tasks.
Written by urgent.news from SiliconANGLE's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.