Alibaba open-weights a Qwen4 preview model

Alibaba has published weights for Qwen3.8-Flash-Next, billed as a preview of its coming Qwen4 architecture, with NVIDIA detailing Day 0 support in a developer blog post.

The model is a multimodal mixture-of-experts system: a 125B-parameter main model plus 51B in N-gram embeddings, with 6B parameters active per token. Native context is 262,144 tokens, extensible to 1M with YaRN.

The architectural change is a hybrid attention scheme. Three of every four layers use Gated DeltaNet, compressing history into a fixed-size recurrent state so KV cache does not grow with sequence length; the fourth uses Qwen Sparse Attention, which aggregates the sequence into micro-blocks and scores importance at the block level rather than per token, cutting indexing overhead that rises with context length.

Alibaba's published figures put the QSA attention kernel at up to 7.6x faster than full attention during prefill and 4.9x during decoding. In a serving test at 1M-token context with a 90% prefix-cache hit rate, the model reached 8.6x the prefill throughput of Qwen3.7-Plus.

NVIDIA says it validated inference on GB300 NVL72 — 72 Blackwell Ultra GPUs on a 130 TB/s NVLink domain — reporting over 16K tokens per second per GPU and over 200 tokens per second per user. It lists functional support via SGLang, vLLM and TensorRT LLM, plus NeMo AutoModel and NeMo RL post-training recipes, and says the model also runs on DGX Station, DGX Spark clusters and four-GPU RTX PRO 6000 workstations.

Weights are on Hugging Face and ModelScope. The post gives no accuracy benchmarks, licensing terms, or Qwen4 timeline.