Research & models Oct 26, 2026

Trend · preprint, October 23

Ke: BF16 fix keeps four Qwen RL runs stable; Qwen only

A preprint by Yijing Ke and two co-authors aligned three BF16 rounding points on the training side with the inference side, and reinforcement-learning runs on four Qwen models didn't collapse in the study's setup. In RL post-training, an inference engine generates samples and a training engine updates the weights. Both sides hold the same weights in BF16. Their kernels differ, though, so the probabilities the two sides compute still don't match. An update meant to be on-policy is in practice off-policy. The paper has the training side keep intermediate values in FP32 at RoPE, SwiGLU and the residual RMSNorm, converting to BF16 only at the output, as the inference side does (arXiv 2610.32951, 2026-10-23). The inference engine, the matrix-multiplication and attention kernels and the training objective are left unchanged. The authors write that in the runs studied, aligning RoPE alone was enough. The result is a collapse-or-no-collapse comparison with no percentages. Only the Qwen family was tested, and we have read only the abstract. Our read: for teams running their own RL post-training, a numerical mismatch between the training and inference sides is one of the first places to look when training collapses.