Model watch · This week (published September 2)
From the Sep 3, 2026 daily brief
First the background: most large models today use a mixture of experts design. Rather than running the whole model for every token, it routes each token to a handful of specialist sub-models, which lets the model be enormous while only a slice of it computes each time. A common cost-saving move is to prune the least important experts, and the usual proxy for importance is how often an expert gets routed to. Alexey Fateev, an independent researcher, ran the deployed GLM-5.3 Flash checkpoint (320B total parameters, 18B active) on four RTX PRO 6000s and measured it directly rather than reading the config file. Two results: first, routing frequency barely tracks an expert's actual importance (Spearman correlation −0.222); second, dropping the least important 2% of experts still changed 64% of output sequences (dropping the most important 2% changed 79%) (@superalesha, 09-02). That is bad news for pruning as a cost-reduction path: the usual ruler measures the wrong thing, and even with the right ruler, the least important slice does not come out painlessly. ⚠️ We hold no record on this researcher's track record, this is a single-source measurement, and nobody has reproduced it.
Just an email address, unsubscribe anytime. This is the only thing we ask of you.
What follows is not a preprint. It is a set of readings from the appendix of the measureme…
The instrument here belongs to someone else — an evaluation called CoT-Control, which appe…
What follows is not an arXiv preprint but a research team's own write-up of its own paper …
The CAI team at Multiverse Computing, writing up its paper on Hugging Face, reports that a…