Buy thinking time or distill a small model? Our call: both, split by type of work — resolves 2027-07-14

Letting a big model think longer when it answers (test-time compute) buys better answers, while distilling a strong model’s skill into a small one is far cheaper, and the debate is which path wins.

It decides whether AI budgets go to expensive thinking time or cheap specialized small models, and whether frontier labs can keep charging a premium for thinking.

One side bets on thinking time and the other on distilled small models; SecondSource reads the two as splitting by type of work rather than excluding each other.

SecondSource’s reading: the two paths will coexist and split by workload. Open-ended, unfamiliar problems will pay for thinking time; narrow, high-volume tasks will be distilled into small models (cheap small models trained on a big model’s answers).

Thinking time wins on generalization, since thinking longer makes up for problems a model has not seen. Distillation wins on unit cost, since a known domain does not need fresh thinking every time, and the roughly 17-fold cost gap between them is real commercial pressure.

arXiv
“generating a chain of thought -- a series of…”

arXiv / Google DeepMind + UC Berkeley (Snell, Lee, Xu, Kumar)
“in a FLOPs-matched evaluation, we find that on…”

Reviewed July 2027: a hit if both forms keep expanding, with frontier labs continuing to push test-time compute pricing and commercial cases of narrow-domain distillation increasing.

What would overturn it

If distilled small models match frontier test-time compute on open-ended tasks, erasing the 17x cost gap, the premium on test-time compute collapses.