Research & models Oct 16, 2026

This week · preprint October 13

RBC lifts playbook success to 66.9; it's one preprint

An arXiv paper uses randomized on/off trials to screen the lesson playbooks agents write: screened playbooks average a success rate of 66.9, against 54.1 with no playbook, 55.5 using the whole playbook and 53.8 letting the agent choose. The method, RBC, randomly switches each lesson in the playbook on or off and estimates each lesson's effect from the difference in success rates. The abstract says: "the same procedure can write a playbook that helps or one that falls up to 69.6 points below no playbook" (arXiv 2610.18427). The abstract doesn't name the models or benchmarks tested, or give the unit and baseline of the success rate, and the full paper hasn't been reviewed. Three other preprints the same day also change only prompts, memory or workflow rather than weights, and use held-out data or independent checks to decide whether to adopt a change (arXiv 2610.18216; arXiv 2610.18813; arXiv 2610.18845). Our reading: in these papers, agent "self-improvement" means an outer recipe plus statistical gatekeeping, not changes to model weights. Four papers on the same day may be coincidence, so this stays an observation still being verified. How we checked →