A four-author preprint says that picking one model from a group of similarly accurate models can flip some individual predictions and still pass a black-box audit that checks only overall performance; the experiments are limited to tabular data and tree models.

Concept illustration: two matching panels hang side by side by wooden pegs from a line strung from a wall fixture at the left, each topped by a black band lettered PREPRINT and showing a person with dark shoulder-length hair and a lanyard badge holding up a clipboard behind an open black laptop whose screen shows a speech bubble. Below each laptop are two open trays; a red folder lettered HELOC sits in the left tray on the left panel and in the right tray on the right panel, and the other tray is empty. Printed on the image: 61.9% HELOC cases flippable, 5% tolerance, per authors.

In this paper's experiments, black-box audits that look only at overall performance can't stop a party set on picking a model, while white-box audits catch direct parameter edits, and the paper hasn't been peer reviewed. The paper calls this adversarial multiplicity. A group of models with similar accuracy can disagree on individual cases, so a motivated party can pick whichever model suits it best. Earlier work treated this multiplicity as a positive property that can help with fairness, and this paper turns it around. The authors tested on datasets including HELOC (about 10,000 credit applicants) and COMPAS (5,278 defendants). The paper describes two kinds of manipulation. The first edits the tree model's parameters directly: every individual decision in the validation set could be flipped while overall error changed by only 1%. The second is a "soft" pick that changes only hyperparameters and feature engineering. The tolerance is how much worse the picked model's overall performance may be than the original's. With a soft pick at a 5% tolerance, 61.9% of HELOC test individuals could be flipped; tightened to 1%, the share falls to 12.8%. A black-box audit measures only overall performance, so flipped individual predictions don't show up in that metric; a white-box audit catches the direct parameter edits. Large neural networks weren't tested (arXiv 2611.07378, 2026-11). How we checked →

Source and checks

This item ran in the brief of Nov 7, 2026, alongside 13 other items. How we checked it, the quote checks and what was still missing that day are on that issue's How we checked page.

Cite this item

SecondSource (2026). "A four-author preprint says that picking one model from a group of similarly accurate models can flip some individual predictions and still pass a black-box audit that checks only overall performance; the experiments are limited to tabular data and tree models.." https://secondsource.io/en/research/2026-11-07-3/. Accessed access date.