SecondSourceJudgment rebuilt from primary sources

A conference paper from the Indian voice-AI company Gnani.AI reports that scores on the MATH math benchmark jumped after reinforcement learning, and the authors say about 95% of the gain came from the model more often putting its answer in the box the grader requires.

Concept illustration in two side-by-side panels: in each, a black laptop whose screen shows a blue-and-pink mark and the lettering GNANI.AI sits on a shelf, and below the shelf a beige box lettered MATH holds a stack of papers with grey lines, the front sheet having one line enclosed in a red rectangle, next to a brown wastebasket with a grey-lined sheet that has no red rectangle. In the right panel a book lettered TRAINING stands beside the laptop, the paper stack in the MATH box is taller, and the wastebasket holds only that sheet and a few scraps, while the left panel's wastebasket overflows with crumpled paper. Printed on the image: About 95% Share of MATH score gain after reinforcement learning from boxing answers more often, Gnani.AI says.

The gain came mainly from answer format, with accuracy on boxed answers barely changing, and the result comes from a single training seed on Gnani.AI's own model. The MATH grader extracts answers only from the answer box, and an unboxed answer counts as wrong. According to the paper, the MATH score rose from 48.69 to 84.76 and the share of boxed answers from 55.2% to 91.6%, while accuracy on boxed answers moved only from 91.1% to 92.6%. According to a second paper, training that rewarded only task outcomes cut the F1 score on XQuAD-Indic, a question-answering benchmark in Indian languages, from 36.51 to 17.19. F1 measures how closely answers overlap with the reference answers. The authors attribute the drop to the model switching to English, with the share of answers in English rising from 8.7% to 91.7%. Both papers come from the same group of authors, and no one else has repeated them (arXiv 2611.15717, 2026-11-11; arXiv 2611.16228, 2026-11-12). How we checked →

Source and checks

This item ran in the brief of Nov 15, 2026, alongside 9 other items. How we checked it, the quote checks and what was still missing that day are on that issue's How we checked page.

Cite this item

SecondSource (2026). "A conference paper from the Indian voice-AI company Gnani.AI reports that scores on the MATH math benchmark jumped after reinforcement learning, and the authors say about 95% of the gain came from the model more often putting its answer in the box the grader requires.." https://secondsource.io/en/research/2026-11-15-1/. Accessed access date.