Research & models Nov 15, 2026
A conference paper from the Indian voice-AI company Gnani.AI reports that scores on the MATH math benchmark jumped after reinforcement learning, and the authors say about 95% of the gain came from the model more often putting its answer in the box the grader requires.

The gain came mainly from answer format, with accuracy on boxed answers barely changing, and the result comes from a single training seed on Gnani.AI's own model. The MATH grader extracts answers only from the answer box, and an unboxed answer counts as wrong. According to the paper, the MATH score rose from 48.69 to 84.76 and the share of boxed answers from 55.2% to 91.6%, while accuracy on boxed answers moved only from 91.1% to 92.6%. According to a second paper, training that rewarded only task outcomes cut the F1 score on XQuAD-Indic, a question-answering benchmark in Indian languages, from 36.51 to 17.19. F1 measures how closely answers overlap with the reference answers. The authors attribute the drop to the model switching to English, with the share of answers in English rising from 8.7% to 91.7%. Both papers come from the same group of authors, and no one else has repeated them (arXiv 2611.15717, 2026-11-11; arXiv 2611.16228, 2026-11-12). How we checked →
Source and checks
This item ran in the brief of Nov 15, 2026, alongside 9 other items. How we checked it, the quote checks and what was still missing that day are on that issue's How we checked page.
Cite this item
SecondSource (2026). "A conference paper from the Indian voice-AI company Gnani.AI reports that scores on the MATH math benchmark jumped after reinforcement learning, and the authors say about 95% of the gain came from the model more often putting its answer in the box the grader requires.." https://secondsource.io/en/research/2026-11-15-1/. Accessed access date.