Sample Many Reasoning Paths, Then Vote on the Answer

Setting: chain-of-thought prompting asks a language model to write its reasoning step by step before answering. Normally the model decodes greedily, committing to its single most likely path. But a hard problem usually has several valid ways to reach the one right answer, while wrong paths tend to scatter across different wrong answers.

Wang et al. (ICLR 2023, "Self-Consistency Improves Chain of Thought Reasoning in Language Models") turn that into a decoding strategy. Sample a diverse set of reasoning paths instead of only the greedy one, then pick the answer that the most paths agree on, which marginalizes out the paths themselves.

What they report on top of chain-of-thought prompting, as absolute accuracy gains in percentage points with PaLM-540B or GPT-3: GSM8K (grade-school math) +17.9, SVAMP +11.0, AQuA +12.2, StrategyQA +6.4, and ARC-challenge +3.9. Their main results average 10 runs, sampling 40 outputs each.

Toy arithmetic (not their data): suppose each independently sampled path is right 60% of the time. A strict majority of 5 paths is then right 68.3% of the time, and of 21 paths 82.6%. The vote needs only the most common answer, not a strict majority, so for this rule those are lower bounds. Real paths are not independent, so treat this as the direction of the effect.

Their own stated limitation: it costs more computation, because you generate many paths instead of one.

The habit: when an answer comes from a single sampled chain of reasoning, ask whether five other chains would agree.

Source: Wang et al., ICLR 2023, read November 4, 2026.