论文信息与摘要（原文）

原文标题：s1: Simple test-time scaling arXiv：2501.19393（v1）　方向：第三批 推理进化：测试时计算与反思

以下为 arXiv 官方 HTML 全文，一字不落保留原文结构、图表与参考文献，仅供学习研究参考，版权归原作者所有。

Niklas Muennighoff

Affiliation: Stanford University

Affiliation: Allen Institute for AI

Affiliation: Contextual AI

Zitong Yang

Affiliation: Stanford University

Weijia Shi

Affiliation: University of Washington, Seattle

Xiang Lisa Li

Affiliation: Stanford University

Li Fei-Fei

Affiliation: Stanford University

Hannaneh Hajishirzi

Affiliation: University of Washington, Seattle

Affiliation: Allen Institute for AI

Luke Zettlemoyer

Affiliation: University of Washington, Seattle

Percy Liang

Affiliation: Stanford University

Emmanuel Candès

Affiliation: Stanford University

Tatsunori Hashimoto

Affiliation: Stanford University

Abstract

Test-time scaling is a promising new approach to language modeling that uses extra test-time compute to improve performance. Recently, OpenAI’s o1 model showed this capability but did not publicly share its methodology, leading to many replication efforts. We seek the simplest approach to achieve test-time scaling and strong reasoning performance. First, we curate a small dataset s1K of 1,000 questions paired with reasoning traces relying on three criteria we validate through ablations: difficulty, diversity, and quality. Second, we develop budget forcing to control test-time compute by forcefully terminating the model’s thinking process or lengthening it by appending “Wait” multiple times to the model’s generation when it tries to end. This can lead the model to double-check its answer, often fixing incorrect reasoning steps. After supervised finetuning the Qwen2.5-32B-Instruct language model on s1K and equipping it with budget forcing, our model s1-32B exceeds o1-preview on competition math questions by up to 27% (MATH and AIME24). Further, scaling s1-32B with budget forcing allows extrapolating beyond its performance without test-time intervention: from 50% to 57% on AIME24. Our model, data, and code are open-source at https://github.com/simplescaling/s1.

Keywords: Machine Learning, ICML, Large language models, Test-time scaling, Test-time compute

††affiliationnotice: Equal contribution