4 Experiments

本文为论文《Search-o1：让推理模型学会边思考边检索》「论文原文」板块的对应章节，按 arXiv 官方全文逐字保留，层级与「中文全文翻译」板块一一对应。

4.1 Tasks and Datasets

The evaluations used in this experiment include the following two categories:

Challenging reasoning tasks: (1) GPQA [52] is a PhD-level science multiple-choice QA dataset. The questions are authored by domain experts in physics, chemistry, and biology. In our main experiments, we use the highest quality diamond set containing 198 questions, and in Table 2, we use a more comprehensive extended set containing 546 questions to compare with the performance of human experts. (2) Math benchmarks include MATH500 [38], AMC2023 11 1

https://huggingface.co/datasets/AI-MO/aimo-validation-amc, and AIME2024 22 2

https://huggingface.co/datasets/AI-MO/aimo-validation-aime. MATH500 consists of 500 questions from the MATH test set [16]. AMC2023 and AIME2024 are middle school math competitions covering arithmetic, algebra, geometry, etc., containing 40 and 30 questions respectively. Among these three datasets, MATH500 and AMC are relatively simple, while AIME is more difficult. (3) LiveCodeBench [23] is a benchmark for evaluating LLMs’ coding capabilities, consisting of easy, medium, and hard difficulty problems. It collects recently published programming problems from competitive platforms to avoid data contamination. We utilize problems from August to November 2024, comprising 112 problems.

Open-domain QA tasks: (1) Single-hop QA datasets: Natural Questions (NQ) [29] contains questions from real Google search queries with answers from Wikipedia articles. TriviaQA [28] is a large-scale dataset with questions from trivia websites and competitions, featuring complex entity relationships. (2) Multi-hop QA datasets: HotpotQA [68] is the first large-scale dataset requiring reasoning across multiple Wikipedia paragraphs. 2WikiMultihopQA (2WIKI) [18] provides explicit reasoning paths for multi-hop questions. MuSiQue [55] features 2-4 hop questions built from five existing single-hop datasets. Bamboogle [47] collects complex questions that Google answers incorrectly to evaluate models’ compositional reasoning across various domains.

4.2 Baselines

We evaluate our approach against the following baseline methods:

Direct Reasoning: These methods utilize the model’s internal knowledge without retrieval. The open-source models include Qwen2.5-32B-Instruct [50], Qwen2.5-Coder-32B-Instruct [20], QwQ-32B-Preview [54], Qwen2.5-72B-Instruct [50], and Llama3.3-70B-Instruct [13]. Closed-source non-proprietary models include DeepSeek-R1-Lite-Preview [7], OpenAI GPT-4o [21], and o1-preview [22]. Results for open-source models are based on our implementations, while closed-source model results are sourced from their official releases.

Retrieval-augmented Reasoning: These methods retrieve external information to enhance the reasoning process. We consider two retrieval augmentation approaches: (1) Standard RAG: Retrieves the top-10 documents for the original question and inputs them alongside the question into the model for reasoning and answer generation. (2) RAG Agent (RAgent): Allows the model to decide when to generate queries for retrieval, as detailed in Section 3.3. To manage the length of retrieved documents, inspired by ReAct [70], we first retrieve the top-10 snippets during reasoning. The model then decides which URLs to obtain for the full documents when necessary.

4.3 Implementation Details

For the backbone large reasoning model in Search-o1, we utilize the open-sourced QwQ-32B-Preview [54]. For generation settings, we use a maximum of 32,768 tokens, temperature of 0.7, top_p of 0.8, top_k of 20, and a repetition penalty of 1.05 across all models. For retrieval, we employ the Bing Web Search API, setting the region to US-EN and the top-$k$ retrieved documents to 10. We use Jina Reader API to fetch the content of web pages for given URLs. For all retrieval-based methods, following [52], we apply a back-off strategy where, when a final answer is not provided, we use the result from direct reasoning. For baseline models not specifically trained for o1-like reasoning, we apply Chain-of-Thought (CoT) [59] prompting to perform reasoning before generating answers. Detailed instructions for all models are provided in Appendix A. All experiments are conducted on eight NVIDIA A800-80GB GPUs.

Table 1: Main results on challenging reasoning tasks, including PhD-level science QA, math, and code benchmarks. We report Pass@1 metric for all tasks. For models with 32B parameters, the best results are in bold and the second-best are underlined. Results from larger or non-proprietary models are in gray color for reference. Symbol “†” indicates results from their official releases.

4.4 Results on Challenging Reasoning Tasks

Main Results.

Table 1 presents Search-o1’s performance on complex reasoning tasks, with the main results outlined below:

For both settings without retrieval and with retrieval augmentation, the large reasoning model QwQ-32B-Preview consistently shows superior performance compared to traditional instruction-tuned LLMs. The QwQ model with 32B parameters even outperforms larger LLMs such as Qwen2.5-72B and Llama3.3-70B in the direct reasoning setting, demonstrating the effectiveness of the o1-like long CoT approach in complex reasoning.

RAgent-QwQ-32B surpasses both standard RAG-based models and direct reasoning QwQ-32B in most tasks, thanks to its agentic search mechanism, which autonomously retrieves information to supplement the knowledge required for reasoning at each step. Additionally, we find that the non-reasoning model Qwen2.5-32B using agentic RAG performs similarly to standard RAG on GPQA and even shows decreased performance on math and code tasks. This indicates that ordinary LLMs cannot effectively utilize search as a tool to solve complex reasoning tasks.

Our Search-o1 further outperforms RAgent-QwQ-32B in most tasks, demonstrating the effectiveness of our Reason-in-Documents strategy by integrating external knowledge while ensuring that it does not affect the coherence of the original reasoning. Specifically, on average across all five datasets, Search-o1 exceeds RAgent-QwQ-32B and QwQ-32B by 4.7% and 3.1%, respectively, and significantly outperforms non-reasoning models Qwen2.5-32B and Llama3.3-70B by 44.7% and 39.3%.

Scaling Analysis on Number of Retrieved Documents.

Table 2: Performance comparison with human experts on the GPQA extended set [52].

In this experiment, we analyze the performance variation with respect to the number of retrieved documents, as shown in Figure 3. Our results demonstrate that Search-o1 can effectively leverage an increasing number of retrieved documents, leading to improvements in handling complex reasoning tasks. We also observe that for overall performance, retrieving even one document can surpass Direct Reasoning and standard RAG models that use ten retrieved documents, showcasing the effectiveness of the agentic search and Reason-in-Documents strategies.

Figure 3: Scaling analysis of top-k retrieved documents utilized in reasoning. All results are based on QwQ-32B-Preview model.

Comparison with Human Experts.

We compare the performance of our Search-o1 with human experts across various domains in the GPQA extended set. Table 2 presents the evaluation of human experts from various disciplines, including physics, chemistry, and biology. Our Search-o1 model outperforms human experts in overall performance (57.9), as well as in both physics (68.7) and biology (69.5), demonstrating superior handling of complex reasoning tasks. While Search-o1 slightly trails chemists in the chemistry subdomain (40.7 vs. 72.6), it still provides a competitive edge overall, particularly in terms of general performance across multiple domains. This highlights the effectiveness of Search-o1 in leveraging document retrieval and reasoning to achieve cross-domain performance that rivals or exceeds expert-level capabilities.

4.5 Results on Open-Domain QA Tasks

In addition to the reasoning tasks where LRMs excel, we also explore the performance of our Search-o1 on open-domain QA tasks. Table 3 presents the overall results. The key observations are:

For direct reasoning without retrieval, the performance of the LRM QwQ-32B is overall similar to the non-reasoning LLM Qwen2.5-32B, with a slight decrease in average EM across all QA datasets (31.3 vs. 30.7). This indicates that LRMs do not perform as strongly on open-domain QA tasks as they do on reasoning tasks.

When employing retrieval-augmented reasoning, retrieval significantly improves performance for both reasoning and non-reasoning models across all tasks, suggesting that models have knowledge gaps in these tasks. Additionally, for the QwQ-32B model, agentic RAG achieves an average EM improvement of 23.2% over standard RAG on multi-hop QA tasks, demonstrating the effectiveness of our agentic RAG strategy in knowledge-based multi-hop QA. However, we also observe that there is no significant performance change for single-hop tasks (47.8 vs. 47.6 on average EM), as these questions only require information from a single knowledge point without the need for multiple retrievals. This also verifies that the agentic search mechanism can better unleash the potential of LRMs in more complex and challenging reasoning tasks.

For our Search-o1, we find that it generally outperforms all baselines on multi-hop tasks. Specifically, in terms of the average EM metric, our Search-o1 exceeds RAG-QwQ-32B and RAgent-QwQ-32B by 29.6% and 5.3%, respectively, demonstrating the effectiveness of our Reason-in-Documents strategy in complex QA tasks. This further emphasizes the importance of maintaining consistency between external knowledge and the logical chain of reasoning.

Table 3: Performance comparison on open-domain QA tasks, including single-hop QA and multi-hop QA datasets. For models with 32B parameters, the best results are in bold and the second-best are underlined. Results from larger models are in gray color for reference.