References

本文为论文《Qwen3：兼顾思考与非思考模式的开源模型家族》「论文原文」板块的对应章节，按 arXiv 官方全文逐字保留，层级与「中文全文翻译」板块一一对应。

• Abdin et al. (2024)

Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al.

Phi-4 technical report.

arXiv preprint arXiv:2412.08905, 2024.

• AIME (2025)

AIME.

AIME problems and solutions, 2025.

URL https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions.

• Ainslie et al. (2023)

Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai.

GQA: Training generalized multi-query Transformer models from multi-head checkpoints.

In EMNLP, pp. 4895–4901. Association for Computational Linguistics, 2023.

• An et al. (2024)

Chenxin An, Fei Huang, Jun Zhang, Shansan Gong, Xipeng Qiu, Chang Zhou, and Lingpeng Kong.

Training-free long-context scaling of large language models.

CoRR, abs/2402.17463, 2024.

• Anthropic (2025)

Anthropic.

Claude 3.7 Sonnet, 2025.

URL https://www.anthropic.com/news/claude-3-7-sonnet.

• Austin et al. (2021)

Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton.

Program synthesis with large language models.

CoRR, abs/2108.07732, 2021.

• Bai et al. (2023)

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu.

Qwen technical report.

CoRR, abs/2309.16609, 2023.

• Bai et al. (2025)

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al.

Qwen2.5-VL technical report.

arXiv preprint arXiv:2502.13923, 2025.

• Bandarkar et al. (2023)

Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa.

The Belebele benchmark: A parallel reading comprehension dataset in 122 language variants.

CoRR, abs/2308.16884, 2023.

• Brown et al. (2020)

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei.

Language models are few-shot learners.

In NeurIPS, 2020.

• Cassano et al. (2023)

Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps-Costin, Donald Pinckney, Ming-Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q. Feldman, Arjun Guha, Michael Greenberg, and Abhinav Jangda.

MultiPL-E: A scalable and polyglot approach to benchmarking neural code generation.

IEEE Trans. Software Eng., 49(7):3675–3691, 2023.

• Chen et al. (2021)

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba.

Evaluating large language models trained on code.

CoRR, abs/2107.03374, 2021.

• Cobbe et al. (2021)

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman.

Training verifiers to solve math word problems.

CoRR, abs/2110.14168, 2021.

• Dai et al. (2024)

Damai Dai, Chengqi Deng, Chenggang Zhao, R. X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y. K. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang.

DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models.

CoRR, abs/2401.06066, 2024.

• Dauphin et al. (2017)

Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier.

Language modeling with gated convolutional networks.

In ICML, volume 70 of Proceedings of Machine Learning Research, pp. 933–941. PMLR, 2017.

• DeepMind (2025)

Google DeepMind.

Gemini 2.5, 2025.

URL https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/.

• Dehghani et al. (2023)

Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Arnab, Xiao Wang, Carlos Riquelme Ruiz, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin Fathy Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Collier, Alexey A. Gritsenko, Vighnesh Birodkar, Cristina Nader Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetic, Dustin Tran, Thomas Kipf, Mario Lucic, Xiaohua Zhai, Daniel Keysers, Jeremiah J. Harmsen, and Neil Houlsby.

Scaling vision transformers to 22 billion parameters.

In ICML, volume 202 of Proceedings of Machine Learning Research, pp. 7480–7512. PMLR, 2023.

• Du et al. (2025)

Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, King Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, et al.

SuperGPQA: Scaling LLM evaluation across 285 graduate disciplines.

arXiv preprint arXiv:2502.14739, 2025.

• Dubey et al. (2024)

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Nail, Grégoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al.

The Llama 3 herd of models.

CoRR, abs/2407.21783, 2024.

• Fan et al. (2023)

Simin Fan, Matteo Pagliardini, and Martin Jaggi.

DoGE: Domain reweighting with generalization estimation.

arXiv preprint arXiv:2310.15393, 2023.

• Gema et al. (2024)

Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, et al.

Are we done with MMLU?

CoRR, abs/2406.04127, 2024.

• Gu et al. (2024)

Alex Gu, Baptiste Rozière, Hugh Leather, Armando Solar-Lezama, Gabriel Synnaeve, and Sida I. Wang.

CRUXEval: A benchmark for code reasoning, understanding and execution.

arXiv preprint arXiv:2401.03065, 2024.

• Guo et al. (2025)

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al.

DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning.

arXiv preprint arXiv:2501.12948, 2025.

• He et al. (2024)

Yun He, Di Jin, Chaoqi Wang, Chloe Bi, Karishma Mandyam, Hejia Zhang, Chen Zhu, Ning Li, Tengyu Xu, Hongjiang Lv, et al.

Multi-IF: Benchmarking LLMs on multi-turn and multilingual instructions following.

arXiv preprint arXiv:2410.15553, 2024.

• Hendrycks et al. (2021a)

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.

Measuring massive multitask language understanding.

In ICLR. OpenReview.net, 2021a.

• Hendrycks et al. (2021b)

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt.

Measuring mathematical problem solving with the MATH dataset.

In NeurIPS Datasets and Benchmarks, 2021b.

• Hsieh et al. (2024)

Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg.

RULER: What’s the real context size of your long-context language models?

CoRR, abs/2404.06654, 2024.

• Huang et al. (2023)

Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He.

C-Eval: A multi-level multi-discipline chinese evaluation suite for foundation models.

In NeurIPS, 2023.

• Hui et al. (2024)

Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al.

Qwen2.5-Coder technical report.

CoRR, abs/2409.12186, 2024.

• Jain et al. (2024)

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica.

LiveCodeBench: Holistic and contamination free evaluation of large language models for code.

CoRR, abs/2403.07974, 2024.

• Jiang et al. (2023)

Zixuan Jiang, Jiaqi Gu, Hanqing Zhu, and David Z. Pan.

Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and efficient pre-LN Transformers.

CoRR, abs/2305.14858, 2023.

• Lambert et al. (2024)

Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, and Hannaneh Hajishirzi.

Tülu 3: Pushing frontiers in open language model post-training.

CoRR, abs/2411.15124, 2024.

• Li et al. (2024)

Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E. Gonzalez, and Ion Stoica.

From crowdsourced data to high-quality benchmarks: Arena-Hard and BenchBuilder pipeline.

CoRR, abs/2406.11939, 2024.

• Lightman et al. (2023)

Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe.

Let’s verify step by step.

CoRR, abs/2305.20050, 2023.

• Lin et al. (2025)

Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi.

ZebraLogic: On the scaling limits of LLMs for logical reasoning.

CoRR, abs/2502.01100, 2025.

• Liu et al. (2024a)

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al.

DeepSeek-V3 technical report.

arXiv preprint arXiv:2412.19437, 2024a.

• Liu et al. (2023a)

Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang.

Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation.

In NeurIPS, 2023a.

• Liu et al. (2024b)

Qian Liu, Xiaosen Zheng, Niklas Muennighoff, Guangtao Zeng, Longxu Dou, Tianyu Pang, Jing Jiang, and Min Lin.

RegMix: Data mixture as regression for language model pre-training.

arXiv preprint arXiv:2407.01492, 2024b.

• Liu et al. (2023b)

Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Zhuoer Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, and Jie Tang.

AlignBench: Benchmarking Chinese alignment of large language models.

CoRR, abs/2311.18743, 2023b.

• Meta-AI (2025)

Meta-AI.

The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation, 2025.

URL https://ai.meta.com/blog/llama-4-multimodal-intelligence/.

• OpenAI (2024)

OpenAI.

Hello GPT-4o, 2024.

URL https://openai.com/index/hello-gpt-4o/.

• OpenAI (2024)

OpenAI.

Multilingual massive multitask language understanding, 2024.

URL https://huggingface.co/datasets/openai/MMMLU.

• OpenAI (2024)

OpenAI.

Learning to reason with LLMs, 2024.

URL https://openai.com/index/learning-to-reason-with-llms/.

• OpenAI (2025)

OpenAI.

Introducing openai o3 and o4-mini, 2025.

URL https://openai.com/index/introducing-o3-and-o4-mini/.

• Paech (2024)

Samuel J. Paech.

Creative writing v3, 2024.

URL https://eqbench.com/creative_writing.html.

• Peng et al. (2023)

Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole.

YaRN: Efficient context window extension of large language models.

CoRR, abs/2309.00071, 2023.

• Qiu et al. (2025)

Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, and Junyang Lin.

Demons in the detail: On implementing load balancing loss for training specialized mixture-of-expert models.

CoRR, abs/2501.11873, 2025.

• Quan et al. (2025)

Shanghaoran Quan, Jiaxi Yang, Bowen Yu, Bo Zheng, Dayiheng Liu, An Yang, Xuancheng Ren, Bofei Gao, Yibo Miao, Yunlong Feng, Zekun Wang, Jian Yang, Zeyu Cui, Yang Fan, Yichang Zhang, Binyuan Hui, and Junyang Lin.

CodeElo: Benchmarking competition-level code generation of LLMs with human-comparable Elo ratings.

CoRR, abs/2501.01257, 2025.

• Qwen Team (2024)

Qwen Team.

QwQ: Reflect deeply on the boundaries of the unknown, November 2024.

URL https://qwenlm.github.io/blog/qwq-32b-preview/.

• Qwen Team (2025)

Qwen Team.

QwQ-32B: Embracing the power of reinforcement learning, March 2025.

URL https://qwenlm.github.io/blog/qwq-32b/.

• Rein et al. (2023)

David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman.

GPQA: A graduate-level Google-proof Q&A benchmark.

CoRR, abs/2311.12022, 2023.

• Romanou et al. (2024)

Angelika Romanou, Negar Foroutan, Anna Sotnikova, Zeming Chen, Sree Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary, Micol Altomare, Mohamed A. Haggag, Snegha A, Alfonso Amayuelas, Azril Hafizi Amirudin, Viraat Aryabumi, Danylo Boiko, Michael Chang, Jenny Chim, Gal Cohen, Aditya Kumar Dalmia, Abraham Diress, Sharad Duwal, Daniil Dzenhaliou, Daniel Fernando Erazo Florez, Fabian Farestam, Joseph Marvin Imperial, Shayekh Bin Islam, Perttu Isotalo, Maral Jabbarishiviari, Börje F. Karlsson, Eldar Khalilov, Christopher Klamm, Fajri Koto, Dominik Krzeminski, Gabriel Adriano de Melo, Syrielle Montariol, Yiyang Nan, Joel Niklaus, Jekaterina Novikova, Johan Samir Obando Ceron, Debjit Paul, Esther Ploeger, Jebish Purbey, Swati Rajwal, Selvan Sunitha Ravi, Sara Rydell, Roshan Santhosh, Drishti Sharma, Marjana Prifti Skenduli, Arshia Soltani Moakhar, Bardia Soltani Moakhar, Ran Tamir, Ayush Kumar Tarun, Azmine Toushik Wasi, Thenuka Ovin Weerasinghe, Serhan Yilmaz, Mike Zhang, Imanol Schlag, Marzieh Fadaee, Sara Hooker, and Antoine Bosselut.

INCLUDE: evaluating multilingual language understanding with regional knowledge.

CoRR, abs/2411.19799, 2024.

• Sennrich et al. (2016)

Rico Sennrich, Barry Haddow, and Alexandra Birch.

Neural machine translation of rare words with subword units.

In ACL (1). The Association for Computer Linguistics, 2016.

• Shao et al. (2024)

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo.

DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.

CoRR, abs/2402.03300, 2024.

• Shi et al. (2023)

Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei.

Language models are multilingual chain-of-thought reasoners.

In ICLR. OpenReview.net, 2023.

• Son et al. (2025)

Guijin Son, Jiwoo Hong, Hyunwoo Ko, and James Thorne.

Linguistic generalizability of test-time scaling in mathematical reasoning.

CoRR, abs/2502.17407, 2025.

• Su et al. (2024)

Jianlin Su, Murtadha H. M. Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu.

Roformer: Enhanced Transformer with rotary position embedding.

Neurocomputing, 568:127063, 2024.

• Suzgun et al. (2023)

Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei.

Challenging BIG-Bench tasks and whether chain-of-thought can solve them.

In ACL (Findings), pp. 13003–13051. Association for Computational Linguistics, 2023.

• Team et al. (2025)

Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al.

Gemma 3 technical report.

arXiv preprint arXiv:2503.19786, 2025.

• Wang et al. (2020)

Changhan Wang, Kyunghyun Cho, and Jiatao Gu.

Neural machine translation with byte-level subwords.

In AAAI, pp. 9154–9160. AAAI Press, 2020.

• Wang et al. (2025)

Yiming Wang, Pei Zhang, Jialong Tang, Haoran Wei, Baosong Yang, Rui Wang, Chenshu Sun, Feitong Sun, Jiran Zhang, Junxuan Wu, Qiqian Cang, Yichang Zhang, Fei Huang, Junyang Lin, Fei Huang, and Jingren Zhou.

PolyMath: Evaluating mathematical reasoning in multilingual contexts, 2025.

• Wang et al. (2024)

Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen.

MMLU-Pro: A more robust and challenging multi-task language understanding benchmark.

CoRR, abs/2406.01574, 2024.

• White et al. (2024)

Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Benjamin Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Siddartha Naidu, Chinmay Hegde, Yann LeCun, Tom Goldstein, Willie Neiswanger, and Micah Goldblum.

LiveBench: A challenging, contamination-free LLM benchmark.

CoRR, abs/2406.19314, 2024.

• Wu et al. (2025)

Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, and Fei Huang.

WritingBench: A comprehensive benchmark for generative writing.

CoRR, abs/2503.05244, 2025.

• xAI (2025)

xAI.

Grok 3 beta — the age of reasoning agents, 2025.

URL https://x.ai/news/grok-3.

• Xie et al. (2023)

Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy S Liang, Quoc V Le, Tengyu Ma, and Adams Wei Yu.

Doremi: Optimizing data mixtures speeds up language model pretraining.

Advances in Neural Information Processing Systems, 36:69798–69818, 2023.

• Xiong et al. (2023)

Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, Madian Khabsa, Han Fang, Yashar Mehdad, Sharan Narang, Kshitiz Malik, Angela Fan, Shruti Bhosale, Sergey Edunov, Mike Lewis, Sinong Wang, and Hao Ma.

Effective long-context scaling of foundation models.

CoRR, abs/2309.16039, 2023.

• Yan et al. (2024)

Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, Tianjun Zhang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez.

Berkeley function calling leaderboard.

https://gorilla.cs.berkeley.edu/blogs/8_berkeley_function_calling_leaderboard.html, 2024.

• Yang et al. (2024a)

An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan.

Qwen2 technical report.

CoRR, abs/2407.10671, 2024a.

• Yang et al. (2024b)

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al.

Qwen2.5 technical report.

arXiv preprint arXiv:2412.15115, 2024b.

• Yang et al. (2024c)

An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al.

Qwen2.5-Math technical report: Toward mathematical expert model via self-improvement.

CoRR, abs/2409.12122, 2024c.

• Zhang et al. (2024)

Yidan Zhang, Boyi Deng, Yu Wan, Baosong Yang, Haoran Wei, Fei Huang, Bowen Yu, Junyang Lin, and Jingren Zhou.

P-MMEval: A parallel multilingual multitask benchmark for consistent evaluation of LLMs.

CoRR, abs/2411.09116, 2024.

• Zhou et al. (2023)

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou.

Instruction-following evaluation for large language models.

CoRR, abs/2311.07911, 2023.

• Zhu et al. (2025)

Qin Zhu, Fei Huang, Runyu Peng, Keming Lu, Bowen Yu, Qinyuan Cheng, Xipeng Qiu, Xuanjing Huang, and Junyang Lin.

AutoLogi: Automated generation of logic puzzles for evaluating reasoning abilities of large language models.

CoRR, abs/2502.16906, 2025.