• 论文信息与摘要
• 1 Introduction
• 2 Architecture
• 3 Infrastructures
• 4 Pre-Training
• 5 Post-Training
• 6 Conclusion, Limitations, and Future Directions
• References
• Appendix
• Appendix A Contributions and Acknowledgments
• Appendix B Ablation Studies for Low-Precision Training
• Appendix C Expert Specialization Patterns of the 16B Aux-Loss-Based and Aux-Loss-Free Models