Appendix C Expert Specialization Patterns of the 16B Aux-Loss-Based and Aux-Loss-Free Models

本文为论文《DeepSeek-V3：强大高效的开源混合专家大模型》「论文原文」板块的对应章节，按 arXiv 官方全文逐字保留，层级与「中文全文翻译」板块一一对应。

We record the expert load of the 16B auxiliary-loss-based baseline and the auxiliary-loss-free model on the Pile test set. The auxiliary-loss-free model tends to have greater expert specialization across all layers, as demonstrated in Figure 11.

(a) Layers 1-7

(b) Layers 7-13

(c) Layers 13-19

(d) Layers 19-25

(e) Layers 25-27

Figure 11: Expert load of auxiliary-loss-free and auxiliary-loss-based models on three domains in the Pile test set. The auxiliary-loss-free model shows greater expert specialization patterns than the auxiliary-loss-based one. The relative expert load denotes the ratio between the actual expert load and the theoretically balanced expert load.