论文信息与摘要（原文）

原文标题：Gemma 3 Technical Report arXiv：2503.19786（v1）　方向：第三批 大模型基座：开源前沿再跃迁

以下为 arXiv 官方 HTML 全文，一字不落保留原文结构、图表与参考文献，仅供学习研究参考，版权归原作者所有。

\authfootnotetext 1See Contributions and Acknowledgments section for full author list. Please send correspondence to gemma-3-report@google.com.

Gemma Team

Google DeepMind\authfootnotemark1

Abstract

We introduce Gemma 3, a multimodal addition to the Gemma family of lightweight open models, ranging in scale from 1 to 27 billion parameters. This version introduces vision understanding abilities, a wider coverage of languages and longer context – at least 128K tokens. We also change the architecture of the model to reduce the KV-cache memory that tends to explode with long context. This is achieved by increasing the ratio of local to global attention layers, and keeping the span on local attention short. The Gemma 3 models are trained with distillation and achieve superior performance to Gemma 2 for both pre-trained and instruction finetuned versions. In particular, our novel post-training recipe significantly improves the math, chat, instruction-following and multilingual abilities, making Gemma3-4B-IT competitive with Gemma2-27B-IT and Gemma3-27B-IT comparable to Gemini-1.5-Pro across benchmarks. We release all our models to the community.