2 Related Work

本文为论文《Wan 2.1：开放先进的视频生成基座模型》「论文原文」板块的对应章节，按 arXiv 官方全文逐字保留，层级与「中文全文翻译」板块一一对应。

Driven by advances in generative modeling, the landscape of large-scale video models has evolved significantly, particularly in diffusion-based frameworks. Our review focuses on two broad categories: closed-source models and contributions from the open-source community.

Closed-source models. Closed-source models are mainly those developed by major technology companies, which aims at high-quality, professional video generation due to the extensive resources invested. We organize the notable models released over the past year in chronological order. In February 2024, OpenAI introduced Sora (OpenAI, 2024) and marked a significant leap in AI-generated content. In June 2024, Kling (Kuaishou, 2024.06) by Kuaishou and Luma 1.0 (LumaLabs, 2024.06) by LUMA AI were opened up for public testing, offering powerful video generation capabilities. Meanwhile, Runway unveiled Gen-3 (Runway, 2024.06), building upon Gen-2 (Runway, 2023), to further elevate the standards in video creation. In July 2024, Shengshu AI released Vidu (Bao et al., 2024) equipped with self-designed U-ViT (Bao et al., 2023) architecture. In September 2024, both Kling and Luma undergone updates to version 1.5. During the same period, MiniMax introduced Hailuo Video (MiniMax, 2024.09), delivering impressive visual results to the public. In October 2024, Pika Labs launched Pika 1.5 (PikaLabs, 2024.10), enabling users to customize visual and physical attributes in videos. In addition, Meta introduced Movie Gen (Polyak et al., ), a series of video foundation models that detailed their training processes and applications. In December 2024, Kling advanced to version 1.6 and, concurrently, Google released Veo 2 (DeepMind, 2024.12) with an improved understanding of real-world physics and human movement nuances.

These developments underscore the intense global competition in the video generation sector. In this context, our open-source Wan has shown competitive or even superior performance over these commercial models across both internal and external benchmarks, leading in multiple attributes.

Contributions from open-source community. On the other hand, the open-source community has made substantial contributions not only to the holistic video generative models but also to the exploration of essential model components. Diffusion-based video generation models typically build on Stable Diffusion (Rombach et al., 2022) architecture. It mainly involves three critical modules: an autoencoder to map the original video into a compact latent space, a text encoder to extract text embeddings, and a neural network optimized via the diffusion model to learn the distribution of these video latents. For network structure, U-Net (Ronneberger et al., 2015), initially used in image generation (Ho et al., 2020; Song et al., 2020), is adapted for video generation by incorporating temporal dimensions. VDM (Ho et al., 2022) extends the 2D U-Net to a 3D version, while another paradigm (Zhou et al., 2022; Wang et al., 2023a; Guo et al., 2024b) introduces 1D temporal attention combined with 2D spatial attention block to reduce computation costs. Notably, Diffusion Transformers (DiT (Peebles & Xie, 2023)), which rely solely on transformer blocks, are superior to U-Net in visual generation tasks (Chen et al., 2023a). This structure has also been transferred to video models (Ma et al., 2024), giving rise to two common variants: the original DiT (Peebles & Xie, 2023; HaCohen et al., 2024) that uses cross-attention for text embeddings and MM-DiT (GenmoTeam, 2024; Kong et al., 2024), where text embeddings are concatenated to visual embeddings for full-attention processing. Regarding autoencoders, while the early approach (Rombach et al., 2022) adopts standard VAE (Kingma, 2013), recent autoencoders like VQ-VAE (Van Den Oord et al., 2017) and VQGAN (Esser et al., 2021) improve model design for better reconstruction and compression. LTX-Video (HaCohen et al., 2024) modifies the VAE decoder to perform the final denoising step and convert latents into pixels, where the missing high-frequency details are generated during decoding. The text encoder is also critical for text-based video generation. Current powerful video generation models primarily utilize the T5 series (Raffel et al., 2020) as the main text encoder, often in conjunction with CLIP (Radford et al., 2021). In the case of HunyuanVideo (Kong et al., 2024), T5 is replaced with a Multimodal Large Language Model (Liu et al., 2023c; Li et al., 2023) to achieve more robust alignment between text embeddings and visual features.

By integrating these key modules with effective diffusion-based optimization techniques (Ho et al., 2020; Song & Ermon, 2019; Lipman et al., 2022), multiple promising open-source video generation models have emerged (GenmoTeam, 2024; Kong et al., 2024; HaCohen et al., 2024; Zheng et al., 2024; Lin et al., 2024; Jin et al., 2024; Yang et al., 2025b). In Wan, we have carefully designed or selected each critical component to ensure high-quality video synthesis. We provide detailed design information and ablation study, which can facilitate the design of future video generation models.

In addition, numerous studies have explored downstream tasks in video generation. These tasks include repainting (AI, 2022; Zhou et al., 2023), editing (Meng et al., 2021; Brooks et al., 2023; Zhang et al., 2023a; Wang et al., 2023b; Wei et al., 2024b), controllable generation (Zhang et al., 2023b; Jiang et al., 2024; Wang et al., 2024c), and frame reference generation (Yang et al., 2025b; Guo et al., 2024a), often by leveraging adapter-based and ControlNet-like architectures (Zhang et al., 2023b; Chu et al., 2023) to incorporate user-specified conditions. We also develop various downstream applications based on Wan, which have demonstrated excellent performance.