References

本文为论文《Wan 2.1：开放先进的视频生成基座模型》「论文原文」板块的对应章节，按 arXiv 官方全文逐字保留，层级与「中文全文翻译」板块一一对应。

  • AI (2022)

Runway AI.

Stable Diffusion Inpainting Model Card, https://huggingface.co/runwayml/stable-diffusion-inpainting, 2022.

  • Bai et al. (2023)

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou.

Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond.

arXiv preprint arXiv.2308.12966, 2023.

  • Bao et al. (2023)

Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu.

All are worth words: A vit backbone for diffusion models.

In IEEE Conf. Comput. Vis. Pattern Recog., 2023.

  • Bao et al. (2024)

Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu.

Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models.

arXiv preprint arXiv:2405.04233, 2024.

  • Betker et al. (2023)

James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al.

Improving image generation with better captions (2023).

URL https://cdn. openai. com/papers/dall-e-3. pdf, 2023.

  • Blattmann et al. (2023)

Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al.

Stable video diffusion: Scaling latent video diffusion models to large datasets.

arXiv preprint arXiv:2311.15127, 2023.

  • Brooks et al. (2023)

Tim Brooks, Aleksander Holynski, and Alexei A. Efros.

InstructPix2Pix: Learning To Follow Image Editing Instructions.

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 18392–18402, 2023.

  • Caron et al. (2021)

Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.

Emerging properties in self-supervised vision transformers.

arXiv preprint arXiv:2104.14294, 2021.

  • Chen et al. (2023a)

Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al.

Pixart-$\alpha$: Fast training of diffusion transformer for photorealistic text-to-image synthesis.

arXiv preprint arXiv:2310.00426, 2023a.

  • Chen et al. (2016)

Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin.

Training deep nets with sublinear memory cost.

arXiv preprint arXiv:1604.06174, 2016.

  • Chen et al. (2023b)

Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao.

AnyDoor: Zero-shot Object-level Image Customization.

arXiv preprint arXiv:2307.09481, 2023b.

  • Chen et al. (2024)

Xi Chen, Zhifei Zhang, He Zhang, Yuqian Zhou, Soo Ye Kim, Qing Liu, Yijun Li, Jianming Zhang, Nanxuan Zhao, Yilin Wang, Hui Ding, Zhe Lin, and Hengshuang Zhao.

UniReal: Universal Image Generation and Editing via Learning Real-world Dynamics.

arXiv preprint arXiv:2412.07774, 2024.

  • Cheng et al. (2024)

Ho Kei Cheng, Masato Ishii, Akio Hayakawa, Takashi Shibuya, Alexander Schwing, and Yuki Mitsufuji.

Taming multimodal joint training for high-quality video-to-audio synthesis.

arXiv preprint arXiv:2412.15322, 2024.

  • Chu et al. (2023)

Ruihang Chu, Enze Xie, Shentong Mo, Zhenguo Li, Matthias Nießner, Chi-Wing Fu, and Jiaya Jia.

Diffcomplete: Diffusion-based generative 3d shape completion.

Adv. Neural Inform. Process. Syst., 2023.

  • Chu et al. (2024)

Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, et al.

Qwen2-audio technical report.

arXiv preprint arXiv:2407.10759, 2024.

  • Chung et al. (2023)

Hyung Won Chung, Noah Constant, Xavier Garcia, Adam Roberts, Yi Tay, Sharan Narang, and Orhan Firat.

Unimax: Fairer and more effective language sampling for large-scale multilingual pretraining.

arXiv preprint arXiv:2304.09151, 2023.

  • DeepMind (2024.12)

Google DeepMind.

Veo 2.

https://deepmind.google/technologies/veo/veo-2/, 2024.12.

  • Deng et al. (2019)

Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou.

Arcface: Additive angular margin loss for deep face recognition.

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 4690–4699, 2019.

  • Elfwing et al. (2018)

Stefan Elfwing, Eiji Uchibe, and Kenji Doya.

Sigmoid-weighted linear units for neural network function approximation in reinforcement learning.

Neural networks, 2018.

  • Esser et al. (2021)

Patrick Esser, Robin Rombach, and Bjorn Ommer.

Taming transformers for high-resolution image synthesis.

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 12873–12883, 2021.

  • Esser et al. (2024)

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al.

Scaling rectified flow transformers for high-resolution image synthesis.

In Int. Conf. Mach. Learn., 2024.

  • Fang & Zhao (2024)

Jiarui Fang and Shangchun Zhao.

Usp: A unified sequence parallelism approach for long context generative ai.

arXiv preprint arXiv:2405.07719, 2024.

  • Feng et al. (2024)

Ruili Feng, Han Zhang, Zhantao Yang, Jie Xiao, Zhilei Shu, Zhiheng Liu, Andy Zheng, Yukun Huang, Yu Liu, and Hongyang Zhang.

The matrix: Infinite-horizon world generation with real-time moving control.

arXiv preprint arXiv:2412.03568, 2024.

  • Fu et al. (2024)

Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al.

Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.

arXiv preprint arXiv:2405.21075, 2024.

  • GenmoTeam (2024)

GenmoTeam.

Mochi 1.

https://github.com/genmoai/models, 2024.

  • Guo et al. (2024a)

Xun Guo, Mingwu Zheng, Liang Hou, Yuan Gao, Yufan Deng, Pengfei Wan, Di Zhang, Yufan Liu, Weiming Hu, Zhengjun Zha, Haibin Huang, and Chongyang Ma.

I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models.

In SIGGRAPH, pp. 1–12, 2024a.

  • Guo et al. (2024b)

Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai.

Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.

International Conference on Learning Representations, 2024b.

  • HaCohen et al. (2024)

Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi.

Ltx-video: Realtime video latent diffusion.

arXiv preprint arXiv:2501.00103, 2024.

  • Han et al. (2025)

Zhen Han, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang, Chaojie Mao, Chenwei Xie, Yu Liu, and Jingren Zhou.

ACE: All-round Creator and Editor Following Instructions via Diffusion Transformer.

In Int. Conf. Learn. Represent., 2025.

  • Heusel et al. (2017)

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.

Gans trained by a two time-scale update rule converge to a local nash equilibrium.

Adv. Neural Inform. Process. Syst., 2017.

  • Ho & Salimans (2022)

Jonathan Ho and Tim Salimans.

Classifier-free diffusion guidance.

arXiv preprint arXiv:2207.12598, 2022.

  • Ho et al. (2020)

Jonathan Ho, Ajay Jain, and Pieter Abbeel.

Denoising diffusion probabilistic models.

Adv. Neural Inform. Process. Syst., 33:6840–6851, 2020.

  • Ho et al. (2022)

Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet.

Video diffusion models.

arXiv preprint arXiv:2204.03458, 2022.

  • Huang et al. (2023)

Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, and Nattapol Chanpaisit.

Vbench: Comprehensive benchmark suite for video generative models.

IEEE Conf. Comput. Vis. Pattern Recog., 2023.

  • Jacobs et al. (2023)

Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He.

Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.

arXiv preprint arXiv:2309.14509, 2023.

  • Jiang et al. (2023)

Zeyinzi Jiang, Chaojie Mao, Ziyuan Huang, Ao Ma, Yiliang Lv, Yujun Shen, Deli Zhao, and Jingren Zhou.

Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone.

In Adv. Neural Inform. Process. Syst., 2023.

  • Jiang et al. (2024)

Zeyinzi Jiang, Chaojie Mao, Yulin Pan, Zhen Han, and Jingfeng Zhang.

SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing.

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 8995–9004, 2024.

  • Jiang et al. (2025)

Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhange, Yulin Pan, and Yu Liu.

Vace: All-in-one video creation and editing.

arXiv preprint arXiv:2503.07598, 2025.

  • Jin et al. (2024)

Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin.

Pyramidal flow matching for efficient video generative modeling.

arXiv preprint arXiv:2410.05954, 2024.

  • Ke et al. (2021)

Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang.

Musiq: Multi-scale image quality transformer.

arXiv preprint arXiv:2108.05997, 2021.

  • Kingma (2013)

Diederik P Kingma.

Auto-encoding variational bayes.

arXiv preprint arXiv:1312.6114, 2013.

  • Kingma & Ba (2014)

Diederik P Kingma and Jimmy Ba.

Adam: A method for stochastic optimization.

arXiv preprint arXiv:1412.6980, 2014.

  • Kong et al. (2024)

Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al.

Hunyuanvideo: A systematic framework for large video generative models.

arXiv preprint arXiv:2412.03603, 2024.

  • (44)

Vijay Korthikanti, Jared Casper, Sangkug Lym, Lawrence McAfee, Michael Andersch, Mohammad Shoeybi, and Bryan Catanzaro.

Reducing activation recomputation in large transformer models, 2022.

arXiv preprint arXiv:2205.05198.

  • Kuaishou (2024.06)

Kuaishou.

Kling ai.

https://klingai.kuaishou.com/, 2024.06.

  • Lab & etc. (2024)

PKU-Yuan Lab and Tuzhan AI etc.

Open-sora-plan, April 2024.

  • LAION-AI (2022)

LAION-AI.

aesthetic-predictor, 2022.

URL https://github.com/LAION-AI/aesthetic-predictor.

  • Li et al. (2024)

Hengjia Li, Haonan Qiu, Shiwei Zhang, Xiang Wang, Yujie Wei, Zekun Li, Yingya Zhang, Boxi Wu, and Deng Cai.

PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation.

arXiv preprint arXiv:2411.17048, 2024.

  • Li et al. (2023)

Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia.

Mini-gemini: Mining the potential of multi-modality vision language models.

arXiv preprint arXiv:2403.18814, 2023.

  • Lin et al. (2024)

Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al.

Open-sora plan: Open-source large video generation model.

arXiv preprint arXiv:2412.00131, 2024.

  • Lipman et al. (2022)

Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le.

Flow matching for generative modeling.

arXiv preprint arXiv:2210.02747, 2022.

  • Liu et al. (2024a)

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al.

Deepseek-v3 technical report.

arXiv preprint arXiv:2412.19437, 2024a.

  • Liu et al. (2024b)

Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan.

Timestep embedding tells: It’s time to cache for video diffusion model.

arXiv preprint arXiv:2411.19108, 2024b.

  • Liu et al. (2023a)

Hao Liu, Matei Zaharia, and Pieter Abbeel.

Ring attention with blockwise transformers for near-infinite context.

arXiv preprint arXiv:2310.01889, 2023a.

  • Liu et al. (2023b)

Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley.

Audioldm: Text-to-audio generation with latent diffusion models.

arXiv preprint arXiv:2301.12503, 2023b.

  • Liu et al. (2024c)

Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Qiao Tian, Yuping Wang, Wenwu Wang, Yuxuan Wang, and Mark D Plumbley.

Audioldm 2: Learning holistic audio generation with self-supervised pretraining.

IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024c.

  • Liu et al. (2023c)

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.

Visual instruction tuning.

arXiv preprint arXiv:2304.08485, 2023c.

  • Liu et al. (2025a)

Lijie Liu, Tianxiang Ma, Bingchuan Li, Zhuowei Chen, Jiawei Liu, Qian He, and Xinglong Wu.

Phantom: Subject-consistent video generation via cross-modal alignment.

arXiv preprint arXiv:2502.11079, 2025a.

  • Liu et al. (2023d)

Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang.

Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection.

arXiv preprint arXiv:2303.05499, 2023d.

  • Liu et al. (2025b)

Zhihang Liu, Chen-Wei Xie, Bin Wen, Feiwu Yu, Jixuan Chen, Boqiang Zhang, Nianzu Yang, Pandeng Li, Yun Zheng, and Hongtao Xie.

What is a good caption? a comprehensive visual caption benchmark for evaluating both correctness and coverage of mllms.

arXiv preprint arXiv:2502.14914, 2025b.

  • Loshchilov & Hutter (2017)

Ilya Loshchilov and Frank Hutter.

Decoupled weight decay regularization.

arXiv preprint arXiv:1711.05101, 2017.

  • LumaLabs (2024.06)

LumaLabs.

Dream machine.

https://lumalabs.ai/dream-machine, 2024.06.

  • Luo et al. (2023)

Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao.

Latent consistency models: Synthesizing high-resolution images with few-step inference.

arXiv preprint arXiv:2310.04378, 2023.

  • Lv et al. (2024)

Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan-Yee K. Wong.

Fastercache: Training-free video diffusion model acceleration with high quality.

arXiv preprint arXiv:2410.19355, 2024.

  • Ma et al. (2025)

Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al.

Step-video-t2v technical report: The practice, challenges, and future of video foundation model.

arXiv preprint arXiv:2502.10248, 2025.

  • Ma et al. (2024)

Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao.

Latte: Latent diffusion transformer for video generation.

arXiv preprint arXiv:2401.03048, 2024.

  • Mao et al. (2025)

Chaojie Mao, Jingfeng Zhang, Yulin Pan, Zeyinzi Jiang, Zhen Han, Yu Liu, and Jingren Zhou.

ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Filling.

arXiv preprint arXiv:2501.02487, 2025.

  • Meng et al. (2021)

Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon.

SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations.

In Int. Conf. Learn. Represent., 2021.

  • MiniMax (2024.09)

MiniMax.

Hailuo ai.

https://hailuoai.com/video, 2024.09.

  • NVIDIA (2023)

NVIDIA.

Nvidia tensorrt: Programmable inference accelerator, 2023.

Version 8.6.

  • OpenAI (2024)

OpenAI.

Video generation models as world simulators, 2024.

URL https://openai.com/index/video-generation-models-as-world-simulators/.

  • Pan et al. (2024)

Yulin Pan, Chaojie Mao, Zeyinzi Jiang, Zhen Han, and Jingfeng Zhang.

Locate, Assign, Refine: Taming Customized Image Inpainting with Text-Subject Guidance.

arXiv preprint arXiv:2403.19534, 2024.

  • Paszke et al. (2019)

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.

Pytorch: An imperative style, high-performance deep learning library.

Adv. Neural Inform. Process. Syst., 32, 2019.

  • Peebles & Xie (2023)

William Peebles and Saining Xie.

Scalable diffusion models with transformers.

In Int. Conf. Comput. Vis., pp. 4195–4205, 2023.

  • Perez et al. (2018)

Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville.

Film: Visual reasoning with a general conditioning layer.

In Assoc. Adv. Artif. Intell., volume 32, 2018.

  • PikaLabs (2024.10)

PikaLabs.

Pika 1.5.

https://pika.art/, 2024.10.

  • (77)

Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, Luxin Zhang, Mannat Singh, Mary Williamson, Matt Le, Matthew Yu, Mitesh Kumar Singh, Peizhao Zhang, Peter Vajda, Quentin Duval, Rohit Girdhar, Roshan Sumbaly, Sai Saketh Rambhatla, Sam Tsai, Samaneh Azadi, Samyak Datta, Sanyuan Chen, Sean Bell, Sharadh Ramaswamy, Shelly Sheynin, Siddharth Bhattacharya, Simran Motwani, Tao Xu, Tianhe Li, Tingbo Hou, Wei-Ning Hsu, Xi Yin, Xiaoliang Dai, Yaniv Taigman, Yaqiao Luo, Yen-Cheng Liu, Yi-Chiao Wu, Yue Zhao, Yuval Kirstain, Zecheng He, Zijian He, Albert Pumarola, Ali Thabet, Artsiom Sanakoyeu, Arun Mallya, Baishan Guo, Boris Araya, Breena Kerr, Carleigh Wood, Ce Liu, Cen Peng, Dimitry Vengertsev, Edgar Schonfeld, Elliot Blanchard, Felix Juefei-Xu, Fraylie Nord, Jeff Liang, John Hoffman, Jonas Kohler, Kaolin Fire, Karthik Sivakumar, Lawrence Chen, Licheng Yu, Luya Gao, Markos Georgopoulos, Rashel Moritz, Sara K. Sampson, Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petrovic, and Yuming Du.

Movie gen: A cast of media foundation models.

arXiv preprint arXiv:2410.13720.

  • QwenTeam (2024)

QwenTeam.

Qwen2.5: A party of foundation models, September 2024.

URL https://qwenlm.github.io/blog/qwen2.5/.

  • Radford et al. (2021)

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.

Learning transferable visual models from natural language supervision.

In Int. Conf. Mach. Learn., pp. 8748–8763. PMLR, 2021.

  • Raffel et al. (2020)

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.

Exploring the limits of transfer learning with a unified text-to-text transformer.

The Journal of Machine Learning Research, 21(1):5485–5551, 2020.

  • Ravi et al. (2025)

Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer.

SAM 2: Segment Anything in Images and Videos.

In Int. Conf. Learn. Represent., 2025.

  • Redmon & Farhadi (2018)

Joseph Redmon and Ali Farhadi.

Yolov3: An incremental improvement.

arXiv preprint arXiv:1804.02767, 2018.

  • Rhu et al. (2016)

Minsoo Rhu, Natalia Gimelshein, Jason Clemons, Arslan Zulfiqar, and Stephen W. Keckler.

vdnn: virtualized deep neural networks for scalable, memory-efficient neural network design.

In The 49th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO-49. IEEE Press, 2016.

  • Rombach et al. (2022)

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.

High-resolution image synthesis with latent diffusion models.

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 10684–10695, 2022.

  • Ronneberger et al. (2015)

Olaf Ronneberger, Philipp Fischer, and Thomas Brox.

U-net: Convolutional networks for biomedical image segmentation.

In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234–241. Springer, 2015.

  • Runway (2023)

Runway.

Gen-2: Generate novel videos with text, images or video clips.

https://runwayml.com/research/gen-2, 2023.

  • Runway (2024.06)

Runway.

Gen-3.

https://runwayml.com/, 2024.06.

  • Schuhmann et al. (2021)

Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki.

Laion-400m: Open dataset of clip-filtered 400 million image-text pairs.

arXiv preprint arXiv:2111.02114, 2021.

  • Shah et al. (2025)

Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao.

Flashattention-3: Fast and accurate attention with asynchrony and low-precision.

Adv. Neural Inform. Process. Syst., 37:68658–68685, 2025.

  • ShengShu-AI (2024.07)

ShengShu-AI.

Vidu.

https://www.vidu.studio/, 2024.07.

  • Shoeybi et al. (2019)

Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro.

Megatron-LM: Training multi-billion parameter language models using model parallelism.

arXiv preprint arXiv:1909.08053, 2019.

  • Song et al. (2020)

Jiaming Song, Chenlin Meng, and Stefano Ermon.

Denoising diffusion implicit models.

arXiv:2010.02502, October 2020.

  • Song & Ermon (2019)

Yang Song and Stefano Ermon.

Generative modeling by estimating gradients of the data distribution.

Adv. Neural Inform. Process. Syst., 32, 2019.

  • Tan et al. (2024)

Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang.

OminiControl: Minimal and Universal Control for Diffusion Transformer.

arXiv preprint arXiv:2411.15098, 2024.

  • TeamGLM et al. (2024)

TeamGLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang, Shulin Cao, Shuxun Yang, Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang, Xixuan Song, Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang, Yueyan Li, Yushi Bai, Yuxiao Dong, Zehan Qi, Zhaoyu Wang, Zhen Yang, Zhengxiao Du, Zhenyu Hou, and Zihan Wang.

Chatglm: A family of large language models from glm-130b to glm-4 all tools, 2024.

  • Teed & Deng (2021)

Zachary Teed and Jia Deng.

Raft: Recurrent all-pairs field transforms for optical flow (extended abstract).

In Assoc. Adv. Artif. Intell., 2021.

  • torchao maintainers & contributors (2024)

torchao maintainers and contributors.

torchao: Pytorch native quantization and sparsity for training and inference, 2024.

  • Unterthiner et al. (2018)

Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, and Sylvain Gelly.

Towards accurate generative models of video: A new metric and challenges, 2018.

  • Van Den Oord et al. (2017)

Aaron Van Den Oord, Oriol Vinyals, et al.

Neural discrete representation learning.

Adv. Neural Inform. Process. Syst., 30, 2017.

  • Wang et al. (2024a)

Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny.

VGGSfM: Visual geometry grounded deep structure from motion.

In IEEE Conf. Comput. Vis. Pattern Recog., 2024a.

Highlight.

  • Wang et al. (2023a)

Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang.

Modelscope text-to-video technical report.

arXiv preprint arXiv:2308.06571, 2023a.

  • Wang et al. (2024b)

Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen.

Instantid: Zero-shot identity-preserving generation in seconds.

arXiv preprint arXiv:2401.07519, 2024b.

  • Wang et al. (2023b)

Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou.

VideoComposer: Compositional video synthesis with motion controllability.

Adv. Neural Inform. Process. Syst., 36:7594–7611, 2023b.

  • Wang et al. (2023c)

Xiang Wang, Shiwei Zhang, Han Zhang, Yu Liu, Yingya Zhang, Changxin Gao, and Nong Sang.

VideoLCM: Video latent consistency model.

arXiv preprint arXiv:2312.09109, 2023c.

  • Wang et al. (2024c)

Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, and Nong Sang.

Unianimate: Taming unified video diffusion models for consistent human image animation.

arXiv preprint arXiv:2406.01188, 2024c.

  • Wei et al. (2024a)

Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan.

DreamVideo: Composing your dream videos with customized subject and motion.

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 6537–6549, 2024a.

  • Wei et al. (2024b)

Yujie Wei, Shiwei Zhang, Hangjie Yuan, Xiang Wang, Haonan Qiu, Rui Zhao, Yutong Feng, Feng Liu, Zhizhong Huang, Jiaxin Ye, Yingya Zhang, and Hongming Shan.

DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Control.

arXiv preprint arXiv:2410.13830, 2024b.

  • Wei et al. (2025)

Yujie Wei, Shiwei Zhang, Hangjie Yuan, Biao Gong, Longxiang Tang, Xiang Wang, Haonan Qiu, Hengjia Li, Shuai Tan, Yingya Zhang, and Hongming Shan.

DreamRelation: Relation-Centric Video Customization.

arXiv preprint arXiv:2503.07602, 2025.

  • Wu et al. (2024)

Pingyu Wu, Kai Zhu, Yu Liu, Liming Zhao, Wei Zhai, Yang Cao, and Zheng-Jun Zha.

Improved video vae for latent video diffusion model.

arXiv preprint arXiv:2411.06449, 2024.

  • Wu & He (2018)

Yuxin Wu and Kaiming He.

Group normalization.

In Eur. Conf. Comput. Vis., pp. 3–19, 2018.

  • Xiao et al. (2024)

Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu.

OmniGen: Unified Image Generation.

arXiv preprint arXiv:2409.11340, 2024.

  • Xie et al. (2022)

Chen-Wei Xie, Jianmin Wu, Yun Zheng, Pan Pan, and Xian-Sheng Hua.

Token embeddings alignment for cross-modal retrieval.

In ACM Int. Conf. Multimedia, 2022.

  • Yang et al. (2025a)

Nianzu Yang, Pandeng Li, Liming Zhao, Yang Li, Chen-Wei Xie, Yehui Tang, Xudong Lu, Zhihang Liu, Yun Zheng, Yu Liu, and Junchi Yan.

Rethinking video tokenization: A conditioned diffusion-based approach.

arXiv preprint arXiv:2503.03708, 2025a.

  • Yang et al. (2022)

Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang.

Maniqa: Multi-dimension attention network for no-reference image quality assessment.

arXiv e-prints, 2022.

  • Yang et al. (2025b)

Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Xiaotao Gu, Yuxuan Zhang, Weihan Wang, Yean Cheng, Ting Liu, Bin Xu, Yuxiao Dong, and Jie Tang.

CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer.

In Int. Conf. Learn. Represent., 2025b.

  • Yao et al. (2021)

Zhuoyuan Yao, Di Wu, Xiong Wang, Binbin Zhang, Fan Yu, Chao Yang, Zhendong Peng, Xiaoyu Chen, Lei Xie, and Xin Lei.

Wenet: Production oriented streaming and non-streaming end-to-end speech recognition toolkit.

In InterSpeech, 2021.

  • Ye et al. (2023)

Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang.

Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.

arXiv preprint arXiv:2308.06721, 2023.

  • Yu et al. (2023)

Lijun Yu, José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al.

Language model beats diffusion–tokenizer is key to visual generation.

arXiv preprint arXiv:2310.05737, 2023.

  • Yuan et al. (2024a)

Hangjie Yuan, Shiwei Zhang, Xiang Wang, Yujie Wei, Tao Feng, Yining Pan, Yingya Zhang, Ziwei Liu, Samuel Albanie, and Dong Ni.

InstructVideo: Instructing video diffusion models with human feedback.

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 6463–6474, 2024a.

  • Yuan et al. (2025)

Shenghai Yuan, Jinfa Huang, Xianyi He, Yunyuan Ge, Yujun Shi, Liuhan Chen, Jiebo Luo, and Li Yuan.

Identity-Preserving Text-to-Video Generation by Frequency Decomposition.

In IEEE Conf. Comput. Vis. Pattern Recog., 2025.

  • Yuan et al. (2024b)

Zhihang Yuan, Hanling Zhang, Lu Pu, Xuefei Ning, Linfeng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, and Yu Wang.

DiTFastattn: Attention compression for diffusion transformer models.

In Adv. Neural Inform. Process. Syst., 2024b.

  • Zhai et al. (2023)

Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer.

Sigmoid loss for language image pre-training.

In Int. Conf. Comput. Vis., pp. 11975–11986, 2023.

  • Zhang & Sennrich (2019)

Biao Zhang and Rico Sennrich.

Root mean square layer normalization.

Adv. Neural Inform. Process. Syst., 32, 2019.

  • Zhang et al. (2024)

Jintao Zhang, Haofeng Huang, Pengle Zhang, Jun Zhu, Jianfei Chen, et al.

Sageattention: Accurate 8-bit attention for plug-and-play inference acceleration.

arXiv preprint arXiv:2410.02367, 2024.

  • Zhang et al. (2023a)

Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su.

MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing.

In Adv. Neural Inform. Process. Syst., 2023a.

  • Zhang et al. (2023b)

Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.

Adding conditional control to text-to-image diffusion models.

In Int. Conf. Comput. Vis., pp. 3836–3847, 2023b.

  • Zhang et al. (2023c)

Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou.

I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models.

arXiv preprint arXiv:2311.04145, 2023c.

  • Zhang et al. (2023d)

Youcai Zhang, Xinyu Huang, Jinyu Ma, Zhaoyang Li, Zhaochuan Luo, Yanchun Xie, Yuzhuo Qin, Tong Luo, Yaqian Li, Shilong Liu, Yandong Guo, and Lei Zhang.

Recognize Anything: A Strong Image Tagging Model.

arXiv preprint arXiv:2306.03514, 2023d.

  • Zhao et al. (2023)

Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, and Shen Li.

Pytorch fsdp: Experiences on scaling fully sharded data parallel, 2023.

  • Zheng et al. (2024)

Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You.

Open-sora: Democratizing efficient video production for all, March 2024.

  • Zhou et al. (2022)

Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng.

Magicvideo: Efficient video generation with latent diffusion models.

arXiv preprint arXiv:2211.11018, 2022.

  • Zhou et al. (2023)

Shangchen Zhou, Chongyi Li, Kelvin C. K. Chan, and Chen Change Loy.

ProPainter: Improving Propagation and Transformer for Video Inpainting.

In Int. Conf. Comput. Vis., pp. 10477–10486, 2023.