References

本文为论文《Wan 2.1：开放先进的视频生成基座模型》「中文全文翻译」板块的对应章节，与「论文原文」板块同一章节一一对应，层级保持一致。

  • AI (2022)

Runway AI.

Stable Diffusion Inpainting 模型卡片, https://huggingface.co/runwayml/stable-diffusion-inpainting, 2022.

  • Bai et al. (2023)

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou.

Qwen-VL：一个用于理解、定位、文本阅读等多用途的视觉语言模型。

arXiv preprint arXiv.2308.12966, 2023.

  • Bao et al. (2023)

Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu.

万物皆值得用文字表达：用于扩散模型的视觉Transformer主干网络。

In IEEE Conf. Comput. Vis. Pattern Recog., 2023.

  • Bao et al. (2024)

Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu.

Vidu：一个基于扩散模型、具有高度一致性、动态性和技巧性的文本生成视频生成器。

arXiv preprint arXiv:2405.04233, 2024.

  • Betker et al. (2023)

James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al.

通过更好的图像描述提升图像生成能力（2023）。

URL https://cdn. openai. com/papers/dall-e-3. pdf, 2023.

  • Blattmann et al. (2023)

Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al.

Stable Video Diffusion：将潜在视频扩散模型扩展到大规模数据集。

arXiv preprint arXiv:2311.15127, 2023.

  • Brooks et al. (2023)

Tim Brooks, Aleksander Holynski, and Alexei A. Efros.

InstructPix2Pix：学习遵循图像编辑指令。

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 18392–18402, 2023.

  • Caron et al. (2021)

Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.

自监督视觉Transformer中涌现的特性。

arXiv preprint arXiv:2104.14294, 2021.

  • Chen et al. (2023a)

Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al.

Pixart-$\alpha$：用于照片级真实感文本到图像合成的快速扩散Transformer训练。

arXiv preprint arXiv:2310.00426, 2023a.

  • Chen et al. (2016)

Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin.

以亚线性内存开销训练深度网络。

arXiv preprint arXiv:1604.06174, 2016.

  • Chen et al. (2023b)

Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao.

AnyDoor：零样本物体级图像定制。

arXiv preprint arXiv:2307.09481, 2023b.

  • Chen et al. (2024)

Xi Chen, Zhifei Zhang, He Zhang, Yuqian Zhou, Soo Ye Kim, Qing Liu, Yijun Li, Jianming Zhang, Nanxuan Zhao, Yilin Wang, Hui Ding, Zhe Lin, and Hengshuang Zhao.

UniReal：通过学习真实世界动态实现通用图像生成与编辑。

arXiv preprint arXiv:2412.07774, 2024.

  • Cheng et al. (2024)

Ho Kei Cheng, Masato Ishii, Akio Hayakawa, Takashi Shibuya, Alexander Schwing, and Yuki Mitsufuji.

驯服多模态联合训练以实现高质量视频到音频合成。

arXiv preprint arXiv:2412.15322, 2024.

  • Chu et al. (2023)

Ruihang Chu, Enze Xie, Shentong Mo, Zhenguo Li, Matthias Nießner, Chi-Wing Fu, and Jiaya Jia.

Diffcomplete：基于扩散的生成式三维形状补全。

Adv. Neural Inform. Process. Syst., 2023.

  • Chu et al. (2024)

Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, et al.

Qwen2-Audio 技术报告。

arXiv preprint arXiv:2407.10759, 2024.

  • Chung et al. (2023)

Hyung Won Chung, Noah Constant, Xavier Garcia, Adam Roberts, Yi Tay, Sharan Narang, and Orhan Firat.

Unimax：面向大规模多语言预训练更公平且更有效的语言采样。

arXiv preprint arXiv:2304.09151, 2023.

  • DeepMind (2024.12)

Google DeepMind.

Veo 2.

https://deepmind.google/technologies/veo/veo-2/, 2024.12.

  • Deng et al. (2019)

Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou.

ArcFace：用于深度人脸识别的加性角度间隔损失。

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 4690–4699, 2019.

  • Elfwing et al. (2018)

Stefan Elfwing, Eiji Uchibe, and Kenji Doya.

用于强化学习中神经网络函数逼近的Sigmoid加权线性单元。

Neural networks, 2018.

  • Esser et al. (2021)

Patrick Esser, Robin Rombach, and Bjorn Ommer.

驯服Transformer以实现高分辨率图像合成。

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 12873–12883, 2021.

  • Esser et al. (2024)

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al.

扩展校正流Transformer以实现高分辨率图像合成。

In Int. Conf. Mach. Learn., 2024.

  • Fang & Zhao (2024)

Jiarui Fang and Shangchun Zhao.

USP：一种用于长上下文生成式AI的统一序列并行方法。

arXiv preprint arXiv:2405.07719, 2024.

  • Feng et al. (2024)

Ruili Feng, Han Zhang, Zhantao Yang, Jie Xiao, Zhilei Shu, Zhiheng Liu, Andy Zheng, Yukun Huang, Yu Liu, and Hongyang Zhang.

The Matrix：具有实时移动控制的无限时域世界生成。

arXiv preprint arXiv:2412.03568, 2024.

  • Fu et al. (2024)

Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al.

Video-MME：首个面向视频分析中多模态大语言模型的综合评测基准。

arXiv preprint arXiv:2405.21075, 2024.

  • GenmoTeam (2024)

GenmoTeam.

Mochi 1.

https://github.com/genmoai/models, 2024.

  • Guo et al. (2024a)

Xun Guo, Mingwu Zheng, Liang Hou, Yuan Gao, Yufan Deng, Pengfei Wan, Di Zhang, Yufan Liu, Weiming Hu, Zhengjun Zha, Haibin Huang, and Chongyang Ma.

I2V-Adapter：一种用于扩散模型的通用图像到视频适配器。

In SIGGRAPH, pp. 1–12, 2024a.

  • Guo et al. (2024b)

Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai.

AnimateDiff：无需特定调优即可动画化你的个性化文本到图像扩散模型。

International Conference on Learning Representations, 2024b.

  • HaCohen et al. (2024)

Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi.

LTX-Video：实时视频潜在扩散。

arXiv preprint arXiv:2501.00103, 2024.

  • Han et al. (2025)

Zhen Han, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang, Chaojie Mao, Chenwei Xie, Yu Liu, and Jingren Zhou.

ACE：基于扩散Transformer、遵循指令的全能创作与编辑模型。

In Int. Conf. Learn. Represent., 2025.

  • Heusel et al. (2017)

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.

采用双时间尺度更新规则训练的GAN收敛到局部纳什均衡。

Adv. Neural Inform. Process. Syst., 2017.

  • Ho & Salimans (2022)

Jonathan Ho and Tim Salimans.

无分类器扩散引导。

arXiv preprint arXiv:2207.12598, 2022.

  • Ho et al. (2020)

Jonathan Ho, Ajay Jain, and Pieter Abbeel.

去噪扩散概率模型。

Adv. Neural Inform. Process. Syst., 33:6840–6851, 2020.

  • Ho et al. (2022)

Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet.

视频扩散模型。

arXiv preprint arXiv:2204.03458, 2022.

  • Huang et al. (2023)

Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, and Nattapol Chanpaisit.

VBench：面向视频生成模型的综合性基准测试套件。

IEEE Conf. Comput. Vis. Pattern Recog., 2023.

  • Jacobs et al. (2023)

Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He.

DeepSpeed Ulysses：用于支持极长序列Transformer模型训练的系统优化。

arXiv preprint arXiv:2309.14509, 2023.

  • Jiang et al. (2023)

Zeyinzi Jiang, Chaojie Mao, Ziyuan Huang, Ao Ma, Yiliang Lv, Yujun Shen, Deli Zhao, and Jingren Zhou.

Res-Tuning：一种通过将调优器与主干网络解耦实现的灵活高效调优范式。

In Adv. Neural Inform. Process. Syst., 2023.

  • Jiang et al. (2024)

Zeyinzi Jiang, Chaojie Mao, Yulin Pan, Zhen Han, and Jingfeng Zhang.

SCEdit：通过跳跃连接编辑实现高效可控的图像扩散生成。

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 8995–9004, 2024.

  • Jiang et al. (2025)

Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhange, Yulin Pan, and Yu Liu.

VACE：一站式视频创作与编辑。

arXiv preprint arXiv:2503.07598, 2025.

  • Jin et al. (2024)

Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin.

面向高效视频生成建模的金字塔流匹配。

arXiv preprint arXiv:2410.05954, 2024.

  • Ke et al. (2021)

Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang.

MUSIQ：多尺度图像质量Transformer。

arXiv preprint arXiv:2108.05997, 2021.

  • Kingma (2013)

Diederik P Kingma.

变分自编码贝叶斯方法。

arXiv preprint arXiv:1312.6114, 2013.

  • Kingma & Ba (2014)

Diederik P Kingma and Jimmy Ba.

Adam：一种随机优化方法。

arXiv preprint arXiv:1412.6980, 2014.

  • Kong et al. (2024)

Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al.

HunyuanVideo：一个面向大型视频生成模型的系统化框架。

arXiv preprint arXiv:2412.03603, 2024.

  • (44)

Vijay Korthikanti, Jared Casper, Sangkug Lym, Lawrence McAfee, Michael Andersch, Mohammad Shoeybi, and Bryan Catanzaro.

减少大型Transformer模型中的激活重计算，2022。

arXiv preprint arXiv:2205.05198.

  • Kuaishou (2024.06)

Kuaishou.

可灵 AI。

https://klingai.kuaishou.com/, 2024.06.

  • Lab & etc. (2024)

PKU-Yuan Lab and Tuzhan AI etc.

Open-Sora-Plan，2024年4月。

  • LAION-AI (2022)

LAION-AI.

美学预测器，2022。

URL https://github.com/LAION-AI/aesthetic-predictor.

  • Li et al. (2024)

Hengjia Li, Haonan Qiu, Shiwei Zhang, Xiang Wang, Yujie Wei, Zekun Li, Yingya Zhang, Boxi Wu, and Deng Cai.

PersonalVideo：在不损失动态与语义质量的前提下实现高身份保真度的视频定制。

arXiv preprint arXiv:2411.17048, 2024.

  • Li et al. (2023)

Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia.

Mini-Gemini：挖掘多模态视觉语言模型的潜力。

arXiv preprint arXiv:2403.18814, 2023.

  • Lin et al. (2024)

Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al.

Open-Sora Plan：开源大型视频生成模型。

arXiv preprint arXiv:2412.00131, 2024.

  • Lipman et al. (2022)

Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le.

用于生成式建模的流匹配。

arXiv preprint arXiv:2210.02747, 2022.

  • Liu et al. (2024a)

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al.

DeepSeek-V3 技术报告。

arXiv preprint arXiv:2412.19437, 2024a.

  • Liu et al. (2024b)

Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan.

时间步嵌入透露了秘密：视频扩散模型是时候进行缓存了。

arXiv preprint arXiv:2411.19108, 2024b.

  • Liu et al. (2023a)

Hao Liu, Matei Zaharia, and Pieter Abbeel.

结合分块Transformer的环形注意力，实现近乎无限长的上下文。

arXiv preprint arXiv:2310.01889, 2023a.

  • Liu et al. (2023b)

Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley.

AudioLDM：基于潜在扩散模型的文本到音频生成。

arXiv preprint arXiv:2301.12503, 2023b.

  • Liu et al. (2024c)

Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Qiao Tian, Yuping Wang, Wenwu Wang, Yuxuan Wang, and Mark D Plumbley.

AudioLDM 2：通过自监督预训练学习整体化音频生成。

IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024c.

  • Liu et al. (2023c)

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.

视觉指令微调。

arXiv preprint arXiv:2304.08485, 2023c.

  • Liu et al. (2025a)

Lijie Liu, Tianxiang Ma, Bingchuan Li, Zhuowei Chen, Jiawei Liu, Qian He, and Xinglong Wu.

Phantom：通过跨模态对齐实现主体一致的视频生成。

arXiv preprint arXiv:2502.11079, 2025a.

  • Liu et al. (2023d)

Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang.

Grounding DINO：将DINO与接地预训练相结合以实现开放集目标检测。

arXiv preprint arXiv:2303.05499, 2023d.

  • Liu et al. (2025b)

Zhihang Liu, Chen-Wei Xie, Bin Wen, Feiwu Yu, Jixuan Chen, Boqiang Zhang, Nianzu Yang, Pandeng Li, Yun Zheng, and Hongtao Xie.

什么是好的图像描述？一个用于同时评估多模态大语言模型正确性与覆盖度的综合性视觉描述基准。

arXiv preprint arXiv:2502.14914, 2025b.

  • Loshchilov & Hutter (2017)

Ilya Loshchilov and Frank Hutter.

解耦权重衰减正则化。

arXiv preprint arXiv:1711.05101, 2017.

  • LumaLabs (2024.06)

LumaLabs.

Dream Machine。

https://lumalabs.ai/dream-machine, 2024.06.

  • Luo et al. (2023)

Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao.

潜在一致性模型：以少步推理合成高分辨率图像。

arXiv preprint arXiv:2310.04378, 2023.

  • Lv et al. (2024)

Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan-Yee K. Wong.

FasterCache：无需训练的高质量视频扩散模型加速。

arXiv preprint arXiv:2410.19355, 2024.

  • Ma et al. (2025)

Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al.

Step-Video-T2V 技术报告：视频基础模型的实践、挑战与未来。

arXiv preprint arXiv:2502.10248, 2025.

  • Ma et al. (2024)

Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao.

Latte：用于视频生成的潜在扩散Transformer。

arXiv preprint arXiv:2401.03048, 2024.

  • Mao et al. (2025)

Chaojie Mao, Jingfeng Zhang, Yulin Pan, Zeyinzi Jiang, Zhen Han, Yu Liu, and Jingren Zhou.

ACE++：基于指令、通过上下文感知内容填充实现的图像创作与编辑。

arXiv preprint arXiv:2501.02487, 2025.

  • Meng et al. (2021)

Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon.

SDEdit：基于随机微分方程的引导式图像合成与编辑。

In Int. Conf. Learn. Represent., 2021.

  • MiniMax (2024.09)

MiniMax.

海螺 AI。

https://hailuoai.com/video, 2024.09.

  • NVIDIA (2023)

NVIDIA.

NVIDIA TensorRT：可编程推理加速器，2023。

Version 8.6.

  • OpenAI (2024)

OpenAI.

作为世界模拟器的视频生成模型，2024。

URL https://openai.com/index/video-generation-models-as-world-simulators/.

  • Pan et al. (2024)

Yulin Pan, Chaojie Mao, Zeyinzi Jiang, Zhen Han, and Jingfeng Zhang.

定位、分配、精炼：以文本-主体引导驯服定制化图像修复。

arXiv preprint arXiv:2403.19534, 2024.

  • Paszke et al. (2019)

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.

PyTorch：一种命令式风格的高性能深度学习库。

Adv. Neural Inform. Process. Syst., 32, 2019.

  • Peebles & Xie (2023)

William Peebles and Saining Xie.

可扩展的基于Transformer的扩散模型。

In Int. Conf. Comput. Vis., pp. 4195–4205, 2023.

  • Perez et al. (2018)

Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville.

FiLM：使用通用条件化层进行视觉推理。

In Assoc. Adv. Artif. Intell., volume 32, 2018.

  • PikaLabs (2024.10)

PikaLabs.

Pika 1.5。

https://pika.art/, 2024.10.

  • (77)

Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, Luxin Zhang, Mannat Singh, Mary Williamson, Matt Le, Matthew Yu, Mitesh Kumar Singh, Peizhao Zhang, Peter Vajda, Quentin Duval, Rohit Girdhar, Roshan Sumbaly, Sai Saketh Rambhatla, Sam Tsai, Samaneh Azadi, Samyak Datta, Sanyuan Chen, Sean Bell, Sharadh Ramaswamy, Shelly Sheynin, Siddharth Bhattacharya, Simran Motwani, Tao Xu, Tianhe Li, Tingbo Hou, Wei-Ning Hsu, Xi Yin, Xiaoliang Dai, Yaniv Taigman, Yaqiao Luo, Yen-Cheng Liu, Yi-Chiao Wu, Yue Zhao, Yuval Kirstain, Zecheng He, Zijian He, Albert Pumarola, Ali Thabet, Artsiom Sanakoyeu, Arun Mallya, Baishan Guo, Boris Araya, Breena Kerr, Carleigh Wood, Ce Liu, Cen Peng, Dimitry Vengertsev, Edgar Schonfeld, Elliot Blanchard, Felix Juefei-Xu, Fraylie Nord, Jeff Liang, John Hoffman, Jonas Kohler, Kaolin Fire, Karthik Sivakumar, Lawrence Chen, Licheng Yu, Luya Gao, Markos Georgopoulos, Rashel Moritz, Sara K. Sampson, Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petrovic, and Yuming Du.

Movie Gen：一组媒体基础模型。

arXiv preprint arXiv:2410.13720.

  • QwenTeam (2024)

QwenTeam.

Qwen2.5：基础模型家族，2024年9月。

URL https://qwenlm.github.io/blog/qwen2.5/.

  • Radford et al. (2021)

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.

从自然语言监督中学习可迁移的视觉模型。

In Int. Conf. Mach. Learn., pp. 8748–8763. PMLR, 2021.

  • Raffel et al. (2020)

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.

以统一的文本到文本Transformer探索迁移学习的极限。

The Journal of Machine Learning Research, 21(1):5485–5551, 2020.

  • Ravi et al. (2025)

Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer.

SAM 2：在图像与视频中分割一切。

In Int. Conf. Learn. Represent., 2025.

  • Redmon & Farhadi (2018)

Joseph Redmon and Ali Farhadi.

YOLOv3：一项增量式改进。

arXiv preprint arXiv:1804.02767, 2018.

  • Rhu et al. (2016)

Minsoo Rhu, Natalia Gimelshein, Jason Clemons, Arslan Zulfiqar, and Stephen W. Keckler.

vDNN：用于可扩展、内存高效神经网络设计的虚拟化深度神经网络。

In The 49th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO-49. IEEE Press, 2016.

  • Rombach et al. (2022)

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.

基于潜在扩散模型的高分辨率图像合成。

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 10684–10695, 2022.

  • Ronneberger et al. (2015)

Olaf Ronneberger, Philipp Fischer, and Thomas Brox.

U-Net：用于生物医学图像分割的卷积网络。

In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234–241. Springer, 2015.

  • Runway (2023)

Runway.

Gen-2：利用文本、图像或视频片段生成新颖视频。

https://runwayml.com/research/gen-2, 2023.

  • Runway (2024.06)

Runway.

Gen-3。

https://runwayml.com/, 2024.06.

  • Schuhmann et al. (2021)

Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki.

LAION-400M：由CLIP筛选的四亿图像-文本对开放数据集。

arXiv preprint arXiv:2111.02114, 2021.

  • Shah et al. (2025)

Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao.

FlashAttention-3：基于异步与低精度的快速精确注意力。

Adv. Neural Inform. Process. Syst., 37:68658–68685, 2025.

  • ShengShu-AI (2024.07)

ShengShu-AI.

Vidu。

https://www.vidu.studio/, 2024.07.

  • Shoeybi et al. (2019)

Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro.

Megatron-LM：利用模型并行训练数十亿参数的语言模型。

arXiv preprint arXiv:1909.08053, 2019.

  • Song et al. (2020)

Jiaming Song, Chenlin Meng, and Stefano Ermon.

去噪扩散隐式模型。

arXiv:2010.02502, October 2020.

  • Song & Ermon (2019)

Yang Song and Stefano Ermon.

通过估计数据分布梯度进行生成式建模。

Adv. Neural Inform. Process. Syst., 32, 2019.

  • Tan et al. (2024)

Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang.

OminiControl：面向扩散Transformer的极简通用控制。

arXiv preprint arXiv:2411.15098, 2024.

  • TeamGLM et al. (2024)

TeamGLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang, Shulin Cao, Shuxun Yang, Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang, Xixuan Song, Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang, Yueyan Li, Yushi Bai, Yuxiao Dong, Zehan Qi, Zhaoyu Wang, Zhen Yang, Zhengxiao Du, Zhenyu Hou, and Zihan Wang.

ChatGLM：从GLM-130B到GLM-4全工具化的一系列大语言模型，2024。

  • Teed & Deng (2021)

Zachary Teed and Jia Deng.

RAFT：用于光流计算的循环全配对场变换（扩展摘要）。

In Assoc. Adv. Artif. Intell., 2021.

  • torchao maintainers & contributors (2024)

torchao maintainers and contributors.

torchao：面向训练与推理的PyTorch原生量化与稀疏化，2024。

  • Unterthiner et al. (2018)

Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, and Sylvain Gelly.

迈向精确的视频生成模型：一项新度量标准与挑战，2018。

  • Van Den Oord et al. (2017)

Aaron Van Den Oord, Oriol Vinyals, et al.

神经离散表示学习。

Adv. Neural Inform. Process. Syst., 30, 2017.

  • Wang et al. (2024a)

Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny.

VGGSfM：基于视觉几何的深度运动恢复结构。

In IEEE Conf. Comput. Vis. Pattern Recog., 2024a.

Highlight.

  • Wang et al. (2023a)

Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang.

ModelScope 文本到视频技术报告。

arXiv preprint arXiv:2308.06571, 2023a.

  • Wang et al. (2024b)

Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen.

InstantID：秒级零样本身份保持生成。

arXiv preprint arXiv:2401.07519, 2024b.

  • Wang et al. (2023b)

Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou.

VideoComposer：具有运动可控性的组合式视频合成。

Adv. Neural Inform. Process. Syst., 36:7594–7611, 2023b.

  • Wang et al. (2023c)

Xiang Wang, Shiwei Zhang, Han Zhang, Yu Liu, Yingya Zhang, Changxin Gao, and Nong Sang.

VideoLCM：视频潜在一致性模型。

arXiv preprint arXiv:2312.09109, 2023c.

  • Wang et al. (2024c)

Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, and Nong Sang.

UniAnimate：驯服统一视频扩散模型以实现一致的人体图像动画。

arXiv preprint arXiv:2406.01188, 2024c.

  • Wei et al. (2024a)

Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan.

DreamVideo：用定制化主体与运动合成你的梦想视频。

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 6537–6549, 2024a.

  • Wei et al. (2024b)

Yujie Wei, Shiwei Zhang, Hangjie Yuan, Xiang Wang, Haonan Qiu, Rui Zhao, Yutong Feng, Feng Liu, Zhizhong Huang, Jiaxin Ye, Yingya Zhang, and Hongming Shan.

DreamVideo-2：具有精确运动控制的零样本主体驱动视频定制。

arXiv preprint arXiv:2410.13830, 2024b.

  • Wei et al. (2025)

Yujie Wei, Shiwei Zhang, Hangjie Yuan, Biao Gong, Longxiang Tang, Xiang Wang, Haonan Qiu, Hengjia Li, Shuai Tan, Yingya Zhang, and Hongming Shan.

DreamRelation：以关系为中心视频定制。

arXiv preprint arXiv:2503.07602, 2025.

  • Wu et al. (2024)

Pingyu Wu, Kai Zhu, Yu Liu, Liming Zhao, Wei Zhai, Yang Cao, and Zheng-Jun Zha.

用于潜在视频扩散模型的改进版视频VAE。

arXiv preprint arXiv:2411.06449, 2024.

  • Wu & He (2018)

Yuxin Wu and Kaiming He.

组归一化。

In Eur. Conf. Comput. Vis., pp. 3–19, 2018.

  • Xiao et al. (2024)

Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu.

OmniGen：统一图像生成。

arXiv preprint arXiv:2409.11340, 2024.

  • Xie et al. (2022)

Chen-Wei Xie, Jianmin Wu, Yun Zheng, Pan Pan, and Xian-Sheng Hua.

用于跨模态检索的Token嵌入对齐。

In ACM Int. Conf. Multimedia, 2022.

  • Yang et al. (2025a)

Nianzu Yang, Pandeng Li, Liming Zhao, Yang Li, Chen-Wei Xie, Yehui Tang, Xudong Lu, Zhihang Liu, Yun Zheng, Yu Liu, and Junchi Yan.

重新思考视频分词：一种基于条件扩散的方法。

arXiv preprint arXiv:2503.03708, 2025a.

  • Yang et al. (2022)

Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang.

ManiQA：用于无参考图像质量评估的多维注意力网络。

arXiv e-prints, 2022.

  • Yang et al. (2025b)

Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Xiaotao Gu, Yuxuan Zhang, Weihan Wang, Yean Cheng, Ting Liu, Bin Xu, Yuxiao Dong, and Jie Tang.

CogVideoX：采用专家Transformer的文本到视频扩散模型。

In Int. Conf. Learn. Represent., 2025b.

  • Yao et al. (2021)

Zhuoyuan Yao, Di Wu, Xiong Wang, Binbin Zhang, Fan Yu, Chao Yang, Zhendong Peng, Xiaoyu Chen, Lei Xie, and Xin Lei.

WeNet：面向生产的流式与非流式端到端语音识别工具包。

In InterSpeech, 2021.

  • Ye et al. (2023)

Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang.

IP-Adapter：用于文本到图像扩散模型的文本兼容图像提示适配器。

arXiv preprint arXiv:2308.06721, 2023.

  • Yu et al. (2023)

Lijun Yu, José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al.

语言模型战胜扩散模型——分词器是视觉生成的关键。

arXiv preprint arXiv:2310.05737, 2023.

  • Yuan et al. (2024a)

Hangjie Yuan, Shiwei Zhang, Xiang Wang, Yujie Wei, Tao Feng, Yining Pan, Yingya Zhang, Ziwei Liu, Samuel Albanie, and Dong Ni.

InstructVideo：利用人类反馈指导视频扩散模型。

In IEEE Conf. Comput. Vis. Pattern Recog., pp. 6463–6474, 2024a.

  • Yuan et al. (2025)

Shenghai Yuan, Jinfa Huang, Xianyi He, Yunyuan Ge, Yujun Shi, Liuhan Chen, Jiebo Luo, and Li Yuan.

通过频率分解实现身份保持的文本到视频生成。

In IEEE Conf. Comput. Vis. Pattern Recog., 2025.

  • Yuan et al. (2024b)

Zhihang Yuan, Hanling Zhang, Lu Pu, Xuefei Ning, Linfeng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, and Yu Wang.

DiTFastAttn：面向扩散Transformer模型的注意力压缩。

In Adv. Neural Inform. Process. Syst., 2024b.

  • Zhai et al. (2023)

Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer.

用于语言-图像预训练的Sigmoid损失。

In Int. Conf. Comput. Vis., pp. 11975–11986, 2023.

  • Zhang & Sennrich (2019)

Biao Zhang and Rico Sennrich.

均方根层归一化。

Adv. Neural Inform. Process. Syst., 32, 2019.

  • Zhang et al. (2024)

Jintao Zhang, Haofeng Huang, Pengle Zhang, Jun Zhu, Jianfei Chen, et al.

SageAttention：精确的8位注意力，即插即用式推理加速。

arXiv preprint arXiv:2410.02367, 2024.

  • Zhang et al. (2023a)

Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su.

MagicBrush：一个用于指令引导图像编辑的人工标注数据集。

In Adv. Neural Inform. Process. Syst., 2023a.

  • Zhang et al. (2023b)

Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.

为文本到图像扩散模型添加条件控制。

In Int. Conf. Comput. Vis., pp. 3836–3847, 2023b.

  • Zhang et al. (2023c)

Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou.

I2VGen-XL：通过级联扩散模型实现高质量图像到视频合成。

arXiv preprint arXiv:2311.04145, 2023c.

  • Zhang et al. (2023d)

Youcai Zhang, Xinyu Huang, Jinyu Ma, Zhaoyang Li, Zhaochuan Luo, Yanchun Xie, Yuzhuo Qin, Tong Luo, Yaqian Li, Shilong Liu, Yandong Guo, and Lei Zhang.

Recognize Anything：一个强大的图像标注模型。

arXiv preprint arXiv:2306.03514, 2023d.

  • Zhao et al. (2023)

Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, and Shen Li.

PyTorch FSDP：关于扩展全分片数据并行的经验，2023。

  • Zheng et al. (2024)

Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You.

Open-Sora：让高效视频制作惠及所有人，2024年3月。

  • Zhou et al. (2022)

Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng.

MagicVideo：基于潜在扩散模型的高效视频生成。

arXiv preprint arXiv:2211.11018, 2022.

  • Zhou et al. (2023)

Shangchen Zhou, Chongyi Li, Kelvin C. K. Chan, and Chen Change Loy.

ProPainter：改进传播与Transformer以实现视频修复。

In Int. Conf. Comput. Vis., pp. 10477–10486, 2023.