Lune

ICLR2024顶会

Consistent Video-to-Video Transfer Using Synthetic Dataset

Jiaxin Cheng, Tianjun Xiao, Tong He

摘要

We introduce a novel and efficient approach for text-based video-to-video editing that eliminates the need for resource-intensive per-video-per-model finetuning. At the core of our approach is a synthetic paired video dataset tailored for video-tovideo transfer tasks. Inspired by Instruct Pix2Pix's image transfer via editing instruction, we adapt this paradigm to the video domain. Extending the Promptto-Prompt to videos, we efficiently generate paired samples, each with an input video and its edited counterpart. Alongside this, we introduce the Long Video Sampling Correction during sampling, ensuring consistent long videos across batches. Our method surpasses current methods like Tune-A-Video, heralding substantial progress in text-based video-to-video editing and suggesting exciting avenues for further exploration and deployment. https://github.com/ amazon-science/instruct-video-to-video/tree/main Make the car red Porsche and drive alone beach (Editing type: Multiple) Make it watercolor (Editing type: Style) Make it snowy (Editing type: Background) Original video

问 Lune

几分钟读透这篇 Paper,不用再花一下午。

Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。

已有 27 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。

可以从这些问题问起

Lune 可以查阅

全文 · 引用它的顶会 Paper · 相关的 Paper

获取答案

免费开始,无需绑卡

脉络

  • High-Resolution Image Synthesis with Latent Diffusion Models 引用了 Diffusion Models Beat GANs on Image Synthesis
  • High-Resolution Image Synthesis with Latent Diffusion Models 引用了 Denoising Diffusion Probabilistic Models
  • High-Resolution Image Synthesis with Latent Diffusion Models 引用了 Denoising Diffusion Implicit Models
  • MiniMax-Remover: Taming Bad Noise Helps Video Object Removal 引用了 ReVideo: Remake a Video with Motion and Content Control
  • ReVideo: Remake a Video with Motion and Content Control 引用了 High-Resolution Image Synthesis with Latent Diffusion Models
  • ReVideo: Remake a Video with Motion and Content Control 引用了 Diffusion Models Beat GANs on Image Synthesis
  • ReVideo: Remake a Video with Motion and Content Control 引用了 Learning Transferable Visual Models From Natural Language Supervision
  • ReVideo: Remake a Video with Motion and Content Control 引用了 Denoising Diffusion Probabilistic Models
  • Diffusion Models Beat GANs on Image Synthesis 引用了 Learning Transferable Visual Models From Natural Language Supervision
  • Diffusion Models Beat GANs on Image Synthesis 引用了 Denoising Diffusion Probabilistic Models
  • Diffusion Models Beat GANs on Image Synthesis 引用了 Denoising Diffusion Implicit Models
  • IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment 引用了 Learning Transferable Visual Models From Natural Language Supervision
  • IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment 引用了 Denoising Diffusion Implicit Models
  • IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment 引用了 Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset
  • Denoising Diffusion Implicit Models 引用了 Denoising Diffusion Probabilistic Models
  • EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning 引用了 Learning Transferable Visual Models From Natural Language Supervision
  • EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning 引用了 Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset
  • Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset 引用了 EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning
  • Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset 引用了 High-Resolution Image Synthesis with Latent Diffusion Models
它引用的引用它的从左到右按年份排列,引用越多,位置越高。

引用它的顶会 Paper27

  1. ReVideo: Remake a Video with Motion and Content ControlChong Mou, Mingdeng Cao, Xintao Wang, Zhaoyang Zhang 等NeurIPS 2024 · 87 次引用
  2. Scaling Instruction-Based Video Editing with a High-Quality Synthetic DatasetQingyan Bai, Qiuyu Wang, Hao Ouyang, Yue Yu 等CVPR 2026 · 79 次引用
  3. EditVerse: Unifying Image and Video Editing and Generation with In-Context LearningXuan Ju, Tianyu Wang, Yuqian Zhou, He Zhang 等ICLR 2026 · 56 次引用
  4. MiniMax-Remover: Taming Bad Noise Helps Video Object RemovalBojia Zi, Weixuan Peng, Xianbiao Qi, Jianan Wang 等NeurIPS 2025 · 43 次引用
  5. IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing AssessmentYinan Chen, Jiangning Zhang, Teng Hu, Yuxiang Zeng 等ICLR 2026 · 29 次引用

它引用的 Paper39

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 47,906 次引用
  2. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 次引用
  3. Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 次引用
  4. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 13,123 次引用
  5. Denoising Diffusion Implicit ModelsJiaming Song, Chenlin Meng, Stefano ErmonICLR 2021 · 11,743 次引用

相关 Paper

  1. EasyV2V: A High-quality Instruction-based Video Editing FrameworkJinjie Mai, Chaoyang Wang, Gordon Guocheng Qian, Willi Menapace 等CVPR 2026 · 12 次引用
  2. InsViE-1M: Effective Instruction-Based Video Editing with Elaborate Dataset ConstructionYuhui Wu, Liyi Chen, Ruibin Li, Shihao Wang 等ICCV 2025 · 6 次引用
  3. Distilling Vision-Language Models on Millions of VideosYue Zhao, Long Zhao, Xingyi Zhou, Jialin Wu 等CVPR 2024
  4. Temporal In‑Context Fine‑Tuning with Temporal Reasoning for Versatile Control of Video Diffusion ModelsKinam Kim, Junha Hyung, Jaegul ChooNeurIPS 2025 · 4 次引用
  5. Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video GeneratorsLevon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel 等ICCV 2023 · 800 次引用
黄昏的海面,两侧是细线勾勒的悬崖