EasyVFX: Frequency-Driven Decoupling for Resource-Efficient VFX Generation
Yue Ma, Xu Ye, Qinghe Wang, Yucheng Wang, Hongyu Liu, Yinhan Zhang, Xinyu Wang, Yuanpeng Chen, Shanhui Mo, Paul Liang, Fangneng Zhan, Qifeng Chen
摘要
Generating high-fidelity visual effects (VFX) typically demands massive datasets and prohibitive computational power due to the intricate coupling of spatial textures and temporal dynamics. In this paper, we introduce EasyVFX, a resource-efficient framework that achieves realistic VFX synthesis under stringent constraints. Our core philosophy lies in frequency-domain decomposition: we observe that the complexity of VFX can be significantly mitigated by decoupling high-frequency components, which represent intricate spatial appearances, from low-frequency components that encapsulate global motion dynamics. This spectral disentanglement transforms a high-dimensional learning problem into manageable sub-tasks, thereby lowering the optimization barrier and reducing data dependency. Building upon this insight, we propose a two-stage training paradigm. First, we design a Frequency-aware Mixture-of-Experts (Freq-MoE) architecture. By utilizing a soft routing mechanism, our model assigns specialized experts to distinct spectral bands, enabling them to cultivate robust priors for appearance and motion dynamics. This specialization allows the model to acquire foundational VFX knowledge with fewer GPU resources. Second, we introduce a Test-Time Training strategy powered by a novel Frequency-constraint Loss. This allows the pre-trained model to swiftly adapt to specific, unseen effects through localized optimizations, requiring only about 100 steps on a single GPU. Experimental results demonstrate that EasyVFX produces structurally consistent and visually stunning effects, proving that frequency-aware learning is a key catalyst for democratizing professional-grade VFX.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper48
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 被引用 18,833 次
- Follow Your Pose: Pose-Guided Text-to-Video Generation Using Pose-Free VideosYue Ma, Yingqing He, Xiaodong Cun, Xintao Wang 等AAAI 2024 · 被引用 318 次
- TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting MethodsXiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu 等VLDB 2024 · 被引用 292 次
- MagicAnimate: Temporally Consistent Human Image Animation using Diffusion ModelZhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan 等CVPR 2024 · 被引用 106 次
相关 Paper
- Omni-Effects: Unified and Spatially-Controllable Visual Effects GenerationFangyuan Mao, Aiming Hao, Jintao Chen, Dongxia Liu 等AAAI 2026 · 被引用 20 次
- VFXMaster: Unlocking Dynamic Visual Effect Generation via In-Context LearningBaolu Li, Yiming Zhang, Qinghe Wang, Liqian Ma 等SIGGRAPH 2026
- EffectMaker: Unifying Reasoning and Generation for Customized Visual Effect CreationShiyuan Yang, Ruihuang Li, Jiale Tao, Shuai Shao 等CVPR 2026 · 被引用 2 次
- Deep Motion Warping via Phase-Conditioned Diffusion AutoencoderBowen Zheng, Linjun Wu, Xinwei Jiang, Yujin Chai 等SIGGRAPH 2026
- Spectral Mixture-of-Experts for Continual LearningChen Yin, Xingbo Dong, Xuelin Shen, Zhe JinCVPR 2026
