Light-X: Generative 4D Video Rendering with Camera and Illumination Control
Tianqi Liu, Zhaoxi Chen, Zihao Huang, Shaocong Xu, Saining Zhang, Chongjie Ye, Bohan Li, Zhiguo Cao, Wei Li, Hao Zhao, Ziwei Liu
摘要
Recent advances in illumination control extend image-based methods to video, yet still facing a trade-off between lighting fidelity and temporal consistency. Moving beyond relighting, a key step toward generative modeling of real-world scenes is the joint control of camera trajectory and illumination, since visual dynamics are inherently shaped by both geometry and lighting. To this end, we present Light-X, a video generation framework that enables controllable rendering from monocular videos with both viewpoint and illumination control. 1) We propose a disentangled design that decouples geometry and lighting signals: geometry and motion are captured via dynamic point clouds projected along user-defined camera trajectories, while illumination cues are provided by a relit frame consistently projected into the same geometry. These explicit, fine-grained cues enable effective disentanglement and guide high-quality illumination. 2) To address the lack of paired multi-view and multi-illumination videos, we introduce Light-Syn, a degradation-based pipeline with inverse-mapping that synthesizes training pairs from in-the-wild monocular footage. This strategy yields a dataset covering static, dynamic, and AI-generated scenes, ensuring robust training. Extensive experiments show that Light-X outperforms baseline methods in joint camera-illumination control and surpasses prior video relighting methods under both text- and background-conditioned settings.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper3
- NeoVerse: Enhancing 4D World Model with in-the-wild Monocular VideosYuxue Yang, Lue Fan, Ziqi Shi, Junran Peng 等CVPR 2026 · 被引用 42 次
- FreeOrbit4D: Training-Free Arbitrary Camera Redirection for Monocular Videos via Foreground-Complete 4D ReconstructionWei Cao, Hao Zhang, Fengrui Tian, Yulun Wu 等SIGGRAPH 2026 · 被引用 4 次
- Relit-LiVE: Relight Video by Jointly Learning Environment VideoWeiqing Xiao, Hong Li, Xiuyu Yang, Houyuan Chen 等SIGGRAPH 2026
它引用的顶会 Paper51
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 被引用 13,123 次
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 被引用 11,349 次
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 被引用 7,873 次
相关 Paper
- LightCtrl: Training-free Controllable Video RelightingYizuo Peng, Xuelin Chen, Kai Zhang, Xiaodong CunICLR 2026 · 被引用 3 次
- IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video GenerationYuanze Lin, Yi-Wen Chen, Yi-Hsuan Tsai, Ronald Clark 等NeurIPS 2025 · 被引用 6 次
- TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion ModelsMark Yu, Wenbo Hu, Jinbo Xing, Ying ShanICCV 2025 · 被引用 25 次
- Lighting-grounded Video Generation with Renderer-based Agent ReasoningZiqi Cai, Taoyu Yang, Zheng Chang, Si Li 等CVPR 2026 · 被引用 3 次
- Video Camera Trajectory Editing with Generative Rendering from Estimated GeometryJunyoung Seo, Jisang Han, Jaewoo Jung, Siyoon Jin 等AAAI 2026
