BindWeave: Subject-Consistent Video Generation via Cross-Modal Integration
Zhaoyang Li, Dongjun Qian, Kai Su, qishuai diao, Xiangyang Xia, Chang Liu, Wenfei Yang, Tianzhu Zhang, Zehuan Yuan
Abstract
Diffusion Transformer has shown remarkable abilities in generating high-fidelity videos, delivering visually coherent frames and rich details over extended durations. However, existing video generation models still fall short in subject-consistent video generation due to an inherent difficulty in parsing prompts that specify complex spatial relationships, temporal logic, and interactions among multiple subjects. To address this issue, we propose BindWeave, a unified framework that handles a broad range of subject-to-video scenarios from single-subject cases to complex multi-subject scenes with heterogeneous entities. To bind complex prompt semantics to concrete visual subjects, we introduce an MLLM-DiT framework in which a pretrained multimodal large language model performs deep cross-modal reasoning to ground entities and disentangle roles, attributes, and interactions, yielding subject-aware hidden states that condition the diffusion transformer for high-fidelity subject-consistent video generation. Experiments on the OpenS2V benchmark demonstrate that our method achieves superior performance across subject consistency, naturalness, and text relevance in generated videos, outperforming existing open-source and commercial models.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers4
- DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video GenerationXu Guo, Fulong Ye, Qichao Sun, Liyang Chen et al.ICML 2026 · 16 citations
- Scaling Zero-Shot Reference-to-Video GenerationZijian Zhou, Shikun Liu, Haozhe Liu, Haonan Qiu et al.CVPR 2026 · 10 citations
- AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video GenerationSharath Girish, Viacheslav Ivanov, Tsai-Shien Chen, Hao Chen et al.CVPR 2026 · 4 citations
- MiVE: Multiscale Vision-language features for reference-guided video EditingTong Wang, Meng Zou, WU CHENGJING, Xiaochao Qu et al.ICML 2026
Builds on18
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Scalable Diffusion Models with TransformersWilliam Peebles, Saining XieICCV 2023 · 5,568 citations
- Scaling Rectified Flow Transformers for High-Resolution Image SynthesisPatrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari et al.ICML 2024 · 3,620 citations
Related papers
- UniVideo: Unified Understanding, Generation, and Editing for VideosCong Wei, Quande Liu, Zixuan Ye, Qiulin Wang et al.ICLR 2026 · 90 citations
- Composing Concepts from Images and Videos via Concept-prompt BindingXianghao Kong, Zeyu Zhang, Yuwei Guo, Zhuoran Zhao et al.CVPR 2026 · 2 citations
- LumosX: Relate Any Identities with Their Attributes for Personalized Video GenerationJiazheng Xing, Fei Du, Hangjie Yuan, Pengwei Liu et al.ICLR 2026 · 6 citations
- Modular-Cam: Modular Dynamic Camera-view Video Generation with LLMZirui Pan, Xin Wang, Yipeng Zhang, Hong Chen et al.AAAI 2025 · 6 citations
- Compositional 3D-aware Video Generation with LLM DirectorHanxin Zhu, Tianyu He, Anni Tang, Junliang Guo et al.NeurIPS 2024 · 19 citations
