VIAFormer: Voxel-Image Alignment Transformer for High-Fidelity Voxel Refinement
Tiancheng Fang, Bowen Pan, Lingxi Chen, Jiangjing Lyu, Chengfei Lv, Chaoyue Niu, Fan Wu
Abstract
We propose VIAFormer, a Voxel-Image Alignment Transformer model designed for Multi-view Conditioned Voxel Refinement--the task of repairing incomplete noisy voxels using calibrated multi-view images as guidance. Its effectiveness stems from a synergistic design: an Image Index that provides explicit 3D spatial grounding for 2D image tokens, a Correctional Flow objective that learns a direct voxel-refinement trajectory, and a Hybrid Stream Transformer that enables robust cross-modal fusion. Experiments show that VIAFormer establishes a new state of the art in correcting both severe synthetic corruptions and realistic artifacts on the voxel shape obtained from powerful Vision Foundation Models. Beyond benchmarking, we demonstrate VIAFormer as a practical and reliable bridge in real-world 3D creation pipelines, paving the way for voxel-based methods to thrive in large-model, big-data wave.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on29
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations
- Directly Denoising Diffusion ModelsDan Zhang, Jingjing Wang, Feng LuoICML 2024 · 11,724 citations
- Adding Conditional Control to Text-to-Image Diffusion ModelsLvmin Zhang, Anyi Rao, Maneesh AgrawalaICCV 2023 · 6,759 citations
- ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score DistillationZhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao et al.NeurIPS 2023 · 1,498 citations
Related papers
- ShapeFormer: Transformer-based Shape Completion via Sparse RepresentationXingguang Yan, Liqiang Lin, Niloy J. Mitra, Dani Lischinski et al.CVPR 2022 · 124 citations
- VoxFormer: Sparse Voxel Transformer for Camera-Based 3D Semantic Scene CompletionYiming Li, Zhiding Yu, Christopher B. Choy, Chaowei Xiao et al.CVPR 2023
- SivsFormer: Parallax-Aware Transformers for Single-image-based View SynthesisChunlan Zhang, Chunyu Lin, Kang Liao, Lang Nie et al.IEEE VR 2022 · 3 citations
- CVSformer: Cross-View Synthesis Transformer for Semantic Scene CompletionHaotian Dong, Enhui Ma, Lubo Wang, Miaohui Wang et al.ICCV 2023 · 12 citations
- SyFormer: Structure-Guided Synergism Transformer for Large-Portion Image InpaintingJie Wu, Yuchao Feng, Honghui Xu, Chuanmeng Zhu et al.AAAI 2024 · 14 citations
