Lune

NeurIPS2025Top-tier venue

Towards General Modality Translation with Contrastive and Predictive Latent Diffusion Bridge

Nimrod Berman, Omkar Joglekar, Eitan Kosman, Dotan Di Castro, Omri Azencot

Abstract

Recent advances in generative modeling have positioned diffusion models as stateof-the-art tools for sampling from complex data distributions. While these models have shown remarkable success across single-modality domains such as images and audio, extending their capabilities to Modality Translation (MT), translating information across different sensory modalities, remains an open challenge. Existing approaches often rely on restrictive assumptions, including shared dimensionality, Gaussian source priors, and modality-specific architectures, which limit their generality and theoretical grounding. In this work, we propose the Latent Denoising Diffusion Bridge Model (LDDBM), a general-purpose framework for modality translation based on a latent-variable extension of Denoising Diffusion Bridge Models. By operating in a shared latent space, our method learns a bridge between arbitrary modalities without requiring aligned dimensions. We introduce a contrastive alignment loss to enforce semantic consistency between paired samples and design a domain-agnostic encoder-decoder architecture tailored for noise prediction in latent space. Additionally, we propose a predictive loss to guide training toward accurate cross-domain translation and explore several training strategies to improve stability. Our approach supports arbitrary modality pairs and performs strongly on diverse MT tasks, including multi-view to 3D shape generation, image super-resolution, and multi-view scene synthesis. Comprehensive experiments and ablations validate the effectiveness of our framework, establishing a new strong baseline in general modality translation. For more information, see our project page: https://sites.google.com/view/lddbm/home.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • A Diffusion Model for Regular Time Series Generation from Irregular Data with Completion and Masking cites Diffusion Models Beat GANs on Image Synthesis
  • A Diffusion Model for Regular Time Series Generation from Irregular Data with Completion and Masking cites Denoising Diffusion Probabilistic Models
  • Structured Diffusion Bridges: Inductive Bias for Denoising Diffusion Bridges cites Denoising Diffusion Probabilistic Models
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale cites A Simple Framework for Contrastive Learning of Visual Representations
  • Diffusion Models Beat GANs on Image Synthesis cites Learning Transferable Visual Models From Natural Language Supervision
  • Diffusion Models Beat GANs on Image Synthesis cites Denoising Diffusion Probabilistic Models
  • Learning Transferable Visual Models From Natural Language Supervision cites A Simple Framework for Contrastive Learning of Visual Representations
  • Learning Transferable Visual Models From Natural Language Supervision cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • DiffSDA: Unsupervised Diffusion Sequential Disentanglement Across Modalities cites Diffusion Models Beat GANs on Image Synthesis
  • DiffSDA: Unsupervised Diffusion Sequential Disentanglement Across Modalities cites Denoising Diffusion Probabilistic Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. A Diffusion Model for Regular Time Series Generation from Irregular Data with Completion and MaskingGal Fadlon, Idan Arbiv, Nimrod Berman, Omri AzencotNeurIPS 2025 · 12 citations
  2. DiffSDA: Unsupervised Diffusion Sequential Disentanglement Across ModalitiesHedi Zisling, Ilan Naiman, Nimrod Berman, Supasorn Suwajanakorn et al.ICLR 2026 · 2 citations
  3. Structured Diffusion Bridges: Inductive Bias for Denoising Diffusion BridgesEitan Kosman, Gabriele Serussi, Chaim BaskinICML 2026

Builds on37

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
  3. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  4. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  5. Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations

Related papers

  1. Denoising Diffusion Bridge ModelsLinqi Zhou, Aaron Lou, Samar Khanna, Stefano ErmonICLR 2024 · 163 citations
  2. SynthRGB-T: Language-Vision Guided Image Translation for Diversity SynthesisJiangang Ding, Yiquan Du, Pengxiang Li, Lili Pei et al.CVPR 2026 · 1 citation
  3. Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent RepresentationZibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng et al.NeurIPS 2023 · 279 citations
  4. Rethinking Diffusion Bridge Model with Dual Alignments for Medical Image SynthesisJinbao Wei, Yuhang Chen, Zhijie Wang, Gang Yang et al.ACM MM 2025 · 3 citations
  5. Diffusion Bridge Implicit ModelsKaiwen Zheng, Guande He, Jianfei Chen, Fan Bao et al.ICLR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines