Lune

EMNLP Findings2023

DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM

Weijie Xu, Wenxiang Hu, Fanyou Wu, Srinivasan H. Sengamedu

Abstract

In the burgeoning field of natural language processing, Neural Topic Models (NTMs) and Large Language Models (LLMs) have emerged as areas of significant research interest. Despite this, NTMs primarily utilize contextual embeddings from LLMs, which are not optimal for clustering or capable for topic generation. Our study addresses this gap by introducing a novel framework named Diffusion-Enhanced Topic Modeling using Encoder-Decoder-based LLMs (DeTiME). DeTiME leverages Encoder-Decoder-based LLMs to produce highly clusterable embeddings that could generate topics that exhibit both superior clusterability and enhanced semantic coherence compared to existing methods. Additionally, by exploiting the power of diffusion, our framework also provides the capability to generate content relevant to the identified topics. This dual functionality allows users to efficiently produce highly clustered topics and related content simultaneously. DeTiME's potential extends to generating clustered embeddings as well. Notably, our proposed framework proves to be efficient to train and exhibits high adaptability, demonstrating its potential for a wide array of applications.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • High-Resolution Image Synthesis with Latent Diffusion Models cites Denoising Diffusion Probabilistic Models
  • High-Resolution Image Synthesis with Latent Diffusion Models cites Denoising Diffusion Implicit Models
  • Neural Topic Modeling with Large Language Models in the Loop cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Denoising Diffusion Implicit Models cites Denoising Diffusion Probabilistic Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Neural Topic Modeling with Large Language Models in the LoopXiaohao Yang, He Zhao, Weijie Xu, Yuanyuan Qi et al.ACL 2025 · 13 citations
  2. Evaluating Cognitive-Behavioral Fixation via Multimodal User Viewing Patterns on Social MediaYujie Wang, Yunwei Zhao, Jing Yang, Han Han et al.EMNLP 2025 · 1 citation
  3. Understanding Cross-Domain Adaptation in Low-Resource Topic ModelingPritom Saha Akash, Kevin Chen-Chuan ChangACL 2025

Builds on30

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
  3. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  4. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
  5. Denoising Diffusion Implicit ModelsJiaming Song, Chenlin Meng, Stefano ErmonICLR 2021 · 11,743 citations

Related papers

  1. NeuroMax: Enhancing Neural Topic Modeling via Maximizing Mutual Information and Group Topic RegularizationDuy-Tung Pham, Thien Trang Nguyen Vu, Tung Nguyen, Linh Ngo Van et al.EMNLP Findings 2024 · 7 citations
  2. HiCOT: Improving Neural Topic Models via Optimal Transport and Contrastive LearningHoang Tran Vuong, Tue Le, Tu Vu, Tung Nguyen et al.ACL Findings 2025 · 3 citations
  3. From Moments to Milestones: Incremental Timeline Summarization Leveraging Large Language ModelsQisheng Hu, Geonsik Moon, Hwee Tou NgACL 2024 · 8 citations
  4. Self-Supervised Neural Topic ModelingSeyed Ali Bahrainian, Martin Jaggi, Carsten EickhoffEMNLP Findings 2021 · 9 citations
  5. LLM-Guided Semantic-Aware Clustering for Topic ModelingJianghan Liu, Ziyu Shang, Wenjun Ke, Peng Wang et al.ACL 2025 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines