Lune

EMNLP Findings2024

NeuroMax: Enhancing Neural Topic Modeling via Maximizing Mutual Information and Group Topic Regularization

Duy-Tung Pham, Thien Trang Nguyen Vu, Tung Nguyen, Linh Ngo Van, Duc Anh Nguyen, Thien Huu Nguyen

Abstract

Recent advances in neural topic models have concentrated on two primary directions: the integration of the inference network (encoder) with a pre-trained language model (PLM) and the modeling of the relationship between words and topics in the generative model (decoder). However, the use of large PLMs significantly increases inference costs, making them less practical for situations requiring low inference times. Furthermore, it is crucial to simultaneously model the relationships between topics and words as well as the interrelationships among topics themselves. In this work, we propose a novel framework called NeuroMax (Neural Topic Model with Maximizing Mutual Information with Pretrained Language Model and Group Topic Regularization) to address these challenges. NeuroMax maximizes the mutual information between the topic representation obtained from the encoder in neural topic models and the representation derived from the PLM. Additionally, NeuroMax employs optimal transport to learn the relationships between topics by analyzing how information is transported among them. Experimental results indicate that NeuroMax reduces inference time, generates more coherent topics and topic groups, and produces more representative document embeddings, thereby enhancing performance on downstream tasks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Constrained Non-negative Matrix Factorization for Guided Topic Modeling of Minority Topics cites Effective Neural Topic Modeling with Embedding Clustering Regularization
  • Understanding Cross-Domain Adaptation in Low-Resource Topic Modeling cites Effective Neural Topic Modeling with Embedding Clustering Regularization
  • Understanding Cross-Domain Adaptation in Low-Resource Topic Modeling cites Neural Topic Model via Optimal Transport
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Constrained Non-negative Matrix Factorization for Guided Topic Modeling of Minority TopicsSeyedeh Fatemeh Ebrahimi, Jaakko PeltonenEMNLP 2025 · 2 citations
  2. CATCH: A Controllable Theme Detection Framework with Contextualized Clustering and Hierarchical GenerationRui Ke, Jiahui Xu, Shenghao Yang, Kuang Wang et al.AAAI 2026
  3. Understanding Cross-Domain Adaptation in Low-Resource Topic ModelingPritom Saha Akash, Kevin Chen-Chuan ChangACL 2025

Builds on15

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph CompilationJason Ansel, Edward Z. Yang, Horace He, Natalia Gimelshein et al.ASPLOS 2024 · 693 citations
  3. Online Continual Learning through Mutual Information MaximizationYiduo Guo, Bing Liu, Dongyan ZhaoICML 2022 · 139 citations
  4. Neural Topic Model via Optimal TransportHe Zhao, Dinh Phung, Viet Huynh, Trung Le et al.ICLR 2021 · 100 citations
  5. Effective Neural Topic Modeling with Embedding Clustering RegularizationXiaobao Wu, Xinshuai Dong, Thong Thanh Nguyen, Anh Tuan LuuICML 2023 · 87 citations

Related papers

  1. HiCOT: Improving Neural Topic Models via Optimal Transport and Contrastive LearningHoang Tran Vuong, Tue Le, Tu Vu, Tung Nguyen et al.ACL Findings 2025 · 3 citations
  2. Neural Topic Modeling with Large Language Models in the LoopXiaohao Yang, He Zhao, Weijie Xu, Yuanyuan Qi et al.ACL 2025 · 13 citations
  3. Pre-training and Fine-tuning Neural Topic Model: A Simple yet Effective Approach to Incorporating External KnowledgeLinhai Zhang, Xuemeng Hu, Boyu Wang, Deyu Zhou et al.ACL 2022 · 14 citations
  4. Dynamic Topic Models for Temporal Document NetworksDelvin Ce Zhang, Hady W. LauwICML 2022 · 26 citations
  5. OTLDA: A Geometry-aware Optimal Transport Approach for Topic ModelingViet Huynh, He Zhao, Dinh PhungNeurIPS 2020 · 29 citations
Dusk over the sea between two cliffs drawn in fine vertical lines