Lune

EMNLP Findings2024

Unsupervised Hierarchical Topic Modeling via Anchor Word Clustering and Path Guidance

Jiyuan Liu, Hegang Chen, Chunjiang Zhu, Yanghui Rao

Abstract

Hierarchical topic models nowadays tend to capture the relationship between words and topics, often ignoring the role of anchor words that guide text generation. For the first time, we detect and add anchor words to the text generation process in an unsupervised way. Firstly, we adopt a clustering algorithm to adaptively detect anchor words that are highly consistent with every topic, which forms the path of topic→anchor word. Secondly, we add the causal path of anchor word→word to the popular Variational Auto-Encoder (VAE) framework via implicitly using word co-occurrence graphs. We develop the causal path of topic+anchor word→higher-layer topic that aids the expression of topic concepts with anchor words to capture a more semantically tight hierarchical topic structure. Finally, we enhance the model's representation of the anchor words through a novel contrastive learning. After jointly training the aforementioned constraint objectives, we can produce more coherent and diverse topics with a better hierarchical structure. Extensive experiments on three datasets show that our model outperforms state-of-the-art methods.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding cites Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Constrained Non-negative Matrix Factorization for Guided Topic Modeling of Minority TopicsSeyedeh Fatemeh Ebrahimi, Jaakko PeltonenEMNLP 2025 · 2 citations

Builds on10

  1. Contrastive Learning for Neural Topic ModelThong Nguyen, Anh Tuan LuuNeurIPS 2021 · 82 citations
  2. Topic Discovery via Latent Space Clustering of Pretrained Language Model RepresentationsYu Meng, Yunyi Zhang, Jiaxin Huang, Yu Zhang et al.WWW 2022 · 73 citations
  3. Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling DecoderXiaobao Wu, Chunping Li, Yan Zhu, Yishu MiaoEMNLP 2020 · 61 citations
  4. Sawtooth Factorial Topic Embeddings Guided Gamma Belief NetworkZhibin Duan, Dongsheng Wang, Bo Chen, Chaojie Wang et al.ICML 2021 · 49 citations
  5. HyperMiner: Topic Taxonomy Mining with Hyperbolic EmbeddingYishi Xu, Dongsheng Wang, Bo Chen, Ruiying Lu et al.NeurIPS 2022 · 38 citations

Related papers

  1. Document Hashing with Multi-Grained Prototype-Induced Hierarchical Generative ModelQian Zhang, Qinliang Su, Jiayang Chen, Zhenpeng SongEMNLP Findings 2024 · 2 citations
  2. Neural Topic Modeling based on Cycle Adversarial Training and Contrastive LearningBoyu Wang, Linhai Zhang, Deyu Zhou, Yi Cao et al.ACL Findings 2023 · 4 citations
  3. Variational Graph Author Topic ModelingDelvin Ce Zhang, Hady Wirawan LauwKDD 2022 · 13 citations
  4. Neural Topic Modeling via Contextual and Graph Information FusionJiyuan Liu, Jiaxing Yan, Chunjiang Zhu, Xingyu Liu et al.EMNLP 2025
  5. TopicNet: Semantic Graph-Guided Topic DiscoveryZhibin Duan, Yishi Xu, Bo Chen, Dongsheng Wang et al.NeurIPS 2021 · 19 citations
Dusk over the sea between two cliffs drawn in fine vertical lines