Lune

ACL Findings2024

Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion

Xiaobao Wu, Xinshuai Dong, Liangming Pan, Thong Nguyen, Anh Tuan Luu

Abstract

Dynamic topic models track the evolution of topics in sequential documents, which have derived various applications like trend analysis and opinion mining. However, existing models suffer from repetitive topic and unassociated topic issues, failing to reveal the evolution and hindering further applications. To address these issues, we break the tradition of simply chaining topics in existing work and propose a novel neural Chain-Free Dynamic Topic Model. We introduce a new evolution-tracking contrastive learning method that builds the similarity relations among dynamic topics. This not only tracks topic evolution but also maintains topic diversity, mitigating the repetitive topic issue. To avoid unassociated topics, we further present an unassociated word exclusion method that consistently excludes unassociated words from discovered topics. Extensive experiments demonstrate our model significantly outperforms state-of-the-art baselines, tracking topic evolution with high-quality topics, showing better performance on downstream tasks, and remaining robust to the hyperparameter for evolution intensities. Our code is available at https://github.com/bobxwu/CFDTM.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

8 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • DetCo: Unsupervised Contrastive Learning for Object Detection cites A Simple Framework for Contrastive Learning of Visual Representations
  • FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model cites Effective Neural Topic Modeling with Embedding Clustering Regularization
  • Contrastive Learning for Label Efficient Semantic Segmentation cites DetCo: Unsupervised Contrastive Learning for Object Detection
  • Contrastive Learning for Label Efficient Semantic Segmentation cites A Simple Framework for Contrastive Learning of Visual Representations
  • SimCSE: Simple Contrastive Learning of Sentence Embeddings cites A Simple Framework for Contrastive Learning of Visual Representations
  • AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge cites FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model
  • AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge cites Are LLMs Good Zero-Shot Fallacy Classifiers?
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers8

  1. FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic ModelXiaobao Wu, Thong Nguyen, Delvin Zhang, William Yang Wang et al.NeurIPS 2024 · 67 citations
  2. AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World KnowledgeXiaobao Wu, Liangming Pan, Yuxi Xie, Ruiwen Zhou et al.ACL 2025 · 35 citations
  3. Motion-aware Contrastive Learning for Temporal Panoptic Scene Graph GenerationThong Thanh Nguyen, Xiaobao Wu, Yi Bin, Cong-Duy T. Nguyen et al.AAAI 2025 · 8 citations
  4. Multi-Scale Contrastive Learning for Video Temporal GroundingThong Thanh Nguyen, Yi Bin, Xiaobao Wu, Zhiyuan Hu et al.AAAI 2025 · 7 citations
  5. Are LLMs Good Zero-Shot Fallacy Classifiers?Fengjun Pan, Xiaobao Wu, Zongrui Li, Anh Tuan LuuEMNLP 2024 · 7 citations

Builds on19

  1. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  2. SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
  3. DetCo: Unsupervised Contrastive Learning for Object DetectionEnze Xie, Jian Ding, Wenhai Wang, Xiaohang Zhan et al.ICCV 2021 · 364 citations
  4. Contrastive Learning for Label Efficient Semantic SegmentationXiangyun Zhao, Raviteja Vemulapalli, Philip Andrew Mansfield, Boqing Gong et al.ICCV 2021 · 200 citations
  5. Effective Neural Topic Modeling with Embedding Clustering RegularizationXiaobao Wu, Xinshuai Dong, Thong Thanh Nguyen, Anh Tuan LuuICML 2023 · 87 citations

Related papers

  1. Neural Dynamic Focused Topic ModelKostadin Cvejoski, Ramsés J. Sánchez, César OjedaAAAI 2023 · 9 citations
  2. Neural Topic Modeling based on Cycle Adversarial Training and Contrastive LearningBoyu Wang, Linhai Zhang, Deyu Zhou, Yi Cao et al.ACL Findings 2023 · 4 citations
  3. Dynamic Topic Models for Temporal Document NetworksDelvin Ce Zhang, Hady W. LauwICML 2022 · 26 citations
  4. Enhancing Topic Interpretability for Neural Topic Modeling Through Topic-Wise Contrastive LearningXin Gao, Yang Lin, Ruiqing Li, Yasha Wang et al.ICDE 2024 · 3 citations
  5. Dynamic Online Conversation RecommendationXingshan Zeng, Jing Li, Lu Wang, Zhiming Mao et al.ACL 2020 · 10 citations
Dusk over the sea between two cliffs drawn in fine vertical lines