Lune

ACL Findings2023

Reinforcement Learning for Topic Models

Jeremy Costello, Marek Z. Reformat

Abstract

We apply reinforcement learning techniques to topic modeling by replacing the variational autoencoder in ProdLDA with a continuous action space reinforcement learning policy. We train the system with a policy gradient algorithm RE-INFORCE. Additionally, we introduced several modifications: modernize the neural network architecture, weight the ELBO loss, use contextual embeddings, and monitor the learning process via computing topic diversity and coherence for each training step. Experiments are performed on 11 data sets. Our unsupervised model outperforms all other unsupervised models and performs on par with or better than most models using supervised labeling. Our model is outperformed on certain data sets by a model using supervised labeling and contrastive learning. We have also conducted an ablation study to provide empirical evidence of performance improvements from changes we made to ProdLDA and found that the reinforcement learning formulation boosts performance. We open-source our code implementation 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on3

  1. A ConvNet for the 2020sZhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer et al.CVPR 2022 · 6,782 citations
  2. Contrastive Learning for Neural Topic ModelThong Nguyen, Anh Tuan LuuNeurIPS 2021 · 82 citations
  3. Benchmarking Neural Topic Models: An Empirical StudyThanh-Nam Doan, Tuan-Anh HoangACL Findings 2021 · 27 citations

Related papers

  1. A Discrete Variational Recurrent Topic Model without the Reparametrization TrickMehdi Rezaee, Francis FerraroNeurIPS 2020 · 31 citations
  2. Improving Multi-party Dialogue Generation via Topic and Rhetorical CoherenceYaxin Fan, Peifeng Li, Qiaoming ZhuEMNLP 2024 · 1 citation
  3. Understanding The Robustness of Self-supervised Learning Through Topic ModelingZeping Luo, Shiyou Wu, Cindy Weng, Mo Zhou et al.ICLR 2023
  4. Neural Topic Modeling based on Cycle Adversarial Training and Contrastive LearningBoyu Wang, Linhai Zhang, Deyu Zhou, Yi Cao et al.ACL Findings 2023 · 4 citations
  5. Unsupervised Paraphrasing via Deep Reinforcement LearningA. B. Siddique, Samet Oymak, Vagelis HristidisKDD 2020 · 27 citations
Dusk over the sea between two cliffs drawn in fine vertical lines