Lune

ACL Findings2022

Controlled Text Generation Using Dictionary Prior in Variational Autoencoders

Xianghong Fang, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Dit-Yan Yeung

Abstract

While variational autoencoders (VAEs) have been widely applied in text generation tasks, they are troubled by two challenges: insufficient representation capacity and poor controllability. The former results from the posterior collapse and restrictive assumption, which impede better representation learning. The latter arises as continuous latent variables in traditional formulations hinder VAEs from interpretability and controllability. In this paper, we propose Dictionary Prior (DPrior), a new data-driven prior that enjoys the merits of expressivity and controllability. To facilitate controlled text generation with DPrior, we propose to employ contrastive learning to separate the latent space into several parts. Extensive experiments on both language modeling and controlled text generation demonstrate the effectiveness of the proposed approach.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Enhancing Explainable Rating Prediction through Annotated Macro ConceptsHuachi Zhou, Shuang Zhou, Hao Chen, Ninghao Liu et al.ACL 2024
  2. Learning Disentangled Semantic Spaces of Explanations via Invertible Neural NetworksYingji Zhang, Danilo S. Carvalho, André FreitasACL 2024

Builds on8

  1. Plug and Play Language Models: A Simple Approach to Controlled Text GenerationSumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung et al.ICLR 2020 · 1,166 citations
  2. Optimus: Organizing Sentences via Pre-trained Modeling of a Latent SpaceChunyuan Li, Xiang Gao, Yuan Li, Baolin Peng et al.EMNLP 2020 · 132 citations
  3. MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent SpaceXiaoyuan Yi, Ruoyu Li, Cheng Yang, Wenhao Li et al.AAAI 2020 · 41 citations
  4. Guiding Variational Response Generator to Exploit PersonaBowen Wu, Mengyuan Li, Zongsheng Wang, Yifu Chen et al.ACL 2020 · 40 citations
  5. Dispersed Exponential Family Mixture VAEs for Interpretable Text GenerationWenxian Shi, Hao Zhou, Ning Miao, Lei LiICML 2020 · 30 citations

Related papers

  1. Empirical Prior for Text AutoencodersYongjing Yin, Wenyang Gao, Haodong Wu, Jianhao Yan et al.EMNLP Findings 2024
  2. Contrastive Deterministic Autoencoders For Language ModelingAmur Ghose, Pascal PoupartEMNLP Findings 2023 · 1 citation
  3. Variational Autoencoder with Disentanglement Priors for Low-Resource Task-Specific Natural Language GenerationZhuang Li, Lizhen Qu, Qiongkai Xu, Tongtong Wu et al.EMNLP 2022
  4. Dior-CVAE: Pre-trained Language Models and Diffusion Priors for Variational Dialog GenerationTianyu Yang, Thy Thy Tran, Iryna GurevychEMNLP Findings 2023 · 2 citations
  5. On Variational Learning of Controllable Representations for Text without SupervisionPeng Xu, Jackie Chi Kit Cheung, Yanshuai CaoICML 2020 · 69 citations
Dusk over the sea between two cliffs drawn in fine vertical lines