Lune

AAAI2020Top-tier venue

Recurrent Nested Model for Sequence Generation

Wenhao Jiang, Lin Ma, Wei Lu

Abstract

Depth has been shown beneficial to neural network models. In this paper, we make an attempt to make the encoder-decoder model deeper for sequence generation. We propose a module that can be plugged into the middle between the encoder and decoder to increase the depth of the whole model. The proposed module follows a nested structure, which is divided into blocks with each block containing several recurrent transition steps. To reduce the training difficulty and preserve the necessary information for the decoder during transitions, inter-block connections and intra-block connections are constructed in our model. The inter-block connections provide the thought vectors from the current block to all the subsequent blocks. The intra-block connections connect all the hidden states entering the current block to the current transition step. The advantages of our model are illustrated on the image captioning and code captioning tasks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on1

  1. Attention on Attention for Image CaptioningLun Huang, Wenmin Wang, Jie Chen, Xiaoyong WeiICCV 2019 · 992 citations

Related papers

  1. Nested Diffusion Models Using Hierarchical Latent PriorsXiao Zhang, Ruoxi Jiang, Rebecca Willett, Michael MaireCVPR 2025
  2. Recurrent Hierarchical Topic-Guided RNN for Language GenerationDandan Guo, Bo Chen, Ruiying Lu, Mingyuan ZhouICML 2020 · 20 citations
  3. HiTRANS: A Hierarchical Transformer Network for Nested Named Entity RecognitionZhiwei Yang, Jing Ma, Hechang Chen, Yunke Zhang et al.EMNLP Findings 2021 · 15 citations
  4. What Works and Doesn't Work, A Deep Decoder for Neural Machine TranslationZuchao Li, Yiran Wang, Masao Utiyama, Eiichiro Sumita et al.ACL Findings 2022 · 2 citations
  5. Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual UnderstandingZizhao Zhang, Han Zhang, Long Zhao, Ting Chen et al.AAAI 2022 · 216 citations
Dusk over the sea between two cliffs drawn in fine vertical lines