EMNLP Findings2023
RWKV: Reinventing RNNs for the Transformer Era
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella
Abstract
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanisław Woźniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.
Ask Lune
Ask anything. The answer quotes the paper.
Lune reads all of it and the papers around it, then shows you the passages behind every answer.
Lune answers from
the full paper · 136 top-tier papers that cite it · related work
Free to start. No credit card required.
Cited by top-tier papers136
- VMamba: Visual State Space ModelYue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu et al.NeurIPS 2024 · 3,199 citations
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space DualityTri Dao, Albert GuICML 2024 · 1,407 citations
- Diffusion Forcing: Next-token Prediction Meets Full-Sequence DiffusionBoyuan Chen, Diego Marti Monso, Yilun Du, Max Simchowitz et al.NeurIPS 2024 · 751 citations
- xLSTM: Extended Long Short-Term MemoryMaximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer et al.NeurIPS 2024 · 703 citations
- MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse AttentionHuiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu et al.NeurIPS 2024 · 479 citations
Builds on21
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- MLP-Mixer: An all-MLP Architecture for VisionIlya O. Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer et al.NeurIPS 2021 · 3,862 citations
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- Big Bird: Transformers for Longer SequencesManzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie et al.NeurIPS 2020 · 3,159 citations
- WinoGrande: An Adversarial Winograd Schema Challenge at ScaleKeisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI 2020 · 3,037 citations
Related papers
- Cross-lingual Open-Retrieval Question Answering for African LanguagesOdunayo Ogundepo, Tajuddeen Gwadabe, Clara Rivera, Jonathan H. Clark et al.EMNLP Findings 2023 · 4 citations
- Can Language Models Follow Multiple Turns of Entangled Instructions?Chi Han, Xin Liu, Haodong Wang, Shiyang Li et al.EMNLP Findings 2025
- AHA: Aligning Large Audio-Language Models for Reasoning Hallucinations via Counterfactual Hard NegativesYanxi Chen, Wenhui Zhu, Xiwen Chen, Zhipeng Wang et al.ACL Findings 2026
- MorphoBench: A Benchmark with Difficulty Adaptive to Model ReasoningXukai Wang, Xuanbo Liu, Mingrui Chen, Haitian Zhong et al.ACL Findings 2026
- Do Vision-Language Models Have Internal World Models? Towards an Atomic EvaluationQiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen et al.ACL Findings 2025 · 3 citations
