Chiral Symmetry Breaking in Transformers: A Group-Equivariant Framework for Addressing the Reversal Curse via Adjoint Manifold Mappings
Hanji Du
摘要
The "reversal curse" exposes a critical asymmetry in autoregressive models, where models trained on facts in one direction often fail to access the corresponding inverse relation. This work studies the phenomenon from a representation-level perspective, characterizing it as a form of chiral asymmetry between subject- and object-oriented latent states. We introduce the Chiral Transformer, a lightweight framework that encourages an involutive adjoint mapping operator through contrastive regularization. At inference time, Adjoint-Induced Retrieval (AIR) uses this learned map as a structured readout over model-derived entity representations, rather than as an unconstrained autoregressive generation protocol. Empirical validation on inverse-relation benchmarks shows that this symmetry-aware retrieval setting substantially improves inverse factual access, with AIR reaching 65.07% accuracy on Fact-Inv-300. These findings support a representation-access view of the reversal curse: inverse relations may be difficult not only because of missing data, but also because standard autoregressive readout fails to expose useful latent structure.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper9
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 被引用 24,064 次
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni 等NeurIPS 2020 · 被引用 19,162 次
- Generalization through Memorization: Nearest Neighbor Language ModelsUrvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer 等ICLR 2020 · 被引用 1,038 次
- The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"Lukas Berglund, Meg Tong, Maximilian Kaufmann, Mikita Balesni 等ICLR 2024 · 被引用 462 次
- Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis 等EMNLP 2020 · 被引用 142 次
相关 Paper
- Breaking the Reversal Curse in Autoregressive Language Models via Identity BridgeXutao Ma, Yixiao Huang, Hanlin Zhu, Somayeh SojoudiICML 2026 · 被引用 2 次
- Towards a Theoretical Understanding of the 'Reversal Curse' via Training DynamicsHanlin Zhu, Baihe Huang, Shaolun Zhang, Michael I. Jordan 等NeurIPS 2024 · 被引用 37 次
- Bilinear representation mitigates reversal curse and enables consistent model editingDong-Kyum Kim, Minsung Kim, Jea Kwon, Nakyeong Yang 等ICLR 2026 · 被引用 1 次
- The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and MoreOuail Kitouni, Niklas Nolte, Adina Williams, Michael Rabbat 等NeurIPS 2024 · 被引用 29 次
- An Analysis and Mitigation of the Reversal CurseAng Lv, Kaiyi Zhang, Shufang Xie, Quan Tu 等EMNLP 2024
