Chiral Symmetry Breaking in Transformers: A Group-Equivariant Framework for Addressing the Reversal Curse via Adjoint Manifold Mappings
Hanji Du
Abstract
The "reversal curse" exposes a critical asymmetry in autoregressive models, where models trained on facts in one direction often fail to access the corresponding inverse relation. This work studies the phenomenon from a representation-level perspective, characterizing it as a form of chiral asymmetry between subject- and object-oriented latent states. We introduce the Chiral Transformer, a lightweight framework that encourages an involutive adjoint mapping operator through contrastive regularization. At inference time, Adjoint-Induced Retrieval (AIR) uses this learned map as a structured readout over model-derived entity representations, rather than as an unconstrained autoregressive generation protocol. Empirical validation on inverse-relation benchmarks shows that this symmetry-aware retrieval setting substantially improves inverse factual access, with AIR reaching 65.07% accuracy on Fact-Inv-300. These findings support a representation-access view of the reversal curse: inverse relations may be difficult not only because of missing data, but also because standard autoregressive readout fails to expose useful latent structure.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on9
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Generalization through Memorization: Nearest Neighbor Language ModelsUrvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer et al.ICLR 2020 · 1,038 citations
- The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"Lukas Berglund, Meg Tong, Maximilian Kaufmann, Mikita Balesni et al.ICLR 2024 · 462 citations
- Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations
Related papers
- Breaking the Reversal Curse in Autoregressive Language Models via Identity BridgeXutao Ma, Yixiao Huang, Hanlin Zhu, Somayeh SojoudiICML 2026 · 2 citations
- Towards a Theoretical Understanding of the 'Reversal Curse' via Training DynamicsHanlin Zhu, Baihe Huang, Shaolun Zhang, Michael I. Jordan et al.NeurIPS 2024 · 37 citations
- Bilinear representation mitigates reversal curse and enables consistent model editingDong-Kyum Kim, Minsung Kim, Jea Kwon, Nakyeong Yang et al.ICLR 2026 · 1 citation
- The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and MoreOuail Kitouni, Niklas Nolte, Adina Williams, Michael Rabbat et al.NeurIPS 2024 · 29 citations
- An Analysis and Mitigation of the Reversal CurseAng Lv, Kaiyi Zhang, Shufang Xie, Quan Tu et al.EMNLP 2024
