Learning advanced mathematical computations from examples
François Charton, Amaury Hayat, Guillaume Lample
Abstract
Using transformers over large generated datasets, we train models to learn mathematical properties of differential systems, such as local stability, behavior at infinity and controllability. We achieve near perfect prediction of qualitative characteristics, and good approximations of numerical features of the system. This demonstrates that neural networks can learn to perform complex computations, grounded in advanced theory, from examples, without built-in mathematical knowledge. * Equal contribution, names in alphabetic order.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext de10a6e4-5039-4c04-b288-de8f3f586d8eCited by top-tier papers13
- HyperTree Proof Search for Neural Theorem ProvingGuillaume Lample, Timothée Lacroix, Marie-Anne Lachaux, Aurélien Rodriguez et al.NeurIPS 2022 · 271 citations
- Proof Artifact Co-Training for Theorem Proving with Language ModelsJesse Michael Han, Jason Rute, Yuhuai Wu, Edward W. Ayers et al.ICLR 2022 · 149 citations
- Going Beyond Linear Transformers with Recurrent Fast Weight ProgrammersKazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen SchmidhuberNeurIPS 2021 · 101 citations
- SALSA: Attacking Lattice Cryptography with TransformersEmily Wenger, Mingjie Chen, François Charton, Kristin E. LauterNeurIPS 2022 · 61 citations
- The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of TransformersRóbert Csordás, Kazuki Irie, Jürgen SchmidhuberEMNLP 2021 · 55 citations
Builds on2
Related papers
- Global Lyapunov functions: a long-standing open problem in mathematics, with symbolic transformersAlberto Alfarano, François Charton, Amaury HayatNeurIPS 2024 · 54 citations
- Analytical Lyapunov Function Discovery: An RL-based Generative ApproachHaohan Zou, Jie Feng, Hao Zhao, Yuanyuan ShiICML 2025
- Predicting Ordinary Differential Equations with TransformersSören Becker, Michal Klein, Alexander Neitz, Giambattista Parascandolo et al.ICML 2023 · 28 citations
- Deep symbolic regression for recurrence predictionStéphane d'Ascoli, Pierre-Alexandre Kamienny, Guillaume Lample, François ChartonICML 2022 · 22 citations
- Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable TasksRahul Ramesh, Ekdeep Singh Lubana, Mikail Khona, Robert P. Dick et al.ICML 2024 · 17 citations
