Course Identification

Linear algebra, analysis and programming at an undergraduate level. A good level of Python is a plus but not a must.

The course Topics in Machine Learning is an excellent complement to this course, but not a prerequisite. It can be taken before, during or after.

The success of machine learning is evident from its achievements but far from being mathematically understood. This course combines a purely mathematical approach to some aspects of machine learning with an account of the major achievements of machine learning in mathematics.

The mathematical foundations of machine learning date back to the middle of the twentieth century. We will take a purely mathematical approach, without entering in statistical learning theory, to introduce and discuss the basic concepts and ideas.

On the other hand, the effective use of machine learning for mathematics is very recent. Since the 2021 Nature article ‘Advancing mathematics by guiding human intuition with AI’, many applications have  followed. We will learn the basic mathematical notions needed to understand them, explore their algorithms and discuss the nature of their results.

Finally, the students, in small groups, will tinker with the code of an approach of their choice and share their experience through a presentation. For the most advanced and interested students, there will be the opportunity to deal with a new project in collaboration with the lecturer.

Additionally, we expect to have a few invited lecturers to tell us more in depth about some aspects.

We will cover the following topics:
I. What machine learning does from a mathematical viewpoint.
• How to approximate a function? The class of neural networks, loss functions, gradient descent and back propagation.
• What to approximate functions for? Supervised, unsupervised and reinforcement learning [SB+98].
• Why can we approximate functions? A formal proof [LLPS93].
• What is really happening? Interpretability and Shapley values [Sha53].
Expressivity and the importance of depth.
• Other approaches: Kolmogorov-Arnold networks [LWV+25].
II. Applications
• Knot theory: relating knot invariants [DVB+21], unknotting [ABD+24].
• Graph theory: counterexamples via reinforcement learning [Wag21].
• Linear algebra: matrix multiplication algorithms [FBH+22, KM25].
• Geometry: predicting dimensions in algebraic geometry [CKV23], the Hirsch Conjecture [SWW+25].
• Combinatorics: the cap set problem through FunSearch [RPBN+24], extremal combinatorics with PatternBoost [CEWW24].
• Number theory: generating conjectures with the Ramanujan machine [RGM+21].
• Representation theory: Kazhdan-Lusztig polynomials [DVB+21].
• Dynamical systems: global Lyapunov systems via symbolic transformers [ACH24].
III. DIY
• The students will register their interest in a topic, level of expertise and mention other students they would like to work with.
• Depending on their answers, they will be distributed into small teams to work on the code of one of the applications.
• Each team will share their findings in a 30-45 min presentation.
• The most motivated and skilled teams will be offered an additional research question.
Some excellent online resources are: [MLW22, MCA23, Gri24].

Upon completion of this course, the students:
• will have a solid mathematical understanding of the main concepts of machine learning.
• will have familiarity with several applications of machine learning in various areas of mathematics and a deeper understanding of one of them.
• will be able to apply machine learning techniques in mathematical research problems.

[ABD+24] Taylor Applebaum, Sam Blackwell, Alex Davies, Thomas Edlich, András Juhász, Marc Lackenby, Nenad Tomasev, and Daniel Zheng. The unknotting number, hard unknot diagrams, and reinforcement learning. arXiv, 2409.09032, 2024.

[ACH24] Alberto Alfarano, François Charton, and Amaury Hayat. Global lyapunov functions: a long-standing open problem in mathematics, with symbolic transformers. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 93643–93670. Curran Associates, Inc., 2024.

[CEWW24] François Charton, Jordan S. Ellenberg, Adam Zsolt Wagner, and Geordie Williamson. Patternboost: Constructions in mathematics with a little help from AI. arXiv, 2411.00566, 2024.

[CKV23] Tom Coates, Alexander M. Kasprzyk, and Sara Veneziale. Machine learning the dimension of a fano variety. Nature Communications, 14(1), 2023.

[DVB+21] Alex Davies, Petar Velickovic, Lars Buesing, Sam Blackwell, Daniel Zheng, Nenad Tomasev, Richard Tanburn, Peter Battaglia, Charles Blundell, András Juhász, et al. Advancing mathematics by guiding human intuition with AI. Nature, 600(7887):70–74, 2021.

[FBH+22] Alhussein Fawzi, Matej Balog, Aja Huang, Thomas Hubert, Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Francisco J R. Ruiz, Julian Schrittwieser, Grzegorz Swirszcz, et al. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature,
610(7930):47–53, 2022.

[Gri24] Elisenda Grigsby. Graduate topics on deep learning theory. https://www.youtube.com/playlist?list=
PL0NRmB0fnLJSEXFQHGF0q5JcedxTqK4AJ, 2024. Video lectures from course at CMSA, Harvard.

[KM25] Manuel Kauers and Jakob Moosbauer. Some new non-commutative matrix multiplication algorithms of size (n, m, 6). ACM Commun. Comput. Algebra, 58(1):1–11, January 2025.

[LLPS93] Moshe Leshno, Vladimir Ya. Lin, Allan Pinkus, and Shimon Schocken. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks, 6(6):861–867, 1993.

[LWV+25] Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljacic, Thomas Y. Hou, and Max Tegmark. KAN: Kolmogorov–Arnold networks. In The Thirteenth International Conference on Learning Representations, 2025.

[MCA23] Mathematical challenges in AI seminar. https://sites.google.com/view/m-ml-sydney/home, 2023. Organized by Harini Desiraju, Georg Gottwald and Geordie Williamson. Sydney Mathematical Research Institute.

[MLW22] Machine learning for the working mathematician seminar. https://sites.google.com/view/mlwm-seminar-2022, 2022. Organized by Joel Gibson, Georg Gottwald, and Geordie Williamson. Sydney Mathematical Research Institute.

[RGM+21] Gal Raayoni, Shahar Gottlieb, Yahel Manor, George Pisha, Yoav Harris, Uri Mendlovic, Doron Haviv, Yaron Hadad, and Ido Kaminer. Generating conjectures on fundamental constants with the  Ramanujan machine. Nature, 590(7844):67–73, 2021.

[RPBN+24] Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. Mathematical discoveries from program search with large language models. Nature, 625(7995):468 – 475, 2024.

[Sha53] L. S. Shapley. 17. A Value for n-Person Games, pages 307–318. Princeton University Press, Princeton, 1953.

[SB+98] Richard S Sutton, Andrew G Barto, et al. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998.

[SWW+25] Grzegorz Swirszcz, Adam Zsolt Wagner, Geordie Williamson, Sam Blackwell, Bogdan Georgiev, Alex Davies, Ali Eslami, Sebastien Racaniere, Theophane Weber, and Pushmeet Kohli.
Advancing geometry with AI: Multi-agent generation of polytopes. arXiv, 2502.05199, 2025.

[Wag21] Adam Zsolt Wagner. Constructions in combinatorics via neural networks. arXiv, 2104.14516, 2021.