The Next Great Scientific Theory is Hiding Inside a Neural Network

In this talk I will describe a way to approach scientific discovery by interpreting a neural network trained on a given scientific dataset, rather than seeking to model that dataset directly. With enough data, machine learning can sometimes out-perform traditional scientific models for certain problems, such as, for example, numerical modelling of turbulence. If we are able to translate this learned representation into a human-interpretable model, we can use such insights to extend our own theory – in a way not possible by working directly with the data. I will present a broad overview of this idea, and several types of interpretability tools for attacking this, in particular with ‘symbolic regression’ using PySR (github.com/MilesCranmer/PySR), which gives us a way of building human-interpretable surrogate models. I will then transition to a discussion of a new kind of machine learning which is particularly interesting for studying in this context – foundation models – and present the "Polymathic" collaboration (polymathic-ai.org/) which is in the process of building a ‘GPT for Scientific Data'. This collaboration aims to build industry-scale foundation models for scientific numerical data across multiple disciplines, much in the same way that ChatGPT can model multiple languages at once (and leverage shared grammatical concepts). Such a model is interesting for several reasons, one of which is enhancing our own understanding with its uniquely 'polymathic' perspective.

Please join the event.

About Miles Cranmer

Miles Cranmer is an Assistant Professor in Data Intensive Science at the University of Cambridge, with joint appointments in the Department of Applied Mathematics and Theoretical Physics and the Institute of Astronomy. Cranmer hails from Guelph, Ontario, and completed his BSc at McGill University followed by a PhD at Princeton University with Shirley Ho and David Spergel. His research focuses on accelerating scientific discovery by developing and applying novel methods at the intersection of machine learning and physics. Cranmer cares deeply about interpretability in machine learning, and has created a suite of standard software libraries for “symbolic regression” – which have been utilized in numerous scientific discoveries. His work covers various areas of deep learning with an emphasis on physics-motivated architectures. Cranmer is also passionate about pedagogy and leads the machine learning modules for Cambridge's MPhil in Data Intensive Science. Finally, Cranmer is an active open-source software contributor and maintainer, and, in his work and teaching, stresses the importance of high-quality, accessible software for productive science.