The Seminar on Math, Stat, and AI is an interdisciplinary seminar series focusing on problems at the intersection of statistics, probability, artificial intelligence and related fields. The upcoming seminar takes place on Friday, February 27th at 10:30pm in Maxwell Dworkin G125. This week's speaker will be Margalit Glasgow, postdoc of the MIT.
Title: Propagation of Chaos in Shallow Neural Networks
Abstract: The analysis of gradient descent in neural networks remains an outstanding challenge, even for the simplest shallow architectures. In this talk, we'll investigate the gradient dynamics in 2-layer neural networks through the lens of the infinite-width "mean field" limit of neural networks. The infinite-width limit offers analytical simplicity and promises global convergence for certain activations. Yet showing that finite-width neural networks well-approximate their mean field limits throughout training (the so-called Propagation of Chaos phenomenon) is difficult. Typical results based on Gronwall’s inequality guarantee a convergence rate that depends exponentially on the training time. We provide a novel analysis that goes beyond the Gronwall approach by exploiting certain geometric properties of the optimization landscape. In particular, we establish conditions under which the approximation error scales only polynomially in the training time. We apply these results to representative models such as single-index models, establishing polynomial-time learning guarantees.
Joint work with Denny Wu and Joan Bruna (https://arxiv.org/abs/2504.13110, and upcoming work).
Zoom link:
https://harvard.zoom.us/j/96920442319?pwd=681DmL2inq6yfFbbKQ5elVab0aM85I.1
Password: 574355
Meeting ID: 969 2044 2319