The Probabilitas Seminar series focuses on high-dimensional problems that combine statistics, probability, information theory, computer science, and other related fields. The upcoming seminar takes place on Friday, December 6, from 10:30-11:30am EST. The talk will be hybrid, both in-person in Science Center 316 and on Zoom (please contact the department for Zoom information). This week's guest will be Reza Gheissari of Northwestern University.
Spectral transitions in Gaussian mixture classification
Abstract: It has been empirically observed that the spectrum of neural network Hessians after training have a bulk concentrated near zero, and a few outlier eigenvalues. Moreover, the eigenspaces associated to these outliers have been associated to a low-dimensional subspace in which most of the training occurs, and this implicit low-dimensional structure has been used as a heuristic for the success of high-dimensional classification. We will describe recent rigorous results in this direction for the Hessian spectrum over the course of the training by SGD in high-dimensional Gaussian classification tasks with one and two-layer networks. We focus on the separation of outlier eigenvalues from the bulk, and subsequent crystallization of the outlier eigenvectors. Based on joint work with Gerard Ben Arous, Jiaoyang Huang, and Aukosh Jagannath.
LOCATION:Science Center 316 STATUS:CONFIRMED DTSTART:20241206T153000Z DTEND:20241206T163000Z END:VEVENT END:VCALENDAR