The Seminar on Math, Stat, and AI is an interdisciplinary seminar series focusing on problems at the intersection of statistics, probability, artificial intelligence and related fields. The upcoming seminar takes place on Friday, February 20th, at 10:30am in Maxwell-Dworkin G125. This week's speaker will be Denny Wu, faculty fellow at the Center for Data Science at NYU.
Title: Learning shallow neural networks in high dimensions: SGD dynamics and scaling laws
Abstract: We study the sample and time complexity of online stochastic gradient descent (SGD) in learning a two-layer neural network with M orthogonal neurons on isotropic Gaussian data. We focus on the challenging “extensive-width” regime M≫1 and allow for large condition number in the second-layer parameters, covering the power-law scaling a_m= m^{-β} as a special case. We characterize the SGD dynamics for the training of a student two-layer neural network and identify sharp transition times for the recovery of each signal direction. In the power-law setting, our analysis entails that while the learning of individual teacher neurons exhibits abrupt phase transitions, the juxtaposition of emergent learning curves at different timescales results in a smooth scaling law in the cumulative objective.
Zoom:
https://harvard.zoom.us/j/95074790201?pwd=gNqKfG0LybXHhobUTVQPhGWHd7GM6R.1
Password: 454727
LOCATION:Maxwell-Dworkin G125 STATUS:CONFIRMED DTSTART:20260221T033000Z DTEND:20260221T045859Z END:VEVENT END:VCALENDAR