#  Statistics Colloquium: Tengyuan Liang 

 



####  calendar\_today Date and Time 

 **September 16, 2019** 

 12:00PM - 01:00PM EDT 

####  pin\_drop Location 

 **Science Center Lecture Hall A**  



 

 



 

 Tengyuan Liang  
Assistant Professor of Econometrics and Statistics  
University of Chicago, Booth School of Business

 Title: On Adaptivity, Generalization, and Interpolation Motivated from Neural Networks

 Abstract:  
In the absence of explicit regularization, neural network learning and kernel learning have the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for minimum-norm interpolated solutions in reproducing kernel Hilbert spaces (RKHS) which is due to a combination of high dimensionality of the input data, curvature of the kernel function, and favorable geometric properties of the data such as an eigenvalue decay of the empirical covariance and kernel matrices. In a follow-up work generalizing this viewpoint, we derive upper bounds on the risk that exhibit a multiple-descent shape for the various high dimensional scalings. Later, we show how training neural networks connect to the kernel learning theory established so far. Consider the problem: given data pair (x, y) drawn from a population with f\_\*(x) = E\[y|x\], specify a neural network and run gradient flow on the weights over time until reaching any stationarity. How does f\_t, the function computed by the neural network at time t, relate to f\_\*, in terms of approximation and representation? What are the provable benefits of the adaptive representation by neural networks compared to the pre-specified fixed basis representation in the classical nonparametric literature? We answer the above questions via a dynamic kernel approach indexed by the training process of neural networks.



 

 



 

 See also:- [ Colloquia ](/event-type/colloquia)
 
 

 Share on:- [     Facebook ](#)
- [     Twitter ](#)
- [     Linkedin ](#)
 


 Save: [ Add to calendar calendar\_today ](https://statistics.fas.harvard.edu/node/1222797/event-feed.ics)  Copy link link