Colloq: Alexander (Sasha) Rakhlin

An Optimal Aggregation Procedure For Nonparametric Regression

How can one combine a collection of estimators of a regression function into a good aggregate? In the last 15 years, this age-old question has received increasing attention within the Mathematical Statistics community. A closely related question of regression in misspecified models has been studied within Statistical Learning using the techniques of empirical processes. We outline the shortcomings of the existing methods and present a new procedure that improves upon the least squares when the model is non-convex. We compare and contrast our estimator --- which is inspired by online methods in machine learning --- with the classical results on nonparametric regression under the assumption that the model is well-specified.

Based on (R., Sridharan, and Tsybakov, 2014) and (Liang, R., Sridharan, 2015)