A Modern Take on Huber Regression

In the first part of the talk, we discuss the use of a penalized Huber M-estimator for high-dimensional linear regression. We explain how a fairly straightforward analysis yields high-probability error bounds that hold even when the additive errors are heavy-tailed. However, the parameter governing the shape of the Huber loss must be chosen in relation to the scale of the error distribution. We discuss how to use an adaptive technique, based on Lepski's method, to overcome the difficulties traditionally faced by applying Huber M-estimation in a context where both location and scale are unknown.

In the second part of the talk, we turn to a more complicated setting where both the covariates and responses may be heavy-tailed and/or adversarially contaminated. We show how to modify the Huber regression estimator by first applying an appropriate "filtering" procedure to the data based on the covariates. We prove that in low-dimensional settings, this filtered Huber regression estimator achieves near-optimal error rates. We further show that the commonly used least trimmed squares and least absolute deviation estimators may similarly be made robust to contaminated covariates via the same covariate filtering step. This is based on joint work with Ankit Pensia (UW-Madison) and Varun Jog (Cambridge).

Please join the event.

About Po-Ling Loh

 Po-Ling Loh received her PhD in Statistics from UC Berkeley in 2014. From 2014-2016, she was an Assistant Professor of Statistics at the University of Pennsylvania. From 2016-2018, she was an Assistant Professor of Electrical & Computer Engineering at UW-Madison, and from 2019-2020, she was an Associate Professor of Statistics at UW-Madison and a Visiting Associate Professor of Statistics at Columbia University. She began a position as a Lecturer in the Department of Pure Mathematics and Mathematical Statistics at the University of Cambridge in January 2021. Po-Ling's current research interests include high-dimensional statistics, robustness, and differential privacy. She is a recipient of an NSF CAREER Award, an ARO Young Investigator Award, the IMS Tweedie and Bernoulli Society New Researcher Awards, and a Hertz Fellowship.