Dependence Testing in High Dimension

The talk consists of two parts: mean independence testing and mutual independence testing for high dimensional data. In the first part, we first introduce Martingale difference divergence (MDD), which is a metric that quantifies the  mean dependence of a random vector Y given another random vector X and can be viewed as an extension of distance covariance.

We propose a novel test to assess the  mean dependence of a response variable on  a large  number  of  covariates. Our MDD-based procedure  is able to detect certain type of departure from the null hypothesis of  mean independence without making any specific model assumptions. We establish the asymptotic normality of the proposed test statistic under suitable assumptions that can be verified for covariates with banded dependence or Gaussian distribution. Power analysis and a wild bootstrap procedure will also be presented along with some simulation results.

In the second part, we propose a  L2 type test for testing mutual independence and banded dependence structure for high dimensional data. The test is constructed based on the pairwise distance covariance and it accounts for the non-linear and non-monotone dependences among the data, which cannot be fully captured by the existing tests based on either Pearson correlation or rank correlation. Both theoretical results and finite sample results will be presented.

Please register for the event.

About Xiaofeng Shao

Xiaofeng Shao is a professor of Statistics at the department of Statistics at University of Illinois, Urbana-Champaign. He graduated from University of Chicago in 2006 and has been on the University of Illinois faculty ever since.

His current research interests include econometrics, functional data analysis, time series analysis, high dimensional data analysis and Resampling methods. He is serving as an associate editor for Journal of American Statistical Association and Journal of Time Series Analysis.