High Dimensional Data Analysis
High-dimensional statistics focuses on data sets in which the number of features is of comparable size, or larger than the number of observations. Data sets of this type present a variety of new challenges, since classical theory and methodology can break down in surprising and unexpected ways.
Researchers at Berkeley study both the statistical and computational challenges that arise in the high-dimensional setting. On the theoretical side, they bring to bear a range of techniques from statistics, probability, and information theory, including empirical process theory, concentration inequalities, as well as random matrix theory and free probability. Methodological innovations include new estimators for spectral properties of matrices, randomized procedures for sketching and optimization, as well as algorithms for decision-making in sequential settings. The work is motivated and applied to various scientific and engineering disciplines, including computational biology, astronomy, recommender systems, financial time series, and climate forecasting.