Hierarchical Bayesian nonparametric models with applications

January, 2009
Report Number: 
Yee Whye Teh and Michael I. Jordan

Hierarchical modeling is a fundamental concept in Bayesian statistics. The basic idea is that parameters are endowed with distributions which may themselves introduce new parameters, and this construction recurses. A common motif in hierarchical modeling is that of the conditionally independent hierarchy, in which a set of parameters are coupled by making their distributions depend on a shared underlying parameter. These distributions are often taken to be identical, based on an assertion of exchangeability and an appeal to de Finetti's theorem. In this review we discuss a thoroughgoing exploitation of hierarchical modeling ideas in Bayesian nonparametric statistics. The basic idea is that rather than treating distributional parameters parametrically, we treat them nonparametrically. In particular, the base measure $G_0$ in the Dirichlet process can itself be viewed as a random draw from some distribution on measures---specifically it can be viewed as a draw from the Dirichlet process. This yields a natural recursion that we refer to as a hierarchical Dirichlet process. Our focus in this chapter is on nonparametric hierarchies of this kind, where the tools of Bayesian nonparametric modeling are used recursively.

PDF File: