An Introduction to Variational Methods for Graphical Models

An Introduction to Variational Methods for Graphical Models

Report Number
508
Authors
David Aldous and Steven N. Evans
Citation
Electronic Journal of Probability</em>, Vol. 5 (2000) Paper no. 2, pages 1-18
Abstract

This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models. We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, showing how upper and lower bounds can be found for local probabilities, and discussing methods for extending these bounds to bounds on global probabilities of interest. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.

PDF File
Postscript File