Structured Prediction, Dual Extragradient and Bregman Projections

November, 2005
Report Number: 
697
Authors: 
Ben Taskar and Simon Lacoste-Julien and Michael I. Jordan
Abstract: 

We present a simple and scalable algorithm for maximum-margin estimation of structured prediction models, including an important class of Markov networks and combinatorial models. We formulate the estimation problem as a convex-concave saddle-point problem that allows us to use simple projection methods based on the dual extragradient algorithm (Nesterov, 2003). The projection step can be solved using dynamic programming or combinatorial algorithms for min-cost convex flow, depending on the structure of the problem. We show that this approach provides a memory-efficient alternative to formulations based on reductions to a quadratic program (QP). We analyze the convergence of the method and present experiments on two very different structured prediction tasks: 3D image segmentation and word alignment, illustrating the favorable scaling properties of our algorithm.

PDF File: 
Postscript File: