Efficient constrained local model fitting for non-rigid face alignment

Simon Lucey, Yang Wang, Mark Cox, Sridha Sridharan, Jeffery F. Cohn

Research output: Contribution to journalArticlepeer-review

33 Scopus citations


Active appearance models (AAMs) have demonstrated great utility when being employed for non-rigid face alignment/tracking. The "simultaneous" algorithm for fitting an AAM achieves good non-rigid face registration performance, but has poor real time performance (2-3 fps). The "project-out" algorithm for fitting an AAM achieves faster than real time performance (>200 fps) but suffers from poor generic alignment performance. In this paper we introduce an extension to a discriminative method for non-rigid face registration/tracking referred to as a constrained local model (CLM). Our proposed method is able to achieve superior performance to the "simultaneous" AAM algorithm along with real time fitting speeds (35 fps). We improve upon the canonical CLM formulation, to gain this performance, in a number of ways by employing: (i) linear SVMs as patch-experts, (ii) a simplified optimization criteria, and (iii) a composite rather than additive warp update step. Most notably, our simplified optimization criteria for fitting the CLM divides the problem of finding a single complex registration/warp displacement into that of finding N simple warp displacements. From these N simple warp displacements, a single complex warp displacement is estimated using a weighted least-squares constraint. Another major advantage of this simplified optimization lends from its ability to be parallelized, a step which we also theoretically explore in this paper. We refer to our approach for fitting the CLM as the "exhaustive local search" (ELS) algorithm. Experiments were conducted on the CMU MultiPIE database.

Original languageEnglish
Pages (from-to)1804-1813
Number of pages10
JournalImage and Vision Computing
Issue number12
StatePublished - Nov 2009


  • Active appearance models
  • Constrained local models
  • Non-rigid face alignment


Dive into the research topics of 'Efficient constrained local model fitting for non-rigid face alignment'. Together they form a unique fingerprint.

Cite this