Generic Face Tracking

Generic face tracking is one of those problems in computer-vision that everyone claims/thinks is solved but no one seems to have a working system. This project aims to rectify this situation. We have developed a real-time tracker that works from a webcam on typical hardware (i.e. a laptop). We are releasing a research/non-profit version of the tracker. Please visit the FaceTracker website for more info.

Facial Expression Recognition

In this project we develop a system capable of generating expression transcripts of videos in an interview setting. The system uses real-time generic face tracking and a multi-class classifier to discriminate the seven basic expressions. Expression transcripts are meta-data that can be used to better understand the results of discourse and may even detect when politician lie... someday.

Facial Puppetry

This project aims at developing methods that enable natural communication over a distance without revealing one’s identity. The problem consists of two components: real-time generic face tracking and facial expression transfer (i.e. rigging). In order to be a useful tool for communication, the system must be capable of real-time performance on typical hardware (i.e. a laptop). The system should also enable users to generate their own avatars without too much effort of know how. The system requires only a single image of the desired avatar.

Related Publications:

  1. • J. Saragih, S. Lucey, and J. Cohn, ``Real-Time Avatar Animation from a Single Image", IEEE International Conference on Automatic Face and Gesture Recognition (FG’11), 2011. [pdf]

  2. • R. J. Bennets, D. Burke, K. Brooks, J. Kim, S. Lucey, J. Saragih and R. A. Robbins, ``Avatars vs Point-light Faces: Movement Matching is Better Without a Face”, Proceedings of the 38th Australasian Experimental Psychology Conference, 2011.

Related Publications:

  1. • A. Asthana, J. Saragih and R. Goecke, ``Evaluating AAM Fitting Methods for Facial Expression Recognition", International Conference on Affective Computing and Intelligent Interaction (ACII), 2009.

  2. • A. Ryan, J. F. Cohn, S. Lucey, J. Saragih, P. Lucey, F. De la Torre, and A. Rossi, ``Automated Facial Expression Recognition System", IEEE International Carnahan Conference on Security Technology, 2009.

  3. • S. Chew, P. Lucey, S. Lucey, J. Saragih, J. Cohn and S. Sridharan, ``Person-Independent Facial Expression Detection using Constrained Local Models’’, FG 2011 Workshop on Facial Expression Recognition and Analysis Challange (FERA 2011). [pdf]

  4. • P. Lucey, J. Cohn, T. Kanade, J. Saragih, Z. Ambadar and I. Matthews, ``The Extended Cohn-Kanade Dataset (CK+): A Complete Facial Expression Dataset for Action Unit and Emotion-Specified Expression", IEEE CVPR Workshop on Human Communicative Behavior Analysis (CVPR4HB), 2010. [pdf]

Related Publications:

  1. • J. Saragih, S. Lucey and J. Cohn, ``Deformable Model Fitting by Regularized Landmark Mean-Shifts", International Journal of Computer Vision (IJCV), 2010. [pdf]

  2. • S. Lucey, Y. Wang, J. Saragih and J. Cohn, ``Non-rigid Face Tracking with Enforced Convexity and Local Appearance Consistency Constraint", International Journal of Image and Vision Computing (IVC), 2010

  3. •J. Saragih, S. Lucey, and J. Cohn, ``Face Alignment through Subspace Constrained Mean-Shifts", IEEE International Conference on Computer Vision (ICCV), 2009. [pdf]

  4. • J. Saragih, S. Lucey and J. Cohn, ``Deformable Model Fitting with a Mixture of Local Experts", International Conference on Computer Vision (ICCV), 2009. [pdf]