Automated marker placement based real-time facial emotional expression recognition system

Abstract

Facial expression recognition attracted several researchers over the past several decades and most of the researchers in the literature focus on facial expression recognition in “offline” and very few research works concentrated on real-time facial expression recognition. In order to develop an intelligent real-time facial expression recognition system, this thesis proposed an automated marker placement method for classifying six basic facial expressions (happiness, sadness, anger, fear, disgust and surprise) using real-time video sequence. Initially, manual marker placement was carried out to detect the mean position (distance between the centre of the face to the marker’s location) of each marker on the subject’s face. This position was used to expand the automated marker placement algorithm for facial emotion recognition. In this experiment, subjects were requested manually to place ten markers (four markers on the upper face and six markers on lower face) on their face in specified locations based on Facial Action Coding System (FACS). Trial and error approach devised the number of markers used for facial expression detection. Manual markers were placed by clicking the cursor at each position on the facial image in video sequence. The mean marker position distance was calculated from the centre of the face. Calculation of each marker position concerning the middle of the face via manual marker placement was then used to develop the automatic marker placement algorithm.