In this work, a comprehensive study of how the HoG, HoB, BO, LBP features could be effectively exploited for facial expression recognition (FER) purposes has been carried out. In order to perform the above mentioned investigation, a procedure, commonly used in FER analysis and equipped with several features, has been used as the baseline. Results highlighted
a configuration of HoG and LBP parameters able to fit the specific aspects of facial expressions that allows a high classification performance capable to overcome the performances of using only HoG features.
Facial expression is the most important and instant way to express the emotions of a person. Automatic facial expression recognition is a popular research topic of modern-day scenarios. It helps us to understand and prevent some abnormal situations like traffic accidents by identifying drowsiness, special interaction, and perception. The face images are useful for the intelligent vision-based human-computer interaction system. The automatic FER (Facial Expression Recognition) algorithm should be effective and robust. The machine learning approaches provide efficient solution terms of recognition purpose.
Thesis statement
In this thesis, we presented an automatic FER by cascaded features extraction using the histogram of oriented gradients (HOG), the histogram of bars (HOB), block orientation (BO), and linear binary pattern (LBP). The proposed work is developed in the MATLAB software 2016a. The Cohn Kanade (CK) dataset is tested with the proposed method. The Naïve Bayes machine learning classifier performs the classification task. The facial expression recognition rate is improved by the proposed method.
Figure 1 General process of facial expression recognition
Types of facial emotions
The facial expression describes many facial functions like eyes, nose, lips, cheeks, etc. Their facial expression describes the feeling of a human being. The list of emotions and expression are classified into seven universal classes:
Anger
Disgust
Fear
Happy
Sad
Surprise
Neutral
Applications of FER system
There are lots of applications of facial expression recognition in the computer field;
Treatment of Autism
Driver state surveillance
Commercial survey
Human-computer interaction
Affective computing
Proposed Method
The automatic FER has wide applications in the robotics and forensic fields. In this work, we recognize the face expression from an image by extraction histogram-based features. The histogram-based features are HOG, HOB, BO, and LBP used in a cascaded manner. These features are used to train the Naïve Bayes (machine learning) classifier. It provided the face emotions recognition at the output end. The complete work is divided into six main portions.
Take input face image from Cohn Kanade (CK) dataset and pre-process them by detecting the face. We are taken 1648 images from the dataset which has all seven facial expressions. The class of the image is represented by facial expression. The input images are cropped and extract face images from them.
Extract the histogram-based features from the pre-processed facial image like HOG, HOB, BO, and LBP. These features are summed for the use of emotions recognition. Features are extracted using MATLAB of each image one by one and place it to a feature table. Feature-length of each image is shown in table 1. Each facial image comprises 2060 elements in the feature table. There is a total of 1648 images, so the dimension of feature table is 1648×2060.
Table 1: Data attributes
Feature Name
Feature Length
HoG
1-900
HoB
901-1800
BO
1801-2000
LBP
2001-2059
Label of image
2060
Facial expression recognition @free-thesis
Figure 2: Face detection with expression happiness, fear, disgust from CK dataset
We have divided the entire dataset into the training and testing data. We divided the data into 80:20 ratios.
Develop a classification model with the Naïve Bayes classifier. The training data is used to train the classifier.
Analyze the testing data with the trained Naïve Bayes classifiers.
facial expression recognition @free-thesis
Figure 3 Block diagram of the proposed method
Flow chart of the proposed method
facial expression recognition @free-thesis
Figure 3 Flowchart of the Entire Work
Results and Discussion
In this thesis, we proposed Automatic FER using Histogram based feature extraction. The Histogram oriented of Gradient (HOG), Block orientation (BO), Histogram of Bars (HOB), and Linear Binary Pattern (LBP) feature extraction methods are implemented to the Chon Kanade CK dataset. All the codes are developed in the MATLAB 2016a software. Different facial expression labels are listed in table 2. These labels are used to train the classifier.
Table 2: Facial expression Labels
Face expression
Label
Surprise
1
Fear
2
Happy
3
Sadness
4
Disgust
5
Anger
6
Neutral
7
The performance evaluation of the proposed method depends on Accuracy, Recall, Precision, and F score. The comparison of the proposed method is made with the HOG based FER system. Figures 5 and 6 reflect the evaluation parameter comparison among the HOG based FER and Proposed FER.
facial expression recognition @free-thesis
Figure 5: Overall performance comparison of HOG based FER and Proposed FER
facial expression recognition @free-thesis
Figure 6: Confusion Matrix comparison of HOG based FER and Proposed FER
Table 3 shows the accuracy per label using our proposed method and the existing standard HOG method of facial expression recognition. The proposed method has better accuracy per label than the HOG based FER method.
Table 3: Performance per label
Conclusion
In this thesis, an automatic FER task is performed with a hybrid histogram-based feature extraction method. We are used HOG, HOB, BO, and LBP feature extraction methods for recognition of facial expression of an image. A Naïve Bayes classifier is trained with the combined extracted features. A total of seven facial expressions are recognized while testing the CK dataset. The proposed FER method is provided better classification accuracy than the HOG based FER.
V. Patil and P. Bailke, “Real time facial expression recognition using RealSense camera and ANN,” 2016 International Conference on Inventive Computation Technologies (ICICT), Coimbatore, 2016, pp. 1-6.
Kumar, M. K. Bhuyan and B. K. Chakraborty, “Extraction of informative regions of a face for facial expression recognition,” in IET Computer Vision, vol. 10, no. 6, pp. 567-576, 9 2016.
d. A. Fernandes, L. N. Matos and M. G. d. S. Aragão, “Geometrical Approaches for Facial Expression Recognition Using Support Vector Machines,” 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Sao Paulo, 2016, pp. 347-354.
Reviews
There are no reviews yet.
Only logged in customers who have purchased this product may leave a review.
Reviews
There are no reviews yet.