Facial Expression Recognition Using HoG Features

Description

Facial expression is the most important and instant way to express the emotions of a person. Automatic facial expression recognition is a popular research topic of modern-day scenarios. It helps us to understand and prevent some abnormal situations like traffic accidents by identifying drowsiness, special interaction, and perception. The face images are useful for the intelligent vision-based human-computer interaction system. The automatic FER (Facial Expression Recognition) algorithm should be effective and robust. The machine learning approaches provide efficient solution terms of recognition purpose.

Thesis statement

In this thesis, we presented an automatic FER by cascaded features extraction using the histogram of oriented gradients (HOG), the histogram of bars (HOB), block orientation (BO), and linear binary pattern (LBP). The proposed work is developed in the MATLAB software 2016a. The Cohn Kanade (CK) dataset is tested with the proposed method. The Naïve Bayes machine learning classifier performs the classification task. The facial expression recognition rate is improved by the proposed method.

A similar work Age detection by NN trained LBP Features is available on free-thesis.com

Figure 1 General process of facial expression recognition

Types of facial emotions

The facial expression describes many facial functions like eyes, nose, lips, cheeks, etc. Their facial expression describes the feeling of a human being. The list of emotions and expression are classified into seven universal classes:

• Anger
• Disgust
• Fear
• Happy
• Sad
• Surprise
• Neutral

Applications of FER system

There are lots of applications of facial expression recognition in the computer field;

• Treatment of Autism
• Driver state surveillance
• Commercial survey
• Human-computer interaction
• Affective computing

Proposed Method

The automatic FER has wide applications in the robotics and forensic fields. In this work, we recognize the face expression from an image by extraction histogram-based features. The histogram-based features are HOG, HOB, BO, and LBP used in a cascaded manner. These features are used to train the Naïve Bayes (machine learning) classifier. It provided the face emotions recognition at the output end. The complete work is divided into six main portions.

1. Take input face image from Cohn Kanade (CK) dataset and pre-process them by detecting the face. We are taken 1648 images from the dataset which has all seven facial expressions. The class of the image is represented by facial expression. The input images are cropped and extract face images from them.
2. Extract the histogram-based features from the pre-processed facial image like HOG, HOB, BO, and LBP. These features are summed for the use of emotions recognition. Features are extracted using MATLAB of each image one by one and place it to a feature table. Feature-length of each image is shown in table 1. Each facial image comprises 2060 elements in the feature table. There is a total of 1648 images, so the dimension of feature table is 1648×2060.

Table 1: Data attributes

Feature  Name	Feature Length
HoG	1-900
HoB	901-1800
BO	1801-2000
LBP	2001-2059
Label of image	2060

Figure 2: Face detection with expression happiness, fear, disgust from CK dataset

3. We have divided the entire dataset into the training and testing data. We divided the data into 80:20 ratios.
4. Develop a classification model with the Naïve Bayes classifier. The training data is used to train the classifier.
5. Analyze the testing data with the trained Naïve Bayes classifiers.

Figure 3 Block diagram of the proposed method

Flow chart of the proposed method

Figure 3 Flowchart of the Entire Work

Results and Discussion

In this thesis, we proposed Automatic FER using Histogram based feature extraction. The Histogram oriented of Gradient (HOG), Block orientation (BO), Histogram of Bars (HOB), and Linear Binary Pattern (LBP) feature extraction methods are implemented to the Chon Kanade CK dataset. All the codes are developed in the MATLAB 2016a software. Different facial expression labels are listed in table 2. These labels are used to train the classifier.

Table 2: Facial expression Labels

Face expression	Label
Surprise	1
Fear	2
Happy	3
Sadness	4
Disgust	5
Anger	6
Neutral	7

The performance evaluation of the proposed method depends on Accuracy, Recall, Precision, and F score. The comparison of the proposed method is made with the HOG based FER system. Figures 5 and 6 reflect the evaluation parameter comparison among the HOG based FER and Proposed FER.

Figure 5: Overall performance comparison of HOG based FER and Proposed FER

Figure 6: Confusion Matrix comparison of HOG based FER and Proposed FER

Table 3 shows the accuracy per label using our proposed method and the existing standard HOG method of facial expression recognition. The proposed method has better accuracy per label than the HOG based FER method.

Table 3: Performance per label

Conclusion

In this thesis, an automatic FER task is performed with a hybrid histogram-based feature extraction method. We are used HOG, HOB, BO, and LBP feature extraction methods for recognition of facial expression of an image. A Naïve Bayes classifier is trained with the combined extracted features. A total of seven facial expressions are recognized while testing the CK dataset. The proposed FER method is provided better classification accuracy than the HOG based FER.

References

1. Sumati, S. Singh and S. C. Gupta, “Human object detection by HoG, HoB, HoC, and BO features,” 2016 Fourth International Conference on Parallel, Distributed and Grid Computing (PDGC), Waknaghat, 2016, pp. 742-746.
2. Jumani, Sahar Zafar, Fayyaz Ali, Subhash Guriro, Irfan Ali Kandhro, Asif Khan, and Adnan Zaidi. “Facial Expression Recognition with Histogram of Oriented Gradients using CNN.” Indian Journal of Science and Technology12 (2019): 24.
3. V. Patil and P. Bailke, “Real time facial expression recognition using RealSense camera and ANN,” 2016 International Conference on Inventive Computation Technologies (ICICT), Coimbatore, 2016, pp. 1-6.
4. Kumar, M. K. Bhuyan and B. K. Chakraborty, “Extraction of informative regions of a face for facial expression recognition,” in IET Computer Vision, vol. 10, no. 6, pp. 567-576, 9 2016.
5. d. A. Fernandes, L. N. Matos and M. G. d. S. Aragão, “Geometrical Approaches for Facial Expression Recognition Using Support Vector Machines,” 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Sao Paulo, 2016, pp. 347-354.