Comparative Analysis of Facial Expression Recognition Using Image-Based and Landmark-Based Methods

Thanawat Srikaewsiew; Sarunya Kanjanawattana

doi:10.46604/aiti.2025.14589

Authors

Thanawat Srikaewsiew School of Computer Engineering, Institute of Engineering, Suranaree University of Technology, Nakhon Ratchasima, Thailand
Sarunya Kanjanawattana School of Computer Engineering, Institute of Engineering, Suranaree University of Technology, Nakhon Ratchasima, Thailand

DOI:

https://doi.org/10.46604/aiti.2025.14589

Keywords:

face expression recognition, machine model comparison, image-based classification, landmark-based classification, expression dataset

Abstract

This study compares the effectiveness of image-based and landmark-based methods for facial expression recognition (FER) in classifying hurt and normal facial expressions, utilizing datasets from the Delaware Pain Database and UTKFace. Five machine learning models are assessed, including convolutional neural networks (CNN), support vector machines (SVM), random forest classifier (RFC), logistic regression classifier (LRC), and gradient boosting classifier (GBC). The findings indicate that CNN achieves the highest accuracy at 95% when using landmark-based features, while SVM and GBC also perform admirably with these features. Conversely, LRC exhibits inconsistent results, especially when relying on image-based features. These findings offer valuable insights into the strengths and weaknesses of each approach, guiding the selection of effective FER techniques.

References

I. Michael Revina and W. R. Sam Emmanuel, “A Survey on Human Face Expression Recognition Techniques,” Journal of King Saud University - Computer and Information Sciences, vol. 33, no. 6, pp. 619-628, 2021.

D. Canedo and A. J. R. Neves, “Facial Expression Recognition Using Computer Vision: A Systematic Review,” Applied Sciences, vol. 9, no. 21, article no. 4678, 2019.

S. Kanjanawattana, P. Kittichaiwatthana, K. Srivisut, and P. Praneetpholkrang, “Deep Learning-Based Emotion Recognition through Facial Expressions,” Journal of Image and Graphics, vol. 11, no. 2, pp. 140-145, 2023.

S. Schindler, C. Tirloni, M. Bruchmann, and T. Straube, “Face and Emotional Expression Processing under Continuous Perceptual Load Tasks: An ERP Study,” Biological Psychology, vol. 161, article no. 108056, 2021.

R. W. Buck, V. J. Savin, R. E. Miller, and W. F. Caul, “Communication of Affect through Facial Expressions in Humans,” Journal of Personality and Social Psychology, vol. 23, no. 3, pp. 362-371, 1972.

C. L. Lisetti and D. J. Schiano, “Automatic Facial Expression Interpretation: Where Human-Computer Interaction, Artificial Intelligence and Cognitive Science Intersect,” Pragmatics & Cognition, vol. 8, no. 1, pp. 185-235, 2000.

C. Affonso, A. L. D. Rossi, F. H. A. Vieira, and A. C. P. de Leon Ferreira de Carvalho, “Deep Learning for Biological Image Classification,” Expert Systems with Applications, vol. 85, pp. 114-122, 2017.

P. Mende-Siedlecki, J. Qu-Lee, J. Lin, A. Drain, and A. Goharzad, “The Delaware Pain Database: A Set of Painful Expressions and Corresponding Norming Data,” Pain Reports, vol. 5, no. 6, article no. e853, 2020.

U. Sharma, K. N. Faisal, R. R. Sharma, and K. V. Arya, “Facial Landmark-Based Human Emotion Recognition Technique for Oriented Viewpoints in the Presence of Facial Attributes,” SN Computer Science, vol. 4, no. 3, article no. 273, 2023.

D. Lu and Q. Weng, “A Survey of Image Classification Methods and Techniques for Improving Classification Performance,” International Journal of Remote Sensing, vol. 28, no. 5, pp. 823-870, 2007.

M. Bodini, “A Review of Facial Landmark Extraction in 2D Images and Videos Using Deep Learning,” Big Data and Cognitive Computing, vol. 3, no. 1, article no. 14, 2019.

R. Chauhan, K. K. Ghanshala, and R. C. Joshi, “Convolutional Neural Network (CNN) for Image Detection and Recognition,” First International Conference on Secure Cyber Computing and Communication, pp. 278-282, 2018.

J. Hilbe, Logistic Regression Models, Boca Raton: Chapman & Hall/CRC, 2009.

M. A. Chandra and S. S. Bedi, “Survey on SVM and Their Application in Image Classification,” International Journal of Information Technology, vol. 13, no. 5, pp. 1-11, 2021.

N. M. Abdulkareem and A. M. Abdulazeez, “Machine Learning Classification Based on Radom Forest Algorithm: A Review,” International Journal of Science and Business, vol. 5, no. 2, pp. 128-142, 2021.

Z. He, D. Lin, T. Lau, and M. Wu, “Gradient Boosting Machine: A Survey,” https://doi.org/10.48550/arXiv.1908.06951, 2019.

V. N. Vapnik, The Nature of Statistical Learning Theory, 2nd ed., New York: Springer, 1999.

R. W. Picard, E. Vyzas, and J. Healey, “Toward Machine Emotional Intelligence: Analysis of Affective Physiological State,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1175-1191, 2001.

M. S. Bartlett, G. Littlewort, I. Fasel, and J. R. Movellan, “Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction,” Conference on Computer Vision and Pattern Recognition Workshop, pp. 53-53, 2003.

Y. S. Lee and W. H. Park, “Diagnosis of Depressive Disorder Model on Facial Expression Based on Fast R-CNN,” Diagnostics, vol. 12, no. 2, article no. 317, 2022.

E. Bekele, D. Bian, J. Peterman, S. Park, and N. Sarkar, “Design of a Virtual Reality System for Affect Analysis in Facial Expressions (VR-SAAFE); Application to Schizophrenia,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 6, pp. 739-749, 2017.

M. A. Assari and M. Rahmati, “Driver Drowsiness Detection Using Face Expression Recognition,” IEEE International Conference on Signal and Image Processing Applications, pp. 337-341, 2011.

D. Hamester, P. Barros, and S. Wermter, “Face Expression Recognition with a 2-Channel Convolutional Neural Network,” International Joint Conference on Neural Networks, pp. 1-8, 2015.

A. Alsubari, D. N. Satange, and R. J. Ramteke, “Facial Expression Recognition Using Wavelet Transform and Local Binary Pattern,” 2nd International Conference for Convergence in Technology, pp. 338-342, 2017.

Ö. F. Söylemez, B. Ergen, and N. H. Söylemez, “A 3D Facial Expression Recognition System Based on SVM Classifier Using Distance Based Features,” 25th Signal Processing and Communications Applications Conference, pp. 1-3, 2017.

M. I. N. P. Munasinghe, “Facial Expression Recognition Using Facial Landmarks and Random Forest Classifier,” IEEE/ACIS 17th International Conference on Computer and Information Science, pp. 423-427, 2018.

F. Di Luzio, A. Rosato, and M. Panella, “A Randomized Deep Neural Network for Emotion Recognition with Landmarks Detection,” Biomedical Signal Processing and Control, vol. 81, article no. 104418, 2023.

S. Hangaragi, T. Singh, and N. N, “Face Detection and Recognition Using Face Mesh and Deep Neural Network,” Procedia Computer Science, vol. 218, pp. 741-749, 2023.