Learning Representations for Face Recognition: A Review from Holistic to Deep Learning
DOI:
https://doi.org/10.46604/aiti.2022.8308Keywords:
learning representations, deep learning, autoencoders, variational autoencodersAbstract
For decades, researchers have investigated how to recognize facial images. This study reviews the development of different face recognition (FR) methods, namely, holistic learning, handcrafted local feature learning, shallow learning, and deep learning (DL). With the development of methods, the accuracy of recognizing faces in the labeled faces in the wild (LFW) database has been increased. The accuracy of holistic learning is 60%, that of handcrafted local feature learning increases to 70%, and that of shallow learning is 86%. Finally, DL achieves human-level performance (97% accuracy). This enhanced accuracy is caused by large datasets and graphics processing units (GPUs) with massively parallel processing capabilities. Furthermore, FR challenges and current research studies are discussed to understand future research directions. The results of this study show that presently the database of labeled faces in the wild has reached 99.85% accuracy.
References
G. Guo, et al., “A Survey on Deep Learning Based Face Recognition,” Computer Vision and Image Understanding, vol. 189, Article no. 102805, December 2019.
P. Viola, et al., “Rapid Object Detection Using a Boosted Cascade of Simple Features,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1-9, December 2001.
M. Wang, et al., “Deep Face Recognition: A Survey,” https://arxiv.org/pdf/1804.06655v2.pdf, April 2018.
L. Sirovich, et al., “Low-Dimensional Procedure for the Characterization of Human Faces,” Journal of the Optical Society of America A, vol. 4, pp. 519-524, March 1987.
M. Ballantyne, et al., “Woody Bledsoe: His Life and Legacy,” AI Magazine, vol. 17, no. 1, pp. 7-20, 1996.
A. J. Goldstein, et al., “Man-Machine Interaction in Human-Face Identification,” Bell System Technical Journal, vol. 51, no. 2, pp. 399-427, 1972.
M. A. Turk, et al., “Face Recognition Using Eigenfaces,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 586-587, January 1991.
W. Zhao, et al., “Subspace Linear Discriminant Analysis for Face Recognition,” http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.7.6280&rep=rep1&type=pdf, April 1999.
M. S. Bartlett, et al., “Face Recognition by Independent Component Analysis,” IEEE Transactions on Neural Networks, vol. 13, no. 6, pp. 1450-1464, December 2002.
F. Samaria, et al., “HMM-Based Architecture for Face Identification,” Image and Vision Computing, vol. 12, no. 8, pp. 537-543, October 1994.
H. Schneiderman, et al., “Probabilistic Modeling of Local Appearance and Spatial Relationships for Object Recognition,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 45-51, June 1998.
M. H. Yang, et al., “Detecting Faces in Images: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 34-58, August 2002.
X. He, et al., “Face Recognition Using Laplacianfaces,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 328-340, January 2005.
J. Wright, et al., “Robust Face Recognition via Sparse Representation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210-227, April 2008.
L. Zhang, et al., “Sparse Representation or Collaborative Representation: Which Helps Face Recognition?” International Conference on Computer Vision, pp. 471-478, November 2011.
L. Zhang, et al., “Collaborative Representation Based Classification for Face Recognition,” https://arxiv.org/vc/arxiv/papers/1204/1204.2358v1.pdf, April 2012.
L. Yang, et al., “Distance Metric Learning: A Comprehensive Survey,” https://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf, May 2006.
R. Jin, et al., “Regularized Distance Metric Learning: Theory and Algorithm,” Advances in Neural Information Processing Systems, vol. 22, pp. 862-870, December 2009.
J. G. Daugman, “Two-Dimensional Spectral Analysis of Cortical Receptive Field Profiles,” Vision Research, vol. 20, no. 10, pp. 847-856, January 1980.
C. Liu, et al., “A Gabor Feature Classifier for Face Recognition,” 8th IEEE International Conference on Computer Vision, pp. 270-275, July 2001.
T. Barbu, “Gabor Filter-Based Face Recognition Technique,” Proceedings of the Romanian Academy, vol. 11, no. 3, pp. 277-283, March 2010.
M. Yang, et al., “Gabor Feature Based Sparse Representation for Face Recognition with Gabor Occlusion Dictionary,” European Conference on Computer Vision, pp. 448-461, September 2010.
V. Štruc, et al., “Principal Gabor Filters for Face Recognition,” 3rd International Conference on Biometrics: Theory, Applications, and Systems, pp. 1-6, September 2009.
T. Ahonen, et al., “Face Recognition with Local Binary Patterns,” European Conference on Computer Vision, pp. 469-481, May 2004.
T. Ahonen, et al., “Face Description with Local Binary Patterns: Application to Face Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037-2041, October 2006.
T. Ojala, et al., “A Comparative Study of Texture Measures with Classification Based on Featured Distributions,” Pattern Recognition, vol. 29, no. 1, pp. 51-59, January 1996.
D. Huang, et al., “Local Binary Patterns and Its Application to Facial Image Analysis: A Survey,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 41, no. 6, pp. 765-781, March 2011.
R. R. Isnanto, et al., “Multi-Object Face Recognition Using Local Binary Pattern Histogram and Haar Cascade Classifier on Low-Resolution Images,” International Journal of Engineering and Technology Innovation, vol. 11, no. 1, pp. 45-58, January 2021.
D. S. Bolme, “Elastic Bunch Graph Matching,” Master thesis, Department of Computer Science, Colorado State University, CO, 2003.
B. M. Lahasan, et al., “Recognizing Faces Prone to Occlusions and Common Variations Using Optimal Face Subgraphs,” Applied Mathematics and Computation, vol. 283, pp. 316-332, June 2016.
D. G. Lowe, “Object Recognition from Local Scale-Invariant Features,” 7th IEEE International Conference on Computer Vision, pp. 1150-1157, September 1999.
D. G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, November 2004.
J. Luo, et al., “Person-Specific SIFT Features for Face Recognition,” IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 593-596, April 2007.
C. Geng, et al., “Face Recognition Using SIFT Features,” 16th IEEE International Conference on Image Processing, pp. 3313-3316, November 2009.
N. Dalal, et al., “Histograms of Oriented Gradients for Human Detection,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 886-893, June 2005.
O. Déniz, et al., “Face Recognition Using Histograms of Oriented Gradients,” Pattern Recognition Letters, vol. 32, no. 12, pp. 1598-1603, September 2011.
Z. Cao, et al., “Face Recognition with Learning-Based Descriptor,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2707-2714, June 2010.
Z. Lei, et al., “Learning Discriminant Face Descriptor,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 2, pp. 289-302, June 2013.
J. Sánchez, et al., “Image Classification with the Fisher Vector: Theory and Practice,” International Journal of Computer Vision, vol. 105, no. 3, pp. 222-245, December 2013.
T. H. Chan, et al., “PCANet: A Simple Deep Learning Baseline for Image Classification?” IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5017-5032, September 2015.
Y. Taigman, et al., “Deepface: Closing the Gap to Human-Level Performance in Face Verification,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701-1708, June 2014.
Y. Sun, et al., “Deep Learning Face Representation by Joint Identification-Verification,” Advances in Neural Information Processing Systems, pp. 1988-1996, December 2014.
Y. Sun, et al., “Deepid3: Face Recognition with Very Deep Neural Networks,” https://arxiv.org/pdf/1502.00873.pdf, February 2015.
F. Schroff, et al., “Facenet: A Unified Embedding for Face Recognition and Clustering,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815-823, June 2015.
J. Deng, et al., “Arcface: Additive Angular Margin Loss for Deep Face Recognition,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4690-4699, June 2019.
H. Liu, et al., “Adaptiveface: Adaptive Margin and Sampling for Face Recognition,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11947-11956, June 2019.
G. B. Huang, et al., “Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments,” Workshop on Faces in Real-Life Images: Detection, Alignment, and Recognition, pp. 1-11, October 2008.
M. Taskiran, et al., “Face Recognition: Past, Present and Future (A Review),” Digital Signal Processing, vol. 106, Article no. 102809, November 2020.
Y. Bengio, Learning Deep Architectures for AI, Hanover: Now Publishers, 2009.
Y. Bengio, et al., “Representation Learning: A Review and New Perspectives,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798-1828, March 2013.
S. Hayman, “The McCulloch-Pitts Model,” International Joint Conference on Neural Networks, pp. 4438-4439, July 1999.
F. Rosenblatt, “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain,” Psychological Review, vol. 65, no. 6, pp. 386-408, November 1958.
M. Minsky, et al., Perceptrons, Cambridge: MIT Press, 1969.
K. Fukushima, “Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition,” Proceedings of the U.S.-Japan Joint Seminar, pp. 267-285, February 1982.
A. Krizhevsky, et al., “Imagenet Classification with Deep Convolutional Neural Networks,” Advances in Neural Information Processing Systems, vol. 25, pp. 1097-1105, December 2012.
Y. Zheng, et al., “Ring Loss: Convex Feature Normalization for Face Recognition,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5089-5097, June 2018.
M. Yan, et al., “Vargfacenet: An Efficient Variable Group Convolutional Neural Network for Lightweight Face Recognition,” Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp. 2647-2654, October 2019.
J. Schmidhuber, “Deep Learning in Neural Networks: An Overview,” Neural Networks, vol. 61, pp. 85-117, January 2015.
G. E. Hinton, et al., “Reducing the Dimensionality of Data with Neural Networks,” Science, vol. 313, no. 5786, pp. 504-507, July 2006.
I. Goodfellow, et al., Deep Learning, Cambridge: MIT Press, 2016.
D. P. Kingma, et al., “Auto-Encoding Variational Bayes,” https://arxiv.org/pdf/1312.6114v4.pdf, December 2013.
D. P. Kingma, et al., “An Introduction to Variational Autoencoders,” https://arxiv.org/pdf/1906.02691v1.pdf, June 2019.
C. Doersch, “Tutorial on Variational Autoencoders,” https://arxiv.org/pdf/1606.05908v1.pdf, June 2016.
I. Goodfellow, et al., “Generative Adversarial Nets,” Proceedings of the 27th International Conference on Neural Information Processing Systems, pp. 2672-2680, December 2014.
M. You, et al., “Systematic Evaluation of Deep Face Recognition Methods,” Neurocomputing, vol. 388, pp. 144-156, May 2020.
F. N. Iandola, et al., “SqueezeNet: AlexNet-Level Accuracy with 50x Fewer Parameters and <0.5 MB Model Size,” https://arxiv.org/pdf/1602.07360.pdf, November 2016.
A. G. Howard, et al., “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision,” https://arxiv.org/pdf/1704.04861.pdf, April 2017.
S. Ge, et al., “Low-Resolution Face Recognition in the Wild via Selective Knowledge Distillation,” IEEE Transactions on Image Processing, vol. 28, no. 4, pp. 2051-2062, November 2018.
P. Li, et al., “On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques,” IEEE Transactions on Information Forensics and Security, vol. 14, no. 8, pp. 2000-2012, August 2019.
M. Grgic, et al., “SCface-Surveillance Cameras Face Database,” Multimedia Tools Applications, vol. 51, no. 3, pp. 863-879, February 2011.
A. Sapkota, et al., “Large Scale Unconstrained Open Set Face Database,” IEEE 6th International Conference on Biometrics: Theory, Applications, and Systems, pp. 1-8, September 2013.
A. Radford, et al., “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks,” https://arxiv.org/pdf/1511.06434v1.pdf, November 2015.
L. S. Luevano, et al., “A Study on the Performance of Unconstrained Very Low Resolution Face Recognition: Analyzing Current Trends and New Research Directions,” IEEE Access, vol. 9, pp. 75470-75493, May 2021.
R. Ranjan, et al., “A Fast and Accurate System for Face Detection, Identification, and Verification,” IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 1, no. 2, pp. 82-96, April 2019.
B. F. Klare, et al., “Pushing the Frontiers of Unconstrained Face Detection and Recognition: IARPA Janus Benchmark A,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1931-1939, June 2015.
C. Whitelam, et al., “IARPA Janus Benchmark-B Face Dataset,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 90-98, July 2017.
B. Maze, et al., “IARPA Janus Benchmark-C: Face Dataset and Protocol,” International Conference on Biometrics, pp. 158-165, February 2018.
“WHO Director-General’s Opening Remarks at the Media Briefing on COVID-19–11 March 2020,” https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020, March 11, 2020.
A. Heidari, et al., “The COVID-19 Epidemic Analysis and Diagnosis Using Deep Learning: A Systematic Literature Review and Future Directions,” Computers in Biology and Medicine, vol. 141, Article no. 105141, February 2022.
M. Abboah-Offei, et al., “A Rapid Review of the Use of Face Mask in Preventing the Spread of COVID-19,” International Journal of Nursing Studies Advances, vol. 3, Article no. 100013, November 2021.
H. Fouquet, “Paris Tests Face-Mask Recognition Software on Metro Riders,” https://www.bloombergquint.com/politics/paris-tests-face-mask-recognition-software-on-metro-riders, May 07, 2020.
E. Mbunge, et al., “Application of Deep Learning and Machine Learning Models to Detect COVID-19 Face Masks–A Review,” Sustainable Operations and Computers, vol. 2, pp. 235-245, January 2021.
A. Nowrin, et al., “Comprehensive Review on Facemask Detection Techniques in the Context of Covid-19,” IEEE Access, vol. 9, pp. 106839-106864, July 2021.
P. Khandelwal, et al., “Using Computer Vision to Enhance Safety of Workforce in Manufacturing in a Post COVID World,” https://arxiv.org/ftp/arxiv/papers/2005/2005.05287.pdf, May 2020.
M. Loey, et al., “Fighting against COVID-19: A Novel Deep Learning Model Based on YOLO-v2 with ResNet-50 for Medical Face Mask Detection,” Sustainable Cities and Society, vol. 65, Article no. 102600, February 2021.
S. V. Militante, et al., “Real-Time Facemask Recognition with Alarm System Using Deep Learning,” 11th IEEE Control and System Graduate Research Colloquium, pp. 106-110, August 2020.
B. Qin, et al., “Identifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19,” Sensors, vol. 20, no. 18, Article no. 5236, September 2020.
M. Inamdar, et al., “Real-Time Face Mask Identification Using Facemasknet Deep Learning Network,” https://ssrn.com/abstract=3663305, July 2020.
M. Jiang, et al., “Retinamask: A Face Mask Detector,” https://arxiv.org/pdf/2005.03950v1.pdf, May 2020.
Published
How to Cite
Issue
Section
License
Submission of a manuscript implies: that the work described has not been published before that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication. Authors can retain copyright in their articles with no restrictions. is accepted for publication. Authors can retain copyright of their article with no restrictions.
Since Jan. 01, 2019, AITI will publish new articles with Creative Commons Attribution Non-Commercial License, under The Creative Commons Attribution Non-Commercial 4.0 International (CC BY-NC 4.0) License.
The Creative Commons Attribution Non-Commercial (CC-BY-NC) License permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.
