Enhanced Human-Computer Interaction: A Unified Pipeline for Classification and Gesture Analysis

Aya Zuhair Salim; Luma Issa Abdul-Kreem

doi:10.46604/peti.2025.14973

Authors

Aya Zuhair Salim Department of Control and Systems Engineering, University of Technology, Baghdad, Iraq
Luma Issa Abdul-Kreem Department of Control and Systems Engineering, University of Technology, Baghdad, Iraq

DOI:

https://doi.org/10.46604/peti.2025.14973

Keywords:

object detection, gesture recognition, YOLOv3, SVM classifier, computational efficiency

Abstract

The purpose of this study is to develop a unified framework that combines object classification with vision-based gesture recognition. The proposed approach integrates YOLOv3 object detection enhanced by Z-Score Propensity Normalization to minimize false positives in Non-Maximum Suppression. Gesture recognition is performed using geometric contour detection and a Support Vector Machine classifier trained with Principal Component Analysis, which hierarchically refines detected bounding boxes and classifies hand gestures using spatial-temporal distance metrics. Experimental results show an average accuracy of 96.70%, a precision of 0.968, and an F1-score of 0.9671 for recognizing three gestures: hands down, one hand up, and hands up. This integrated method significantly improves computational efficiency and robustness, demonstrating strong potential for practical applications in augmented reality, assistive technologies, and immersive computing.

References

L. I. Abdul-Kreem and H. K. Abdul-Ameer, “Shadow Detection and Elimination for Robot and Machine Vision Applications,” Scientific Visualization, vol. 16, no. 2, pp. 11-22, 2024.

L. I. Abdul-Kreem, “Motion Estimations of Hand Movement Based on a Leap Motion Controller,” IEEE Sensors Journal, vol. 24, no. 11, pp. 17856-17864, 2024.

L. I. Abdul-Kareem, “Human Action Recognition based on Motion Velocity,” Proceedings of 2018 Third Scientific Conference of Electrical Engineering (SCEE), IEEE Press, pp. 45-50, 2018.

H. K. Abdul-Ameer, L. I. Abdul-Kreem, H. Adnan, and Z. Sami, “A Haptic Feedback System Based on Leap Motion Controller for Prosthetic Hand Application,” International Journal of Electrical and Computer Engineering, vol. 10, no. 6, pp. 5772-5778, 2020.

A. Z. Salim and L. I. Abdul-Kareem, “A Review of Advances in Bio-Inspired Visual Models Using Event-and Frame-Based Sensors,” Advances in Technology Innovation, vol. 10, no. 1, pp. 44-57, 2025.

S. A. M. Al-Juboori, H. Almutairi, R. Almajed, A. Ibrahim, and H. M. Gheni, “Detection of Hand Gestures with Human Computer Recognition by Using Support Vector Machine,” Periodicals of Engineering and Natural Sciences, vol. 10, no. 2, pp. 467-476, 2022.

R. Chen and X. Tian, “Gesture Detection and Recognition Based on Object Detection in Complex Background,” Applied Sciences, vol. 13, no. 7, article no. 4480, 2023.

N. Fei, Y. Gao, Z. Lu, and T. Xiang, “Z-Score Normalization, Hubness, and Few-Shot Learning,” Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE Press, pp. 142-151, 2021.

F. Al Farid, N. Hashim, J. B. Abdullah, MD. R. Bhuiyan, M. Kairanbay, Z. Yusoff, et al., “Single Shot Detector CNN and Deep Dilated Masks for Vision-Based Hand Gesture Recognition from Video Sequences,” IEEE Access, vol. 12, pp. 28564-28574, 2024.

C. Miron, A. Pasarica, H. Costin, V. Manta, R. Timofte, and R. Ciucu, “Hand Gesture Recognition Based on SVM Classification,” Proceedings of 2019 7th E-Health and Bioengineering Conference (EHB 2019), pp. 1-6, 2019.

V. Villani, C. Secchi, M. Lippi, and L. Sabattini, “A General Pipeline for Online Gesture Recognition in Human-Robot Interaction,” IEEE Transactions on Human-Machine Systems, vol. 53, no. 2, pp. 315-324, 2023.

P. Tsinganos, B. Jansen, J. Cornelis, and A. Skodras, “Real-Time Analysis of Hand Gesture Recognition with Temporal Convolutional Networks,” Sensors, vol. 22, no. 5, article no. 1694, 2022.

G. Bhaumik, M. Verma, M. C. Govil, and S. K. Vipparthi, “HyFiNet: Hybrid Feature Attention Network for Hand Gesture Recognition,” Multimedia Tools and Applications, vol. 82, no. 4, pp. 4863-4882, 2023.

P. T., V. K. Elumalai, and B. E., “Hand Gesture Classification Framework Leveraging the Entropy Features from sEMG Signals and VMD Augmented Multi-Class SVM,” Expert Systems with Applications, vol. 238, article no. 121972, 2024.

M. Kadavath, M. Nasor, and A. Imran, “Enhanced Hand Gesture Recognition with Surface Electromyogram and Machine Learning,” Sensors, vol. 24, no. 16, article no. 5231, 2024.

J. Qi, L. Ma, Z. Cui, and Y. Yu, “Computer Vision-Based Hand Gesture Recognition for Human-Robot Interaction: A Review,” Complex & Intelligent Systems, vol. 10, no. 1, pp. 1581-1606, 2024.

J. Wang, I. Ivrissimtzis, Z. Li, and L. Shi, “Hand Gesture Recognition for User-Defined Textual Inputs and Gestures,” Universal Access in the Information Society, vol. 27, pp. 1315-1329, 2024.

H. Zhou, H. T. Le, S. Zhang, S. L. Phung, and G. Alici, “Hand Gesture Recognition from Surface Electromyography Signals with Graph Convolutional Network and Attention Mechanisms,” IEEE Sensors Journal, vol. 25, no. 5, pp. 9081-9092, 2025.

S. Duan, “Deep Learning-Based Gesture Key Point Detection for Human-Computer Interaction Applications,” Transactions on Computational and Scientific Methods, vol. 5, no. 1, 2025.

C. Panagiotou, E. Faliagka, C. P. Antonopoulos, and N. Voros, “Multidisciplinary ML Techniques on Gesture Recognition for People with Disabilities in a Smart Home Environment,” AI, vol. 6, no. 1, article no. 17, 2025.

A. Blasco-Moreno, M. Pérez-Casany, P. Puig, M. Morante, and E. Castells, “What Does a Zero Mean? Understanding False, Random and Structural Zeros in Ecology,” Methods in Ecology and Evolution, vol. 10, no. 7, pp.949-959, 2019.

A. J. Shepley, G. Falzon, P. Kwan, and L. Brankovic, “Confluence: A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 10, pp. 11561-11574, 2023.

D. M. Abdullah and A. M. Abdulazeez, “Machine Learning Applications based on SVM Classification: A Review,” Qubahan Academic Journal, vol. 1, no. 2, pp. 81-90, 2021.

K. S. Chong and N. Shah, “Comparison of Naive Bayes and SVM Classification in Grid-Search Hyperparameter Tuned and Non-Hyperparameter Tuned Healthcare Stock Market Sentiment Analysis,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 12, pp. 90-94, 2022.

M. P. Libório, O. da Silva Martinuci, A. M. C. Machado, T. M. Machado-Coelho, S. Laudares, and P. Bernardes, “Principal Component Analysis Applied to Multidimensional Social Indicators Longitudinal Studies: Limitations and Possibilities,” GeoJournal, vol. 87, no. 3, pp. 1453-1468, 2022.

P. Verma, T. Bhardwaj, A. Bhatia, and M. Mursleen, “Sentiment analysis using SVM, KNN and SVM with PCA,” Artificial Intelligence in Cyber Security: Theories and Applications, Springer, Cham, vol. 240, pp. 35-53, 2023.

L. Huu, N. V. Hieu, and T. H. Nguyen, “Hand Gesture Recognition Algorithm Using SVM and HOG Model for Control of Robotic System,” Journal of Robotics, vol. 2021, article no. 3986497, pp. 1-13, 2021.

S. Wang, K. Wang, T. Yang, Y. Li, and D. Fan, “Improved 3D-ResNet Sign Language Recognition Algorithm with Enhanced Hand Features,” Scientific Reports, vol. 12, article no. 17812, 2022.

A. Mahmoud, M. Hu, and S. Patel, “Dynamic Hand Gesture Recognition Using 3DCNN and LSTM with an FSM Context-Aware Model,” Sensors, vol. 19, no. 24, article no. 5429, 2019.

H. Han, H. Zeng, L. Kuang, X. Han, and H. Xue, “A Human Activity Recognition Method Based on Vision Transformer,” Scientific Reports, vol. 14, article no. 15310, 2024.