A Multi-Backbone Feature-Concatenation Framework for Handwritten Prescription Word Classification
DOI:
https://doi.org/10.46604/aiti.2026.16426Keywords:
Handwritten prescription word classification, prescription digitization, deep learning, healthcare automationAbstract
This study proposes a multi-backbone deep learning framework for handwritten prescription word classification to support reliable prescription digitization. EfficientNet-B0, ViT-Base, and Swin-Base operate in parallel to extract complementary local, global, and hierarchical visual features from pre-cropped medicine-name images. The extracted features are concatenated and passed through fully connected and classification layers. Training incorporates mixup regularization, label smoothing, and a warm-up learning schedule. Experiments are conducted on the Doctor's Handwritten Prescription Bangladesh (BD) dataset, containing 4,680 images across 78 medicine-name classes. The framework is evaluated through five repeated runs, ablation studies, class-wise analysis, and comparisons with single- and dual-backbone baselines. It achieves an average accuracy of 88.44 ± 0.99% and a macro F1-score of 0.88 ± 0.01. Holm-corrected paired t-tests show significant improvements over EfficientNet-B0 and ViT-Base, whereas differences from Swin-Base and dual-backbone models are not statistically significant. These results demonstrate the value of combining complementary visual representations for robust prescription-word classification.
References
L. J. Fajardo, N. J. Sorillo, J. Garlit, C. D. Tomines, M. B. Abisado, J. M. R. Imperial, et al., “Doctor's Cursive Handwriting Recognition System Using Deep Learning,” Proceedings of 2019 IEEE 11th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), Laoag, Philippines, 2019.
G. Pavithiran, S. Padmanabhan, N. Divya, A. V, I. J. P, and B. Chandar, “Doctor's Handwritten Prescription Recognition System In Multi-Language Using Deep Learning,” Proceedings of 2022 International Conference on Power, Energy, Control and Transmission Systems (ICPECTS), Chennai, India, 2022.
O. Yakovchuk and M. Vasin, “Increasing the Accuracy of Handwriting Text Recognition in Medical Prescriptions with Generative Artificial Intelligence,” Technology Audit and Production Reserves, vol. 4, no. 2, article no. 72, pp. 18-21, 2023.
A. Thavayee and T. Vijayakumar, "Analyzing Medical Manuscripts through Neural Networks and Handwriting Recognition Algorithms," Journal of Emerging Technologies and Innovative Research, vol. 11, no. 3, pp. f52–f57, 2024.
A. Datta, M. M. Hasan, N. Mahmud, B. J. Ferdosi, R. Islam, Z. Rahman, et al., “Preserving Medical Information from Doctor's Prescription Ensuring Relation Among the Terminology,” Computers in Biology and Medicine, vol. 188, article no. 109812, 2025.
S. Tabassum, N. Abedin, M. M. Rahman, M. M. Rahman, M. T. Ahmed, R. Islam, et al., “An Online Cursive Handwritten Medical Words Recognition System for Busy Doctors in Developing Countries for Ensuring Efficient Healthcare Service Delivery,” Scientific Reports, vol. 12, no. 1, article no. 3601, 2022.
A. Y. Nimbalkar, K. P. Ubale, S. P. Patil, and V. H. Bhutnal, “MediCrypt: Survey on Automated Recognition of Handwritten Medical Prescriptions for Enhanced Healthcare Efficiency,” Proceedings of the International Conference on Multi-Strategy Learning Environment, pp. 413–430, 2024.
R. Achkar, K. Ghayad, R. Haidar, S. Saleh, and R. Al Hajj, “Medical Handwritten Prescription Recognition Using CRNN,” Proceedings of 2019 International Conference on Computer, Information and Telecommunication Systems (CITS), Beijing, China, 2019.
S. Tabassum, R. Takahashi, M. M. Rahman, Y. Imamura, L. Sixian, and M. M. Rahman,, “Recognition of Doctors' Cursive Handwritten Medical Words by Using Bidirectional LSTM and SRP Data Augmentation,” Proceedings of 2021 IEEE Technology & Engineering Management Conference-Europe (TEMSCON-EUR), Dubrovnik, Croatia, 2021.
E. Hassan, H. Tarek, M. Hazem, S. Bahnacy, L. Shaheen, and W. H. Elashmwai, “Medical Prescription Recognition Using Machine Learning,” Proceedings of 2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC), NV, USA, 2021.
N. Palani and N. Sampath, “Detecting and Extracting Information of Medicines from a Medical Prescription Using Deep Learning and Computer Vision,” Proceedings of 2022 International Conference on Knowledge Engineering and Communication Systems (ICKES) , Chickballapur, India, 2022.
S. Samson, C. Tejaswini, G. Rishikesh, and M. Ramu, “Analysis of Scanned Medical Prescription Using Machine Learning,” International Journal of Innovative Science and Research Technology, vol. 9, no. 1, pp. 396-399, 2024.
A. Razdan, A. Raghavan, M. Panandikar, A. Pol, K. Hedaoo, and R. Patil, “Recognition of Handwritten Medical Prescription Using CNN Bi-LSTM With Lexicon Search,” Proceedings of 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT) , Delhi, India, 2023.
A. R. Mia, M. A.-A.-S. Chowdhury, A. Al Mamun, A. M. Ruddra, and N. T. Tanny, “A Deep Neural Network Approach with Pioneering Local Dataset to Recognize Doctor's Handwritten Prescription in Bangladesh,” Proceedings of 2024 International Conference on Advances in Computing, Communication, Electrical, and Smart Systems (iCACCESS) , Dhaka, Bangladesh, 2024.
M. N. Ismael, “A Deep Learning Approach for Multi-Class Classification of Handwritten Prescription Images Using CNN and Grad-CAM Visualization,” International Journal of Engineering in Computer Science, vol. 7, no. 2, pp. 215–225, 2025.
G. R. Rekha, S. Siddesha, and V. N. M. Aradhya, “Treatment Regimen Segmentation from Handwritten Medical Prescriptions Using Advanced Neural Network,” Artificial Intelligence and Applications, in press, 2026.
B. Biswas, B. K. Sarkar, A. Mustafi, and A. Mitra, “A Novel Approach of Doctors' Handwritten Medical Prescription Recognition Using A Deep Neural Network and Generating Digital Records of The Patients,” Journal of Dynamics and Games, pp. 1-36, in press, 2026.
A. Maiti, A. Podder, C. Dutta, and D. Saha, “Improved RNN-based System for Deciphering Doctors' Handwritten Prescriptions,” American Journal of Advanced Computing, vol. 2, no. 2, pp. 1-5, 2023.
T. Joshi, S. Das, S. Ghosh, C. Jha, and V. D. Shinde, “Web Application for Interpretation of Doctor's Handwritten Prescription and Suggesting the Best Price Offer Over Various E-Commerce Websites Using AI,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, vol. 9, no. 2, pp. 1-5, 2023.
A. Al Mobassir, M. R. Hasan, M. A. Sadia Borni, M. R. Ahmed, and R. A. Prodhan, “Multimodal Deep Learning for Handwritten Prescription Recognition with Explainable AI,” Proceedings of 2026 IEEE 2nd International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN), Chittagong, Bangladesh, 2026.
D. Dhar, A. Garain, P. K. Singh, and R. Sarkar, “HP_DocPres: A Method for Classifying Printed and Handwritten Texts in Doctor's Prescription,” Multimedia Tools and Applications, vol. 80, pp. 9779-9812, 2021.
M. Agrawal, V. Mishra, H. S. Jogani, P. Kashyap, and R. G. Mishra, “A Review of Artificial Intelligence in Medical Prescription Analysis,” Proceedings of 2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN), Salem, India, 2023.
S. Vidhya, M. Yukendhira, and K. Sidharth, “MediScan Handwritten Prescription Translator Using Deep Learning,” Innovations and Advances in Cognitive Systems, vol 16, pp. 187-200, 2024.
Y. Li, D. Chen, T. Tang, and X. Shen, “HTR-VT: Handwritten Text Recognition with Vision Transformer,” Pattern Recognit., vol. 158, article no. 110967, 2025.
A. Taneja and S. Sharma, “Integrating EfficientNet and Vision Transformers for Enhanced Medical Image Classification: A Hybrid Approach,” Proceedings of the International Conference on Data Analytics & Management (ICDAM 2024), 2025.
E. Anbazhagan and E. Sophiya, "A Novel Handwritten Prescription Recognition with Stochastic Gradient Descent Using Adaptive Momentum Learning,"Proceedings of International Conference on Recent Innovations in Computing (ICRIC 2023), Lecture Notes in Electrical Engineering, Springer, Singapore.
A. Z. Tariq, F. Ahmed, A. Khandoker, and M. Rahman, “An AI-IoT Framework for Handwritten and Multilingual Prescription Interpretation with Timely Medication Reminder Support,” Proceedings of 2025 IEEE 49th Annual Computers, Software, and Applications Conference (COMPSAC), Toronto, ON, Canada, 2025.
S. R. Hossain, S. Ahmed, S. Sofder, and R. Khan, "Doctor's Handwritten Prescription Recognition Using Cascading Lightweight Hybrid Adaptive Attention (CLHAA)," Proceedings of IEEE Conference Publication, 2025.
M. Mamun, “Doctor's Handwritten Prescription BD Dataset,” Kaggle, 2024. [Online]. Available: https://www.kaggle.com/datasets/mamun1113/doctors-handwritten-prescription-bd-dataset. Accessed: July 4, 2026.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Swetha V Padmavathi Polisetty, Deepthi Godavarthi
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Submission of a manuscript implies: that the work described has not been published before that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication. Authors can retain copyright in their articles with no restrictions. is accepted for publication. Authors can retain copyright of their article with no restrictions.
Since Jan. 01, 2019, AITI will publish new articles with Creative Commons Attribution Non-Commercial License, under The Creative Commons Attribution Non-Commercial 4.0 International (CC BY-NC 4.0) License.
The Creative Commons Attribution Non-Commercial (CC-BY-NC) License permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.
