Quality Model and Quality Characteristics Evaluation Suitable for Software 2.0

Yoshimichi Watanabe; Yunarso Anang; Masakazu Takahashi

doi:10.46604/ijeti.2024.13303

Authors

Yoshimichi Watanabe Department of Computer Science and Engineering, University of Yamanashi, Yamanashi, Japan
Yunarso Anang Department of Computational Statistics, Politeknik Statistika STIS, Jakarta, Indonesia
Masakazu Takahashi Department of Computer Science and Engineering, University of Yamanashi, Yamanashi, Japan

DOI:

https://doi.org/10.46604/ijeti.2024.13303

Keywords:

AI products, software 2.0, software quality model, quality characteristics

Abstract

Software systems incorporating artificial intelligence (AI) technology are called software 2.0, and their development is spreading in various fields. The purpose of this study is to enable quality engineers to easily evaluate the quality of software 2.0. For this purpose, this paper proposes a quality model that can be used by quality engineers. In order to propose a new quality model for quality engineers, this paper selects and reorganizes quality elements from general-purpose quality models and guidelines, and reconstructs quality characteristics and their evaluation methods. The quality model is considered useful for quality engineers. The paper also identifies important quality characteristics in 24 different application areas and confirms the applicability of the proposed model. The result shows that the model is useful for quality engineers to evaluate the quality of software 2.0.

References

OECD, Artificial Intelligence in Society, Paris: OECD Publishing, 2019.

S. Nakajima, T. Nakatani, and S. Takizawa, “AI Quality Assurance,” Journal of Information Processing, vol. 63, no. 11, pp. 602-605, pp. e1-e33, 2022. (In Japanese)

QA4AI Consortium, Guidelines for Quality Assurance of AI-based Products and Services, 2011.

S. Nakajima, Machine Learning Quality Issues Learned from Software Engineering, Tokyo: Maruzen, 2020. (In Japanese)

A. Karpathy, “Software 2.0,” https://karpathy.medium.com/software-2-0-a64152b37c35, November 12, 2017.

Systems and Software Engineering — Systems and Software Quality Requirements and Evaluation (SQuaRE) — Systems and Software Quality Model, ISO/IEC 25010, 2011.

H. Kuwajima and F. Ishikawa, “Adapting SQuaRE for Quality Assessment of Artificial Intelligence Systems,” IEEE International Symposium on Software Reliability Engineering Workshops, pp. 13-18, October 2019.

D. Natale, “Possible Extension of ISO/IEC 25000 Quality Models to Artificial Intelligence in the Context of an International Governance,” 27th Asia-Pacific Software Engineering Conference, pp. 22-24, December 2020.

S. Nakajima and T. Nakatani, “AI Extension of SQuaRE Data Quality Model,” IEEE 21st International Conference on Software Quality, Reliability and Security Companion, pp. 306-313, December 2021.

Software Engineering — Software Product Quality Requirements and Evaluation (SQuaRE) — Data Quality Model, ISO/IEC 25012, 2008.

Digital Architecture Research Center, “Machine Learning Quality Management Guideline, 3rd English Edition,” National Institute of Advanced Industrial Science and Technology, Technical Report DigiARC-TR-2023-01, January 20, 2023.

Software Engineering — Systems and Software Quality Requirements and Evaluation (SQuaRE) — Quality Model for AI Systems, ISO/IEC 25059, 2023.

“Ethics guidelines for trustworthy AI,” https://data.europa.eu/doi/10.2759/346720, 2019.

D. M. W. Powers, “Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation,” https://arxiv.org/pdf/2010.16061.pdf, October 11, 2020.

I. V. Tetko, D. J. Livingstone, and A. I. Luik, “Neural Network Studies. 1. Comparison of Overfitting and Overtraining,” Journal of Chemical Information and Computer Science, vol. 35, no. 5, pp. 826-833, September 1995.

R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection,” Proceedings of the 14th International Joint Conference on Artificial Intelligence, vol. 2, pp. 1137-1143, August 1995.

Y. Tsuzuki, H. Ohira, and S. Takahashi, “Techniques for Quantitative Evaluation of Noise Robustness of AI Models,” Toshiba Review, vol. 76, no. 3, pp. 44-47, May 2021. (In Japanese)

J. Cohen, E. Rosenfeld, and Z. Kolter, “Certified Adversarial Robustness via Randomized Smoothing,” Proceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 1310-1320, 2019.

I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and Harnessing Adversarial Examples,” https://arxiv.org/pdf/1412.6572.pdf, Mar 20, 2015.

Explainable Artificial Intelligence (XAI), Defense Advanced Research Projects Agency, DARPA-BAA-16-53, August 10, 2016.

Guidelines on Assessment of AI Reliability in the Field of Plant Safety, 2nd ed., 2021.

Japan Deep Learning Association (Ed.), The Official Textbook of the Deep Learning G Certificate, 2nd ed., Tokyo: Shoeisha, 2021. (In Japanese)