Adaptive Deep Neural Network for Solving Multiclass Problems
Article Sidebar
Main Article Content
Abstract
In recent years, multiclass classification has gained significant attention due to its wide-ranging applications in fields such as healthcare, finance, and image recognition. The ability to accurately classify data into multiple categories is essential for developing intelligent and robust systems. This research compares the performance of several machine learning and deep learning algorithms for multiclass classification tasks, with a focus on adaptive techniques in neural networks. The evaluated algorithms include Support Vector Machines (SVM), One-vs-Rest Logistic Regression (OvR-LR), Deep Neural Networks (DNN), Dropout-enhanced DNN, and Adaptive Regularization-based DNN. The experimental evaluation was conducted using both the train–test split and 5-fold cross-validation methods to ensure result reliability and generalizability. The Adaptive Regularization-DNN model achieved the highest performance among all tested approaches, with an accuracy of 98.75% under the train–test split and 97.3% under cross-validation. These results highlight the model’s robustness and its effectiveness in minimizing overfitting in structured multiclass classification problems. Performance metrics including precision, recall, F1-score, and accuracy were used to provide a comprehensive evaluation of each model’s capabilities.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All authors need to complete copyright transfer to Journal of Applied Informatics and Technology prior to publication. For more details click this link: https://ph01.tci-thaijo.org/index.php/jait/copyrightlicense
References
Abramovich, F., Grinshtein, V., and Levy, T. (2021). Multiclass classification by sparse multinomial logistic regression. IEEE Transactions on Information Theory, 67(7):4637–4646. DOI: 10.1109/TIT.2021.3075137.
Abuduweili, A., Li, X., Shi, H., Xu, C.-Z., and Dou, D. (2021). Adaptive consistency regularization for semi-supervised transfer learning. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6919–6928. DOI: 10.1109/CVPR46437.2021.00685.
Aksoy Ercan, S. and S¸im¸sek, M. (2023). Mobile phone price classification using machine learning. International Journal of Advanced Natural Sciences and Engineering Researches, 7(4):458–462. DOI: 10.59287/ijanser.791.
Bagla, P. and Kumar, K. (2023). Breaking down health fakes: A hybrid DNN model for multi-class classification on a self-constructed dataset. S¯adhan¯a, 48(4). DOI: 10.1007/s12046-023-02300-2.
Bejani, M. M. and Ghatee, M. (2021). A systematic review on overfitting control in shallow and deep neural networks. Artificial Intelligence Review, 54(8):6391–6438. DOI: 10.1007/s10462-021-09975-1.
Chen, L., Wang, Y., and Li, H. (2022). Enhancement of DNN-based multilabel classification by grouping labels based on data imbalance and label correlation. Pattern Recognition, 132:108964. DOI: 10.1016/j.patcog.2022.108964.
Chen, M. (2023). Mobile phone price prediction with feature reduction. Highlights in Science, Engineering and Technology, 34:155–162. DOI: 10.54097/hset.v34i.5440.
Choudhuri, R. and Paul, A. (2021). Multi class image classification for detection of diseases using chest X ray images. In 2021 8th International Conference on Computing for Sustainable Global Development (INDIACom), pages 769–773.
Dong, Q., Zhu, X., and Gong, S. (2018). Single-label multi-class image classification by deep logistic regression. CoRR, abs/1811.08400.
Feng, S., Zhao, C., and Fu, P. (2020). A deep neural network based hierarchical multi-label classification method. Review of Scientific Instruments, 91(2). DOI: 10.1063/1.5141161.
Gao, B.-B. and Zhou, H.-Y. (2021). Learning to discover multi-class attentional regions for multi-label image recognition. IEEE Transactions on Image Processing, 30:5920–5932. DOI: 10.1109/TIP.2021.3088605.
Garbin, C., Zhu, X., and Marques, O. (2020). Dropout vs. batch normalization: An empirical study of their impact to deep learning. Multimedia Tools and Applications, 79(19–20):12777–12815. DOI: 10.1007/s11042-019-08453-9.
Guo, Y., Zhang, Z., and Tang, F. (2021). Feature selection with kernelized multi-class support vector machine. Pattern Recognition, 117:107988. DOI: 10.1016/j.patcog.2021.107988.
G¨uven¸c, E., C¸ etin, G., and Ko¸cak, H. (2021). Comparison of knn and dnn classifiers performance in predicting mobile phone price ranges. Advances in Artificial Intelligence Research, 1(1):19–28.
Hu, N. (2022). Classification of mobile phone price dataset using machine learning algorithms. In 2022 3rd International Conference on Pattern Recognition and Machine Learning (PRML), pages 438–443. DOI: 10.1109/PRML56267.2022.9882236.
Hussain, S. F. and Ashraf, M. M. (2023). A novel one-vs-rest consensus learning method for crash severity prediction. Expert Systems with Applications, 228:120443. DOI: 10.1016/j.eswa.2023.120443.
Khan, A. H. and Zubair, M. (2020). Classification of multi-lingual tweets, into multi-class model using na¨ıve bayes and semi-supervised learning. Multimedia Tools and Applications, 79(43–44):32749–32767. DOI: 10.1007/s11042-020-09512-2.
Kumar Sahu, S. and Pandey, M. (2023). An optimal hybrid multiclass SVM for plant leaf disease detection using spatial fuzzy C-means model. Expert Systems with Applications, 214:118989. DOI: 10.1016/j.eswa.2022.118989.
Lee, S. and Lee, C. (2020). Revisiting spatial dropout for regularizing convolutional neural networks. Multimedia Tools and Applications, 79(45–46):34195–34207. DOI: 10.1007/s11042-020-09054-7.
Liu, Y., Yang, S., and Bao, Y. (2024). Deep neural network optimization based on binary method for handling multi-class problems. IEEE Access, 12:46881–46890. DOI: 10.1109/access.2024.3382195.
Liu, Y., Zhang, Z., Liu, X., Wang, L., and Xia, X. (2021). Deep learning-based image classification for online multi-coal and multi-class sorting. Computers & Geosciences, 157:104922. DOI: 10.1016/j.cageo.2021.104922.
Lv, X., Wu, D., and Xia, S.-T. (2020). Matrix smoothing: A regularization for Dnn with transition matrix under noisy labels. In 2020 IEEE International Conference on Multimedia and Expo (ICME), page 1–6. IEEE. DOI: 10.1109/icme46284.2020.9102853.
Manita, O. A., Peletier, M. A., Portegies, J. W., Sanders, J., and Senen-Cerda, A. (2022). Universal approximation in dropout neural networks. Journal of Machine Learning Research, 23(19):1–46.
Moral, P. D., Nowaczyk, S., and Pashami, S. (2022). Why is multiclass classification hard? 10:80448–80462. DOI: 10.1109/access.2022.3192514.
Ni, X., Fang, L., and Huttunen, H. (2021). Adaptive L2 regularization in person re-identification. In 2020 25th International Conference on Pattern Recognition (ICPR), page 9601–9607. IEEE. 10.1109/icpr48806.2021.9412481.
Panyatip, T., Phothinam, P., and Komgrich, O. (2022). Optimization classification of Covid-19 effect on liver cancer. Journal of Technology Management Rajabhat Maha Sarakham University, 9(2):66–78.
Park, J., Jung, Y., and Kim, J.-H. (2022). Multiclass classification fault diagnosis of multirotor UAVs utilizing a deep neural network. International Journal of Control, Automation and Systems, 20(4):1316–1326. DOI: 10.1007/s12555-021-0729-1.
Pawar, D. and Dhage, S. (2020). Multiclass covert speech classification using extreme learning machine. Biomedical Engineering Letters, 10(2):217–226. DOI: 10.1007/s13534-020-00152-x.
Pawara, P., Okafor, E., Groefsema, M., He, S., Schomaker, L. R., and Wiering, M. A. (2020). One-vs-One classification for deep neural networks. Pattern Recognition, 108:107528. DOI: 10.1016/j.patcog.2020.107528.
Pipalia, K. and Bhadja, R. (2020). Performance evaluation of different supervised learning algorithms for mobile price classification. International Journal for Research in Applied Science and Engineering Technology, 8(6):1841–1848. DOI: 10.22214/ijraset.2020.6302.
Rice, L., Wong, E., and Kolter, Z. (2020). Overfitting in adversarially robust deep learning. In III, H. D. and Singh, A., editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 8093–8104. PMLR.
Sunariya, N., Singh, A., Alam, M., and Gaur, V. (2024). Classification of mobile price using machine learning. In Symposium on Computing and Intelligent Systems 2024 (SCI 2024), pages 55–66.
Tang, L., Tian, Y., and Pardalos, P. M. (2019). A novel perspective on multiclass classification: Regular simplex support vector machine. Information Sciences, 480:324–338. DOI: 10.1016/j.ins.2018.12.026.
Thakkar, A. and Lohiya, R. (2021). Analyzing fusion of regularization techniques in the deep learning-based intrusion detection system. International Journal of Intelligent Systems, 36(12):7340–7388. DOI: 10.1002/int.22590.
Tiwari, P., Upadhyay, D., Pant, B., and Mohd, N. (2022). Multiclass Classification of Disease Using CNN and SVM of Medical Imaging, page 88–99. Springer International Publishing. DOI: 10.1007/978-3-031-12641-3 8.
Vernikou, S., Lyras, A., and Kanavos, A. (2022). Multiclass sentiment analysis on COVID-19-related tweets using deep learning models. Neural Computing and Applications, 34(22):19615–19627. DOI: 10.1007/s00521-022-07650-2.
Wang, T., Liu, L., Liu, N., Zhang, H., Zhang, L., and Feng, S. (2020). A multi-label text classification method via dynamic semantic representation model and deep neural network. Applied Intelligence, 50(8):2339–2351. DOI: 10.1007/s10489-020-01680-w.
Zehra, W., Javed, A. R., Jalil, Z., Khan, H. U., and Gadekallu, T. R. (2021). Cross corpus multi-lingual speech emotion recognition using ensemble learning. Complex & Intelligent Systems, 7(4):1845–1854. DOI: 10.1007/s40747-020-00250-4.
Zhao, H., Tsai, Y. H., Salakhutdinov, R., and Gordon, G. J. (2019). Learning neural networks with adaptive regularization. CoRR, abs/1907.06288.
Zhu, Y. and Liu, R. (2021). An algorithmic view of L2 regularization and some path-following algorithms. Journal of Machine Learning Research, 22(138):1–62.
