Identity activation structural tolerance online sequential circular extreme learning machine for highly dimensional data
Article Sidebar
Main Article Content
Abstract
The Structural Tolerance Online Sequential Circular Extreme Learning Machine (STOS-CELM) was developed based on the Circular Extreme Learning Machine (CELM) to allow sequential learning and to mitigate the criticality of deciding the number of hidden nodes with the Householder Block exact inverse QRD Recursive Least Squares (HBQRD-RLS) algorithm. A previous study showed significant efficiency improvement using STOS-CELM with sine activation. However, sine activation is periodic. Its periodicity repeatedly maps multiple values of its input to the same output values. Within the context of STOS-CELM, input of the activation is a non-negative real value corresponding to the closeness of the data to CELM kernels. Mapping this non-negative real value to a limited range with a periodic nature causes loss of inherent information. That could restrain the STOS-CELM from reaching its full potential. This article proposes an Identity Activation Structural Tolerance Online Sequential Circular Extreme Learning Machine (ISTOS-CELM) to improve the STOS-CELM by removing the sine function to relieve this issue. Our experimental results show that ISTOS-CELM provides significantly higher accuracy than STOS‑ELM and the original STOS-CELM, while retaining a comparable processing time and robustness to STOS-CELM.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Huang GB, Zhu QY, Siew CK. Extreme learning machines: theory and applications. Neurocomputing. 2006;70(1):489-501.
Huang GB, Wang DH, Lan Y. Extreme learning machines: a survey. Int J Mach Learn Cyb. 2011;2(2):107-22.
Huang GB, Zhou H, Ding X, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern. 2012;42(2):513-29.
Decherchi S, Gastaldo P, Zunino R, Cambria E, Redi J. Circular-elm for the reduced reference assessment of perceived image quality. Neurocomputing. 2013;102:78-89.
Ridella S, Rovetta S, Zunino R. Circular backpropagation networks for classification. IEEE Trans Neural Network. 1997;8(1):84-97.
Gastaldo P, Zunino R, Heynderickx I, Vicario E. Circular back-propagation networks for measuring displayed image quality. Lect Notes Comput Sci. 2002; 2415:1219-24.
Liang NY, Huang GB, Saratchandran P, Sundararajan N. A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Network. 2006;17(6):1411-23.
Atsawaraungsuk S, Katanyukul T. Sin activation structural tolerance of online sequential circular extreme learning machine. Int J Tech. 2017;8(4):601-10.
Apolinario JA, editor. QRD-RLS adaptive filtering. USA: Springer; 2009.
Pan CT, Plemmons R. Least squares modifications with inverse factorizations: parallel implications. J Comput Appl Math. 1989;27(1-2):109-27.
Moonen M, Vandewalle J. A square root covariance algorithm for constrained recursive least squares estimation. J VLSI Sign Process Syst Sign Image Video Technol. 1991;3(3):163-72.
Golub GH, Van Loan CF. Matrix computations. 4th ed. USA: Johns Hopkins University Press; 2013.
Trefethen LN, Bau III D. Numerical linear algebra. USA: Society for Industrial and Applied Mathematics; 1997.
AYRNA research group. AYRNA dataset [Internet]. 1994 [cited 2017 Sep]. Available from: http://www.uco.es/grupos/ayrna/index.php/en/investigacion-y-difusion/partitions-and-datasets
UCI machine learning repository. UCI dataset [Internet]. 1987 [cited 2017 Sep] Available from: https://archive.ics.uci.edu/ml/datasets.html.
Horata P, Chiewchanwattana S, Sunat K. Enhancement of online sequential extreme learning machine based on the householder block exact inverse qrd recursive least squares. Neurocomputing. 2015;149:239-52.
Stefani A, Xenos M. Meta-metric evaluation of e-commerce-related metrics. Electron Notes Theor Comput Sci. 2009;233:59-72.
Horata P, Chiewchanwattana S, Sunat K. Robust extreme learning machine. Neurocomputing. 2013; 102:31-44.
McCallum A, Nigam K. A comparison of event models for naive Bayes text classification. AAAI-98 workshop on learning for text categorization; 1998 Jul 26-27; Wisconsin, USA. USA: AAAI Press; 1998. p. 41-8.
LeCun Y, Boser BE, Denker JS, Henderson D, Howard RE, Hubbard WE, et al. Handwritten digit recognition with a back-propagation network. In: Touretzky DS, editor. Advances in neural information processing systems 2. San Francisco: Morgan Kaufmann Publishers; 1990. p. 396-404.
Joachims T. Text categorization with support vector machines: learning with many relevant features. In: Nédellec C, Rouveirol C, editors. 10th European Conference on Machine Learning; 1998 Apr 21-23; Chemnitz, Germany. Berlin: Springer; 1998. p. 137-42.