An Overview of Online Learning in Reproducing Kernel Hilbert Spaces

Abstract

Learning System is a method to approximate an underlying function from a finite observation data. Since batch learning has a disadvantage in dealing with large data set, online learning is proposed to prevent the computational expensive. Iterative method called Stochastic Gradient Descent (SGD) is applied to solve for the underlying function on reproducing kernel Hilbert spaces (RKHSs). RKHS is widely used in many applications such as kernel method, radial basis function neural networks, Volterra filers and estimation of bandlimited functions. This approach has advantages that there is no local minima problem and convergence is also guaranteed because of using convex optimisation.This paper aims to provide background and theory of learning in RKHS which online kernel method is our main interest. The experiments show the results of learning from 3 test sets and some important parameters are also discussed.