Kernel Functions-Introduction to SVM Kernel & Examples
In our previous Machine Learning blog we have discussed about SVM (Support Vector Machine) in Machine Learning. Now we are going to provide you a detailed description of SVM Kernel and Different Kernel Functions and its examples such as linear, nonlinear, polynomial, Gaussian kernel, Radial basis function (RBF), sigmoid etc.
2. SVM Kernel Functions
SVM algorithms use a set of mathematical functions that are defined as the kernel. The function of kernel is to take data as input and transform it into the required form. Different SVM algorithms use different types of kernel functions. These functions can be different types. For example linear, nonlinear, polynomial, radial basis function (RBF), and sigmoid.
Introduce Kernel functions for sequence data, graphs, text, images, as well as vectors. The most used type of kernel function is RBF. Because it has localized and finite response along the entire x-axis.
The kernel functions return the inner product between two points in a suitable feature space. Thus by defining a notion of similarity, with little computational cost even in very high-dimensional spaces.
3. Kernel Rules
Define kernel or a window function as follows:
This value of this function is 1 inside the closed ball of radius 1 centered at the origin, and 0 otherwise . As shown in the figure below:
For a fixed xi, the function is K(z-xi)/h) = 1 inside the closed ball of radius h centered at xi, and 0 otherwise as shown in the figure below:
So, by choosing the argument of K(·), you have moved the window to be centered at the point xi and to be of radius h.
4. Examples of SVM Kernels
Let us see some common kernels used with SVMs and their uses:
4.1. Polynomial kernel
It is popular in image processing.
where d is the degree of the polynomial.
4.2. Gaussian kernel
It is a general-purpose kernel; used when there is no prior knowledge about the data. Equation is:
4.3. Gaussian radial basis function (RBF)
It is a general-purpose kernel; used when there is no prior knowledge about the data.
Sometimes parametrized using:
4.4. Laplace RBF kernel
It is general-purpose kernel; used when there is no prior knowledge about the data.
4.5. Hyperbolic tangent kernel
We can use it in neural networks.
, for some (not every) k>0 and c<0.
4.6. Sigmoid kernel
We can use it as the proxy for neural networks. Equation is
4.7. Bessel function of the first kind Kernel
We can use it to remove the cross term in mathematical functions. Equation is :
where j is the Bessel function of first kind.
4.8. ANOVA radial basis kernel
We can use it in regression problems. Equation is:
4.9. Linear splines kernel in one-dimension
It is useful when dealing with large sparse data vectors. It is often used in text categorization. The splines kernel also performs well in regression problems. Equation is:
If you have any query about SVM Kernel Functions, So feel free to share with us. We will be glad to solve your queries.
Reference – Machine Learning