Kernel Functions-Introduction to SVM Kernel & Examples

1. Objective

In our previous Machine Learning blog we have discussed about SVM (Support Vector Machine) in Machine Learning. Now we are going to provide you a detailed description of SVM Kernel and Different Kernel Functions and its examples such as linear, nonlinear, polynomial, Gaussian kernel, Radial basis function (RBF), sigmoid etc.

Kernel Functions-Introduction to SVM Kernel & Examples

Kernel Functions-Introduction to SVM Kernel & Examples

2. SVM Kernel Functions

SVM algorithms use a set of mathematical functions that are defined as the kernel. The function of kernel is to take data as input and transform it into the required form. Different SVM algorithms use different types of kernel functions. These functions can be different types. For example linear, nonlinear, polynomial, radial basis function (RBF), and sigmoid.
Introduce Kernel functions for sequence data, graphs, text, images, as well as vectors. The most used type of kernel function is RBF. Because it has localized and finite response along the entire x-axis.
The kernel functions return the inner product between two points in a suitable feature space. Thus by defining a notion of similarity, with little computational cost even in very high-dimensional spaces.

3. Kernel Rules

Define kernel or a window function as follows:

Kernel or a window function

Kernel or a window function

This value of this function is 1 inside the closed ball of radius 1 centered at the origin, and 0 otherwise . As shown in the figure below:

Kernel or a window function

Kernel or a window function

For a fixed xi, the function is K(z-xi)/h) = 1 inside the closed ball of radius h centered at xi, and 0 otherwise as shown in the figure below:

Kernel or a window function

Kernel or a window function

So, by choosing the argument of K(·), you have moved the window to be centered at the point xi and to be of radius h.

4. Examples of SVM Kernels

Let us see some common kernels used with SVMs and their uses:

4.1. Polynomial kernel

It is popular in image processing.
Equation is:

Polynomial kernel equation

Polynomial kernel equation

where d is the degree of the polynomial.

4.2. Gaussian kernel

It is a general-purpose kernel; used when there is no prior knowledge about the data. Equation is:

Gaussian kernel equation

Gaussian kernel equation

4.3. Gaussian radial basis function (RBF)

It is a general-purpose kernel; used when there is no prior knowledge about the data.
Equation is:

Gaussian radial basis function (RBF)

Gaussian radial basis function (RBF)

, for:

Gaussian radial basis function (RBF)

Gaussian radial basis function (RBF)

Sometimes parametrized using:

Gaussian radial basis function (RBF)

Gaussian radial basis function (RBF)

4.4. Laplace RBF kernel

It is general-purpose kernel; used when there is no prior knowledge about the data.
Equation is:

Laplace RBF kernel equation

Laplace RBF kernel equation

4.5. Hyperbolic tangent kernel

We can use it in neural networks.
Equation is:

Hyperbolic tangent kernel equation

Hyperbolic tangent kernel equation

, for some (not every) k>0 and c<0.

4.6. Sigmoid kernel

We can use it as the proxy for neural networks. Equation is

 Sigmoid kernel equation

Sigmoid kernel equation

4.7. Bessel function of the first kind Kernel

We can use it to remove the cross term in mathematical functions. Equation is :

Equation of Bessel function of the first kind kernel

Equation of Bessel function of the first kind kernel

where j is the Bessel function of first kind.

4.8. ANOVA radial basis kernel

We can use it in regression problems. Equation is:

ANOVA radial basis kernel equation

ANOVA radial basis kernel equation

4.9. Linear splines kernel in one-dimension

It is useful when dealing with large sparse data vectors. It is often used in text categorization. The splines kernel also performs well in regression problems. Equation is:

Linear splines kernel equation in one-dimension

Linear splines kernel equation in one-dimension

If you have any query about SVM Kernel Functions, So feel free to share with us. We will be glad to solve your queries.
See Also-

Reference – Machine Learning

6 Responses

  1. K. Chandra Hari says:

    How the optimum input data set is decided?
    When the input data is not characterized for information content say as in Principal Component Analysis, how the hyperplanes spoken of in SVM can be of utility?
    Can the Kernal functions add new information into the system?
    For any input space, how do you decide that the number of variables construed are efficient and not redundant data leading to fudging of the solution?

  2. Niaz ghumro says:

    Very nice information about kernel it is very informative blog.

    • DataFlair Team says:

      Hi Niaz,
      Thanks for connecting DataFlair via SVM Kernel Functions Tutorial. Hope, you are enjoying other Machine Learning tutorials. Please refer our sidebar to maintain your flow of learning.
      Regards,
      Dataflair

  3. dereje says:

    in the gaussian kernel function, what x and y represents?

    • DataFlair Team says:

      Hi Dereje,
      In Gaussian Kernel Function, x and y are the two feature vectors in the input space whose Euclidean Distance is calculated. Hope, it helps!

  4. Rasmi Ranjan Das says:

    Can anyone help me in writing Python Code for SVM with Multiple Kernels

Leave a Reply

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.