{"id":8100,"date":"2018-02-14T14:47:05","date_gmt":"2018-02-14T14:47:05","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=8100"},"modified":"2021-05-28T13:02:10","modified_gmt":"2021-05-28T07:32:10","slug":"data-mining-algorithms","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/","title":{"rendered":"Data Mining Algorithms &#8211; 13 Algorithms Used in Data Mining"},"content":{"rendered":"<p>In our last tutorial, we studied\u00a0 <strong>Data Mining Techniques<\/strong>. Today,\u00a0 we will learn Data Mining Algorithms.<\/p>\n<p>We will cover all types of Algorithms in Data Mining: Statistical Procedure Based Approach, Machine Learning-Based Approach, Neural Network, Classification Algorithms in Data Mining, ID3 Algorithm, C4.5 Algorithm, K Nearest Neighbors Algorithm, Na\u00efve Bayes Algorithm, SVM Algorithm, ANN Algorithm, 48 Decision Trees, Support Vector Machines, and SenseClusters.<\/p>\n<p>So, let&#8217;s start Data Mining Algorithms.<\/p>\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">What are Data Mining Algorithms?<\/h3>\n<p>There are too many Data Mining Algorithms present. We will discuss each of them one by one.<\/p>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">These are the examples, where the data analysis task is Classification\u00a0<span style=\"font-family: Georgia, Georgia, serif; font-weight: inherit;\">Algorithms in Data Mining-<\/span><\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">A bank loan officer wants to analyze the data <span class=\"complexword\">in order to<\/span> know which customer is risky or which are safe.<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">A marketing manager at a company needs to analyze a customer with a given profile, who will buy a new computer.<\/li>\n<\/ul>\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Why Algorithms Used In Data Mining?<\/h3>\n<p>Here, are some reason which gives the answer of usage of Data Mining Algorithms:<\/p>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">In today\u2019s world of \u201cbig data\u201d, a large database is becoming a norm. <span class=\"qualifier\">Just<\/span> imagine there present a database with many terabytes.<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">As Facebook alone crunches 600 terabytes of new data every single day. Also, the primary challenge of big data is how to make sense of it.<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Moreover, the sheer volume is not the only problem. Also, big data need to diverse, unstructure and fast changing. Consider audio and video data, social media posts, 3D data or geospatial data. This kind of data is not <span class=\"adverb\">easily<\/span> categorized or organized.<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Further, to meet this challenge, a range of automatic methods for extracting information.<\/li>\n<\/ul>\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Types of Algorithms In Data Mining<\/h3>\n<p>Here, 13 Data Mining Algorithms are discussed-<\/p>\n<div id=\"attachment_8450\" style=\"width: 1210px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Classification-Methods-01.jpg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-8450\" class=\"wp-image-8450 size-full\" src=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Classification-Methods-01.jpg\" alt=\"Data Mining Algorithms\" width=\"1200\" height=\"628\" srcset=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Classification-Methods-01.jpg 1200w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Classification-Methods-01-150x79.jpg 150w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Classification-Methods-01-300x157.jpg 300w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Classification-Methods-01-768x402.jpg 768w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Classification-Methods-01-1024x536.jpg 1024w\" sizes=\"auto, (max-width: 1200px) 100vw, 1200px\" \/><\/a><p id=\"caption-attachment-8450\" class=\"wp-caption-text\">Data Mining Algorithms &#8211; Types<\/p><\/div>\n<h4>a. Statistical Procedure Based Approach<\/h4>\n<p>There are two main phases present to work on classification. That can <span class=\"passivevoice\">easily identify<\/span> the statistical community.<\/p>\n<p>The second, \u201cmodern\u201d phase concentrated on more flexible classes of models. In which many of which attempt has to take. That provides an estimate of the joint distribution of the feature within each class. That can, in turn, provide a classification rule.<\/p>\n<p>Generally, statistical procedures have to<span class=\"passivevoice\"> characterize by<\/span> having a precise fundamental probability model. That used to provides a probability of being in each class instead of <span class=\"qualifier\">just<\/span> a classification. Also, we can assume that techniques will <span class=\"passivevoice\">use by<\/span> statisticians.<\/p>\n<p>Hence some human involvement has to<span class=\"passivevoice\"> assume<\/span> with regard to variable selection. Also, transformation and <span class=\"complexword\">overall<\/span> structuring of the problem.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">b. Machine Learning-Based Approach<\/h4>\n<p>Generally, it covers automatic computing procedures. That <span class=\"passivevoice\">was based<\/span> on logical or binary operations. That use to learn a task from a series of examples.<\/p>\n<p>Here, we have to focus on decision-tree approaches. As classification results come from a sequence of logical steps. These classification results are capable of representing the most complex problem given. Such as genetic algorithms and inductive logic procedures (I.LP.) are currently under active improvement.<\/p>\n<p>Also, its principle would allow us to deal with more general types of data including cases. In which the number and type of attributes may vary.<\/p>\n<p>This approach aims to generate classifying expressions. That is simple enough to <span class=\"passivevoice\">understand by<\/span> the human. And must mimic human reasoning to provide insight into the decision process.<\/p>\n<p>Like statistical approaches, background knowledge may <span class=\"passivevoice\">use<\/span>\u00a0in development. But the operation <span class=\"passivevoice\">is assumed<\/span> without human interference.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">c. Neural Network<\/h4>\n<p>The field of Neural Networks has arisen from diverse sources. That is ranging from understanding and emulating the human brain to broader issues. That is of copying human abilities such as speech and\u00a0<span class=\"passivevoice\">use<\/span>\u00a0in various fields. Such as banking, in classification program to categorize data as intrusive or normal.<\/p>\n<p>Generally, neural networks consist of layers of interconnected nodes. That each node producing a non-linear function of its input. And input to a node may come from other nodes or <span class=\"adverb\">directly<\/span> from the input data. Also, some nodes <span class=\"passivevoice\">are identified<\/span> with the output of the network.<\/p>\n<p>On the basis of this, there are different applications for neural networks present. That involve recognizing patterns and making simple decisions about them.<\/p>\n<p>In airplanes, we can use a neural network as a basic autopilot. In which input units read signals from the various instruments and output units. That modifying the plane\u2019s controls <span class=\"adverb\">appropriately<\/span> to keep it <span class=\"adverb\">safely<\/span> on course. Inside a factory, we can use a neural network for quality control.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">d. Classification Algorithms in Data Mining<\/h4>\n<p>It is one of the Data Mining. That <span class=\"passivevoice\">is used<\/span> to analyze a given data set and takes each instance of it. It assigns this instance to a particular class. Such that classification error will be least. It <span class=\"passivevoice\">is used<\/span> to extract models. That define important data classes within the given data set. Classification is a two-step process.<\/p>\n<p>During the first step, the model <span class=\"passivevoice\">is created by<\/span> applying a classification algorithm. That is on training data set.<\/p>\n<p>Then in the second step, the extracted model <span class=\"passivevoice\">is tested<\/span> against a predefined test data set. That is to measure the model trained performance and accuracy. So classification is the process to assign class label from a data set whose class label is unknown.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">e. ID3 Algorithm<\/h4>\n<p>This Data Mining Algorithms starts with the original set as the root hub. On every cycle, it emphasizes through every unused attribute of the set and figures. That the entropy of attribute. At that point chooses the attribute. That has the smallest entropy value.<\/p>\n<p>The set is S then split by the selected attribute to produce subsets of the information.<\/p>\n<p>This Data Mining algorithms proceed to recurse on each item in a subset. Also, considering only items never selected before. Recursion on a subset may bring to a halt in one of these cases:<\/p>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Every element in the subset belongs to the same class (+ or -), then the node <span class=\"passivevoice\">is turned<\/span> into a leaf and<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">labeled with the class of the examples<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">If there are no more attributes to s<span class=\"passivevoice\">elect<\/span>\u00a0but the examples still do not belong to the same class. Then the node <span class=\"passivevoice\">is turned<\/span> into a leaf and labeled with the most common class of the examples in that subset.<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">If there are no examples in the subset, then this happens. Whenever parent set found to be matching a specific value of the selected attribute.<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">For example, if there was no example matching with marks &gt;=100. Then a leaf <span class=\"passivevoice\">is created<\/span> and <span class=\"passivevoice\">is labeled<\/span> with the most common class of the examples in the parent set.<\/li>\n<\/ul>\n<p><strong>Working steps of Data Mining Algorithms is as follows,<\/strong><\/p>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Calculate the entropy for each attribute using the data set S.<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Split the set S into subsets using the attribute for which entropy is <span class=\"complexword\">minimum<\/span>.<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Construct a decision tree node containing that attribute in a dataset.<\/li>\n<\/ul>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Recurse on each member of subsets using remaining attributes.<\/li>\n<\/ul>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">f. C4.5 Algorithm<\/h4>\n<p>C4.5 is one of the most important Data Mining algorithms, used to produce a decision tree which is an expansion of prior ID3 calculation. It enhances the ID3 algorithm. That is by managing both continuous and discrete properties, missing values. The decision trees created by C4.5. that <span class=\"passivevoice\">use<\/span>\u00a0for grouping and often referred to as a statistical classifier.<\/p>\n<p>C4.5 creates decision trees from a set of training data same way as an Id3 algorithm. As it is a supervised learning algorithm it requires a set of training examples. That can\u00a0<span class=\"passivevoice\">see<\/span>\u00a0as a pair: input object and the desired output value (class).<\/p>\n<p>The algorithm analyzes the training set and builds a classifier. That must have the capacity to <span class=\"adverb\">accurately<\/span> arrange both training and test cases.<\/p>\n<p>A test example is an input object and the algorithm must predict an output value. Consider the sample training data set S=S1, S2,\u2026Sn which is already classified.<\/p>\n<p>Each sample Si consists of feature vector (x1,i, x2,i, \u2026, xn,i). Where xj represent attributes or features of the sample. The class in which Si falls. At each node of the tree, C4.5 selects one attribute of the data. That most <span class=\"adverb\">efficiently<\/span> splits its set of samples into subsets such that it results in one class or the other.<\/p>\n<p>The splitting condition is the normalized information gain. That is a non-symmetric measure of the difference. The attribute with the highest information gain <span class=\"passivevoice\">is chosen<\/span> to make the decision. General working steps of algorithm is as follows,<\/p>\n<p>Assume all the samples in the list belong to the same class. If it is true, it <span class=\"adverb\">simply<\/span> creates a leaf node for the decision tree so that particular class will <span class=\"passivevoice\">select<\/span>.<\/p>\n<p>None of the features provide any information gain. If it is true, C4.5 creates a decision node higher up the tree using the expected value of the class.<\/p>\n<p>An instance of previously-unseen class encountered. Then, C4.5 creates a decision node higher up the tree using the expected value.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">g. K Nearest Neighbors Algorithm<\/h4>\n<p>The closest neighbor rule distinguishes the classification of an unknown data point. That is on the basis of its closest neighbor whose class is already known.<\/p>\n<p>M. Cover and P. E. Hart purpose k nearest neighbor (KNN). In which nearest neighbor <span class=\"passivevoice\">is computed<\/span> on the basis of estimation of k. That indicates how many nearest neighbors are to c<span class=\"passivevoice\">onsider<\/span>\u00a0to characterize.<\/p>\n<p>It makes use of the more than one closest neighbor to determine the class. In which the given data point belongs to and so it <span class=\"passivevoice\">is called<\/span> as KNN. These data samples <span class=\"passivevoice\">are needed<\/span> to be in the memory at the runtime.<br \/>\nHence they <span class=\"passivevoice\">are referred<\/span> to as memory-based technique.<\/p>\n<p>Bailey and A. K. Jain enhances KNN which <span class=\"passivevoice\">is focused<\/span> on weights. The training points <span class=\"passivevoice\">are assigned<\/span> weights. According to their distances from sample data point. But at the same, computational complexity and memory requirements remain the primary concern.<\/p>\n<p>To overcome memory limitation size of data set <span class=\"passivevoice\">is reduced<\/span>. For this, the repeated patterns. That don\u2019t include <span class=\"complexword\">additional<\/span> data are also eliminated from training data set.<\/p>\n<p>To further enhance the information focuses which don\u2019t influence the result. That are additionally eliminated from training data set.<\/p>\n<p>The NN training data set can\u00a0<span class=\"passivevoice\">organize<\/span>\u00a0utilizing different systems. That is to enhance over memory limit of KNN. The KNN implementation can<span class=\"passivevoice\">\u00a0do<\/span>\u00a0using ball tree, k-d tree, and orthogonal search tree.<\/p>\n<p>The tree-structured training data is further divided into nodes and techniques. Such as NFL and tunable metric divide the training data set according to planes. Using these algorithms we can expand the speed of basic KNN algorithm. Consider that an object <span class=\"passivevoice\">is sampled<\/span> with a set of different attributes.<\/p>\n<p>Assuming its group can <span class=\"passivevoice\">determine<\/span>\u00a0from its attributes. Also, different algorithms can <span class=\"passivevoice\">use<\/span>\u00a0to automate the classification process. In pseudo code, k-nearest neighbor algorithm can <span class=\"passivevoice\">express<\/span>,<\/p>\n<p>K \u2190 number of nearest neighbors<\/p>\n<p>For each object Xin the test set do<\/p>\n<p>calculate the distance D(X,Y) between X and every object Y in the training set<\/p>\n<p>neighborhood \u2190 the k neighbors in the training set closest to X<\/p>\n<p>X.class \u2190 SelectClass (neighborhood)<\/p>\n<p>End for<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">h. Na\u00efve Bayes Algorithm<\/h4>\n<p>The Naive Bayes Classifier technique <span class=\"passivevoice\">is based<\/span> on the Bayesian theorem. It is particularly used when the dimensionality of the inputs is high.<\/p>\n<p>The Bayesian Classifier is capable of calculating the possible output. That <span class=\"passivevoice\">is based<\/span> on the input. It is also possible to add new raw data at runtime and have a better probabilistic classifier.<\/p>\n<p>This classifier considers the presence of a particular feature of a class. That <span class=\"passivevoice\">is unrelated<\/span> to the presence of any other feature when the class variable <span class=\"passivevoice\">is given<\/span>.<\/p>\n<p>For example, a fruit may <span class=\"passivevoice\">consider<\/span>\u00a0to be an apple if it <span class=\"passivevoice\">is red<\/span>, round.<\/p>\n<p>Even if these features depend on each other features of a class. A naive Bayes classifier considers all these properties to contribute to the probability. That it shows this fruit is an apple. Algorithm works as follows,<\/p>\n<p>Bayes theorem provides a way of calculating the posterior probability, P(c|x), from P(c), P(x), and P(x|c). Naive Bayes classifier considers the effect of the value of a predictor (x) on a given class (c). That is independent of the values of other predictors.<\/p>\n<p>P(c|x) is the posterior probability of class (target) given predictor (attribute) of class.<\/p>\n<p>P(c) <span class=\"passivevoice\">is called<\/span> the prior probability of class.<\/p>\n<p>P(x|c) is the likelihood which is the probability of predictor of given class.<\/p>\n<p>P(x) is the prior probability of predictor of class.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">i. SVM Algorithm<\/h4>\n<p>SVM has attracted a great deal of attention in the last decade. It also applied to various domains of applications. SVMs <span class=\"passivevoice\">are used<\/span> for learning classification, regression or ranking function.<\/p>\n<p>SVM <span class=\"passivevoice\">is based<\/span> on statistical learning theory and structural risk minimization principle. And have the aim of determining the location of decision boundaries. It is also known as a hyperplane. That produces the optimal separation of classes. Thereby creating the largest possible distance between the separating hyperplane.<\/p>\n<p>Further, the instances on either side of it have <span class=\"passivevoice\">been proven<\/span>. That is to reduce an upper bound on the expected generalization error.<\/p>\n<p>The efficiency of SVM based does not depend on the dimension of classified entities. Though, SVM is the most robust and accurate classification technique. Also, there are several problems.<\/p>\n<p>The data analysis in SVM <span class=\"passivevoice\">is based<\/span> on convex quadratic programming. Also, expensive, as solving quadratic programming methods. That need large matrix operations as well as time-consuming numerical computations.<\/p>\n<p>Training time for SVM scales in the number of examples. So researchers strive all the time for more efficient training algorithm. That resulting in several variant based algorithm.<\/p>\n<p>SVM can also\u00a0<span class=\"passivevoice\">extend<\/span>\u00a0to learn non-linear decision functions. That is by first projecting the input data onto a high-dimensional feature space. As by using kernel functions and formulating a linear classification problem. The resulting feature space is much larger than the size of a dataset. That is not possible to store on popular computers.<\/p>\n<p>Investigation of this issues leads to several decomposition based algorithms. The basic idea of decomposition method is to split the variables into two parts:<\/p>\n<p>a set of free variables called as a working set. That can <span class=\"passivevoice\">update<\/span>\u00a0in each iteration and set of fixed variables. That <span class=\"passivevoice\">are fix during<\/span> a particular. Now, this procedure have to repeat until the termination conditions <span class=\"passivevoice\">are met<\/span><\/p>\n<p>The SVM <span class=\"passivevoice\">was developed<\/span> for binary classification. And it is not simple to extend it for multi-class classification problem. The basic idea to apply multi-classification to SVM. That is to decompose the multi-class problems into several two-class problems. That can <span class=\"passivevoice\">address<\/span>\u00a0using several SVMs.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">J. ANN Algorithm<\/h4>\n<p>This is the types of computer architecture inspire by biological neural networks. They <span class=\"passivevoice\">are used<\/span> to approximate functions. That can depend on a large number of inputs and are generally unknown.<\/p>\n<p>They <span class=\"passivevoice\">are presented<\/span> as systems of interconnected \u201cneurons\u201d. That can compute values from inputs. Also, they are capable of machine learning as well as pattern recognition. Due to their adaptive nature.<\/p>\n<p>An artificial neural network operates by creating connections between many different processing elements. That each corresponding to a single neuron in a biological brain. These neurons may actually construct or simulate by a digital computer system.<\/p>\n<p>Each neuron takes many input signals. Then based on an internal weighting. That produces a single output signal that <span class=\"passivevoice\">is sent<\/span> as input to another neuron.<\/p>\n<p>The neurons <span class=\"passivevoice\">are interconnected<\/span> and organized into different layers. The input layer receives the input and the output layer produces the final output.<\/p>\n<p>In general, one or more hidden layers <span class=\"passivevoice\">are sandwiched<\/span> between the two. This structure makes it impossible to forecast or know the exact flow of data.<\/p>\n<p>Artificial neural networks start out with randomized weights for all their neurons. This means that they need to<span class=\"passivevoice\"> train<\/span> to solve the particular problem for which they <span class=\"passivevoice\">are proposed<\/span>. A back-propagation ANN <span class=\"passivevoice\">is trained by<\/span> humans to perform specific tasks.<\/p>\n<p>During the training period, we can test whether the ANN\u2019s output is correct by observing a pattern. If it\u2019s correct the neural weightings produce that output <span class=\"passivevoice\">is reinforced<\/span>. if the output is incorrect, those weightings responsible\u00a0<span class=\"passivevoice\">diminish<\/span>.<\/p>\n<p>Implemented on a single computer, a network is slower than more traditional solutions. The ANN\u2019s parallel nature allows it to <span class=\"passivevoice\">built<\/span> using many processors. That gives a great speed advantage at very little development cost.<\/p>\n<p>The parallel architecture allows ANNs to process amounts of data very in less time. It deals with large continuous streams of information. Such as speech recognition or machine sensor data. ANNs can operate faster as compared to other algorithms.<\/p>\n<p>An artificial neural network is useful in a variety of real-world applications. Such as visual pattern recognition and speech recognition. That deals with complex often incomplete data.<\/p>\n<p>Also, recent programs for text-to-speech have utilized ANNs. Many handwriting analysis programs are currently using ANNs.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">K. 48 Decision Trees<\/h4>\n<p>A decision tree is a predictive machine-learning model. That decides the target value of a new sample. That based on various attribute values of the available data. The internal nodes of a decision tree denote the different attributes.<\/p>\n<p>Also, the branches between the nodes tell us the possible values. That these attributes can have in the observed samples. While the terminal nodes tell us the final value of the dependent variable.<\/p>\n<p>The attribute is to\u00a0<span class=\"passivevoice\">predict<\/span>\u00a0<span class=\"passivevoice\">is known<\/span> as the dependent variable. Since its value depends upon, the values of all the other attributes. The other attributes, which help in predicting the value of the dependent variable. That <span class=\"passivevoice\">are <\/span>the independent variables in the dataset.<\/p>\n<p>The J48 Decision tree classifier follows the following simple algorithm. To classify a new item, it first needs to create a decision tree. That based on the attribute values of the available training data.<\/p>\n<p>So, whenever it encounters a set of items. Then it identifies the attribute that discriminates the various instances most <span class=\"adverb\">clearly<\/span>.<\/p>\n<p>This feature is able to tell us most about the data instances. So that we can classify them the best <span class=\"passivevoice\">is said<\/span> to have the highest information gain.<\/p>\n<p>Now, among the possible values of this feature. If there is any value for which there is no ambiguity. That is, for which the data instances falling within its category. It has the same value for the target variable. Then we stop that branch and assign to it the target value that we have obtained.<\/p>\n<p>For other cases, we look for another attribute that gives us the highest information gain. We continue to get a clear decision. That of what combination of attributes gives us a particular target value.<\/p>\n<p>In the event that we run out of attributes. If we cannot get an unambiguous result from the available information. We assign this branch a target value that the majority of the items under this branch own.<\/p>\n<p>Now that we have the decision tree, we follow the order of attribute selection as we have obtained for the tree. By checking all the respective attributes. And their values with those seen in the decision tree model. we can assign or predict the target value of this new instance.<\/p>\n<p>The above description will be more clear and easier to understand with the help of an example. Hence, let us see an example of J48 decision tree classification.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">l. Support Vector Machines<\/h4>\n<p>Support Vector Machines <span class=\"passivevoice\">are supervised<\/span> learning methods. That used for classification, as well as regression. The advantage of this is that they can make use of certain kernels to transform the problem. Such that we can apply linear classification techniques to non-linear data.<\/p>\n<p>Applying the kernel equations. That arranges the data instances in a way within the multi-dimensional space. That there is a hyperplane that separates data instances of one kind from those of another.<\/p>\n<p>The kernel equations may be any function. That transforms the non-separable data in one domain into another domain. In which the instances become separable. Kernel equations may be linear, quadratic, Gaussian, or anything else. That achieves this particular purpose.<\/p>\n<p>Once we manage to divide the data into two distinct categories, our aim is to get the best hyperplane. That is to separate the two types of instances. This hyperplane is important, it decides the target variable value for future predictions. We should decide upon a hyperplane that maximizes the margin. That is between the support vectors on either side of the plane.<\/p>\n<p>Support vectors are those instances that are either on the separating planes. The explanatory diagrams that follow will make these ideas a little more clear.<\/p>\n<p>In Support Vector Machines the data need to <span class=\"passivevoice\">be separate<\/span> to be binary. Even if the data is not binary, these machines handle it as though it is. Further completes the analysis through a series of binary assessments on the data.<\/p>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">M. SenseClusters (an adaptation of the K-means clustering algorithm)<\/h4>\n<p>We have made use of SenseClusters to classify the email messages. SenseCluster available package of Perl programs. As it <span class=\"passivevoice\">was developed<\/span> at the University of Minnesota Duluth. That we<span class=\"passivevoice\"> use<\/span> for automatic text and document classification. The advantage of SenseClusters is that it does not need any training data;<\/p>\n<p>It makes use of unsupervised learning methods to classify the available data.<\/p>\n<p>Now, particularly in this section will understand the K-means clustering algorithm. That has <span class=\"passivevoice\">been used<\/span> in SenseClusters. Clustering is the process in which we divide the available data. That instances of a given number of sub-groups. These sub-groups <span class=\"passivevoice\">are <\/span>clusters, and hence the name \u201cClustering\u201d.<\/p>\n<p>To put it, the K-means algorithm outlines a method. That is to cluster a particular set of instances into K different clusters. Where K is a positive integer. It should notice K-means clustering algorithm requires a number of clusters from the user. It cannot identify the number of clusters by itself.<\/p>\n<p><span class=\"complexword\">However<\/span>, SenseClusters has the facility of identifying the number of clusters. That the data may comprise of.<\/p>\n<p>The K-means clustering algorithm starts by placing K centroids. Then each of the available data instances has to<span class=\"passivevoice\"> assign<\/span> a particular centroid. That depends on a metric like Euclidian distance, Manhattan distance, Minkowski distance, etc.<\/p>\n<p>The position of the centroid has to<span class=\"passivevoice\"> recalculate<\/span> every time an instance <span class=\"passivevoice\">is added<\/span> to the cluster. This continues until all the instances <span class=\"passivevoice\">are group<\/span> into the final required clusters.<\/p>\n<p>Since recalculating the cluster centroids may alter the cluster membership. Also, cluster memberships are also verified once the position of the centroid changes.<\/p>\n<p>This process continues till there is no further change in the cluster membership. And there is as little change in the positions of the centroids as possible.<\/p>\n<p>The initial position of the centroids is thus very important. Since this position affects all the future steps in the K-means clustering algorithm. Hence, it is always advisable to keep the cluster centers as far away from each other as possible.<\/p>\n<p>If there are too many clusters, then clusters resemble each other. And they are in the vicinity of each other that need to be<span class=\"passivevoice\"> club<\/span> together.<\/p>\n<p>Moreover, if there are few clusters then clusters that are too big. And may contain two or more sub-groups of different data instances <span class=\"passivevoice\">that must be a divide<\/span>.<\/p>\n<p>The K-means clustering algorithm is thus a simple to understand. Also, a method by which we can divide the available data into sub-categories.<\/p>\n<p>So, this was all about Data Mining\u00a0Algorithms. Hope you like our explanation.<\/p>\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Conclusion<\/h3>\n<p>As a result, we have studied Data Mining Algorithms. Also, we have learned each type of Data Mining algorithm.\u00a0 Furthermore, if you feel any query, feel free to ask in a comment section.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In our last tutorial, we studied\u00a0 Data Mining Techniques. Today,\u00a0 we will learn Data Mining Algorithms. We will cover all types of Algorithms in Data Mining: Statistical Procedure Based Approach, Machine Learning-Based Approach, Neural&#46;&#46;&#46;<\/p>\n","protected":false},"author":6,"featured_media":34337,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[18],"tags":[137,160,707,2267,2548,3369,6437,7837,8042,8444,8995,9052,12726,13986,14001],"class_list":["post-8100","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-mining","tag-48-decision-trees","tag-a-study-of-classification-techniques","tag-ann-algorithm","tag-c4-5-algorithm","tag-classification-in-data-mining","tag-data-mining-techniques","tag-id3-algorithm","tag-k-nearest-neighbors-algorithm","tag-knn-algorithm","tag-machine-learning-based-approach","tag-naive-bayes-algorithm","tag-neural-network","tag-senseclusters","tag-support-vector-machines","tag-svm-algorithm"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Data Mining Algorithms - 13 Algorithms Used in Data Mining - DataFlair<\/title>\n<meta name=\"description\" content=\"Data Mining Algorithms- What is Classification,Types of Classification methods,ID3 Algorithm, C4.5 Algorithm,SVM,ANN Algorithm\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data Mining Algorithms - 13 Algorithms Used in Data Mining - DataFlair\" \/>\n<meta property=\"og:description\" content=\"Data Mining Algorithms- What is Classification,Types of Classification methods,ID3 Algorithm, C4.5 Algorithm,SVM,ANN Algorithm\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-02-14T14:47:05+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-05-28T07:32:10+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Data-Mining-Algorithms.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"18 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Mining Algorithms - 13 Algorithms Used in Data Mining - DataFlair","description":"Data Mining Algorithms- What is Classification,Types of Classification methods,ID3 Algorithm, C4.5 Algorithm,SVM,ANN Algorithm","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/","og_locale":"en_US","og_type":"article","og_title":"Data Mining Algorithms - 13 Algorithms Used in Data Mining - DataFlair","og_description":"Data Mining Algorithms- What is Classification,Types of Classification methods,ID3 Algorithm, C4.5 Algorithm,SVM,ANN Algorithm","og_url":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-02-14T14:47:05+00:00","article_modified_time":"2021-05-28T07:32:10+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Data-Mining-Algorithms.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"18 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89"},"headline":"Data Mining Algorithms &#8211; 13 Algorithms Used in Data Mining","datePublished":"2018-02-14T14:47:05+00:00","dateModified":"2021-05-28T07:32:10+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/"},"wordCount":3916,"commentCount":1,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Data-Mining-Algorithms.jpg","keywords":["48 Decision Trees","A study of classification techniques","ANN Algorithm","C4.5 Algorithm","classification in data mining","Data Mining Techniques","ID3 Algorithm","K Nearest Neighbors Algorithm","KNN Algorithm","Machine Learning Based Approach","Na\u00efve Bayes Algorithm","Neural network","SenseClusters","Support Vector Machines","SVM Algorithm"],"articleSection":["Data Mining Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/","url":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/","name":"Data Mining Algorithms - 13 Algorithms Used in Data Mining - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Data-Mining-Algorithms.jpg","datePublished":"2018-02-14T14:47:05+00:00","dateModified":"2021-05-28T07:32:10+00:00","description":"Data Mining Algorithms- What is Classification,Types of Classification methods,ID3 Algorithm, C4.5 Algorithm,SVM,ANN Algorithm","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Data-Mining-Algorithms.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Data-Mining-Algorithms.jpg","width":1200,"height":628,"caption":"Data Mining Algorithms - 13 Algorithms Used in Data Mining"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/data-mining-algorithms\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Data Mining Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/data-mining\/"},{"@type":"ListItem","position":3,"name":"Data Mining Algorithms &#8211; 13 Algorithms Used in Data Mining"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"The DataFlair Team provides industry-driven content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our expert educators focus on delivering value-packed, easy-to-follow resources for tech enthusiasts and professionals.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam2\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/8100","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=8100"}],"version-history":[{"count":12,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/8100\/revisions"}],"predecessor-version":[{"id":96129,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/8100\/revisions\/96129"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/34337"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=8100"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=8100"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=8100"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}