

{"id":8899,"date":"2018-02-21T12:09:14","date_gmt":"2018-02-21T12:09:14","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=8899"},"modified":"2025-07-27T20:05:47","modified_gmt":"2025-07-27T14:35:47","slug":"dimensionality-reduction-tutorial","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/","title":{"rendered":"What is Dimensionality Reduction &#8211; Techniques, Methods, Components"},"content":{"rendered":"<div>\n<div class=\"\">\n<p>In this <a href=\"https:\/\/data-flair.training\/blogs\/machine-learning-tutorial\/\"><strong>Machine Learning Tutorial<\/strong><\/a>, we will study What is Dimensionality Reduction. Also, will cover every related aspect of machine learning- Dimensionality Reduction like components &amp; Methods of Dimensionality Reduction, Principle Component analysis &amp; Importance of Dimensionality Reduction, Feature selection, Advantages &amp; Disadvantages of Dimensionality Reduction. Along with this, we will see all W&#8217;s of Dimensionality Reduction.<\/p>\n<\/div>\n<p>So, let&#8217;s start Dimensionality Reduction Tutorial.<\/p>\n<h3>What is Dimensionality Reduction?<\/h3>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">In<strong> machine learning<\/strong> we are having too many factors on which the final classification <span class=\"passivevoice\">is done<\/span>. These factors are b<span class=\"adverb\">asically<\/span>,<span class=\"passivevoice\">\u00a0known<\/span> as variables. The higher the number of features, the harder it gets to visualize the training set and then work on it. Sometimes, most of these features <span class=\"passivevoice\">are correlated<\/span>, and hence redundant. This is where dimensionality reduction algorithms come into play.<\/div>\n<\/div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Motivation<\/h3>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">When we deal with real problems and real data we often deal with high dimensional data that can go up to millions.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">In original high dimensional structure, data represents itself. Although, sometimes we need to reduce its dimensionality.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">We need to reduce the dimensionality that needs to associate with visualizations. Although, that is not always the case.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Components of Dimensionality Reduction<\/h3>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">There are two components of dimensionality reduction:<\/div>\n<\/div>\n<div class=\"\">\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">a. Feature selection<\/h4>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">In this, we need to find a subset of the original set of variables. Also, need a subset which we use to model the problem. It usually involves three ways:<\/div>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><strong>Filter<\/strong><\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><strong>Wrapper<\/strong><\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><strong>Embedded<\/strong><\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">b. Feature Extraction<\/h4>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">We use this, to reduces the data in a high dimensional space to a lower dimension space, i.e. a space with lesser no. of dimensions.<\/div>\n<div><\/div>\n<div>Techniques like PCA and LDA are even used not only for dimensionality reduction but also for those dimensions in which important variance is present. This results in the optimal training of latest machine learning models. For instance, in image processing, the ability to decrease the dimensions of the image has a great impact to the amount of computations while retaining necessary data.<\/div>\n<div><\/div>\n<div>Peculiar application of dimensionality reduction is related to solving the problem of the curse of dimensionality that affects the performance of the machine learning algorithms. Reduction of the features will in a way help eliminate over fitting and therefore help in the generalization of the model.<\/div>\n<div><\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><a href=\"https:\/\/data-flair.training\/blogs\/machine-learning-algorithm\/\"><strong>Read more about machine learning Algorithms in detail<\/strong><\/a><\/div>\n<\/div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Dimensionality Reduction Methods<\/h3>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">The various methods used for dimensionality reduction include:<\/div>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Principal Component Analysis (PCA)<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Linear Discriminant Analysis (LDA)<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Generalized Discriminant Analysis (GDA)<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Dimensionality reduction may be both linear or non-linear, depending upon the method used. The prime linear method, called Principal Component Analysis, or PCA, <span class=\"passivevoice\">is discussed<\/span> below.<\/div>\n<\/div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Principal Component Analysis<\/h3>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Karl Pearson has introduced this method. Also, it works on a condition. That says while the data in a higher dimensional space <span class=\"passivevoice\">need to map<\/span> to data in a lower dimension space. Although, the variance of the data in the lower dimensional space should be <span class=\"complexword\">maximum<\/span>.<\/div>\n<\/div>\n<div class=\"\"><\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">It involves the following steps:<\/div>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Construct the covariance matrix of the data.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Compute the eigenvectors of this <strong><a href=\"https:\/\/data-flair.training\/blogs\/r-matrices-operations-applications\/\">matrix<\/a><\/strong>.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">We use Eigenvectors corresponding to the largest eigenvalues. That is to reconstruct a large fraction of variance of the original data.<\/div>\n<\/div>\n<div class=\"\"><\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Hence, we <span class=\"passivevoice\">are left<\/span> with a lesser number of eigenvectors. And there might have been some data loss in the process. But, the most important variances should <span class=\"passivevoice\">be retained by<\/span> the remaining eigenvectors.<\/div>\n<\/div>\n<\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><strong><a href=\"https:\/\/data-flair.training\/blogs\/machine-learning-applications\/\">Read more about machine learning applications<\/a><\/strong><\/div>\n<div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Importance of Dimensionality Reduction<\/h3>\n<p>Why is Dimension Reduction is important in machine learning predictive modeling?<\/p>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">The problem of unwanted increase in dimension is closely related to other. That was to fixation of measuring\/recording data at a far granular level then it <span class=\"passivevoice\">was done<\/span> in past. This is no way suggesting that this is a recent problem. It has started gaining more importance lately due to a surge in data.<\/div>\n<\/div>\n<div class=\"\"><\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Lately, there has been a tremendous increase in the way sensors are being used in the industry. These sensors <span class=\"adverb\">continuously<\/span> record data and store it for analysis at a later point. In the way data gets captured, there can be a lot of redundancy.<\/div>\n<\/div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">What are Dimensionality Reduction Techniques?<\/h3>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><span class=\"adverb\">Basically<\/span>, dimension reduction refers to the process of converting a set of data. That data needs to having vast dimensions into data with lesser dimensions. Also, it needs to ensure that it conveys similar information <span class=\"adverb\">concisely<\/span>. Although, we use these techniques to solve machine learning problems. And problem is to <span class=\"complexword\">obtain<\/span> better features for a <strong><a href=\"https:\/\/data-flair.training\/blogs\/classification-algorithms\/\">classification<\/a> <\/strong>or regression task.<\/div>\n<\/div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Common Methods to Perform Dimensionality Reduction<\/h3>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">There are many methods to perform Dimension reduction. I have listed the most common methods below:<\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">\n<div id=\"attachment_9051\" style=\"width: 1210px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Methods-to-perform-Dimension-Reduction-01.jpg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-9051\" class=\"wp-image-9051 size-full\" src=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Methods-to-perform-Dimension-Reduction-01.jpg\" alt=\"Methods to perform Dimension Reduction\" width=\"1200\" height=\"628\" srcset=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Methods-to-perform-Dimension-Reduction-01.jpg 1200w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Methods-to-perform-Dimension-Reduction-01-150x79.jpg 150w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Methods-to-perform-Dimension-Reduction-01-300x157.jpg 300w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Methods-to-perform-Dimension-Reduction-01-768x402.jpg 768w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Methods-to-perform-Dimension-Reduction-01-1024x536.jpg 1024w\" sizes=\"auto, (max-width: 1200px) 100vw, 1200px\" \/><\/a><p id=\"caption-attachment-9051\" class=\"wp-caption-text\">Methods to perform Dimension Reduction<\/p><\/div>\n<\/div>\n<\/div>\n<div class=\"\">\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">a. Missing Values<\/h4>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">While exploring data, if we encounter missing values, what we do? Our first step should be to identify the reason. Then need to impute missing values\/ drop variables using appropriate methods. But, what if we have too many missing values? Should we impute missing values or drop the variables?<\/div>\n<\/div>\n<div class=\"\">\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">b. Low Variance<\/h4>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Let\u2019s think of a scenario where we have a constant variable (all observations have the same value, 5) in our data set. Do you think, it can improve the power of model? Of course NOT, because it has zero variance.<\/div>\n<\/div>\n<div class=\"\">\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">c. Decision Trees<\/h4>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">It is one of my favorite techniques. We can use it as an ultimate solution to tackle <span class=\"complexword\">multiple<\/span> challenges. Such as missing values, outliers and identifying significant variables. It worked well in our Data Hackathon also. Several data scientists used decision tree and it worked well for them.<\/div>\n<\/div>\n<div class=\"\">\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">d. Random Forest<\/h4>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Random Forest is <span class=\"complexword\">similar to<\/span> decision tree. <span class=\"qualifier\">Just<\/span> be careful that random forests have a tendency to bias towards variables that have more no. of distinct values i.e. favor numeric variables over binary\/categorical values.<\/div>\n<\/div>\n<div class=\"\">\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">e. High Correlation<\/h4>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Dimensions exhibiting higher correlation can lower down the performance of a model. Moreover, it is not good to have <span class=\"complexword\">multiple<\/span> variables of similar information. You can use Pearsoncorrelation matrix to identify the variables with high correlation. And select one of them using VIF (Variance Inflation Factor). Variables having a higher value ( VIF &gt; 5 ) can <span class=\"passivevoice\">be dropped<\/span>.<\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><a href=\"https:\/\/data-flair.training\/blogs\/advantages-and-disadvantages-of-machine-learning\/\"><strong>Read about Advantages &amp; Disadvantages of Machine learning<\/strong><\/a><\/div>\n<\/div>\n<div class=\"\">\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">f. Backward Feature Elimination<\/h4>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">In this method, we start with all n dimensions. Compute the sum of a square of error (SSR) after eliminating each variable (n times). Then, identifying variables whose removal has produced the smallest increase in the SSR. And thus removing it finally, leaving us with n-1 input features.<\/div>\n<\/div>\n<div class=\"\"><\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Repeat this process until no other variables can <span class=\"passivevoice\">be dropped<\/span>. Recently in Online Hackathon organized by Analytics Vidhya.<\/div>\n<\/div>\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">g. Factor Analysis<\/h4>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">These variables can <span class=\"passivevoice\">be grouped by<\/span> their correlations.. Here each group represents a single underlying construct or factor. These factors are small in number as compared to a large number of dimensions. <span class=\"complexword\">However<\/span>, these factors are difficult to observe. There are <span class=\"adverb\">basically<\/span> two methods of performing factor analysis:<\/div>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">EFA (Exploratory Factor Analysis)<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">CFA (Confirmatory Factor Analysis)<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<h4 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">h. Principal Component Analysis (PCA)<\/h4>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Particularly, in this we need to transform variables into a new set of variables. As these are a linear combination of original variables. These new set of variables <span class=\"passivevoice\">are known<\/span> as principal components. Further, we need to <span class=\"complexword\">obtain<\/span> these in particular way. As first principle component accounts for the possible variation of original data. after which each succeeding component has the highest possible variance.<\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">\n<div id=\"attachment_8907\" style=\"width: 339px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image-11.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-8907\" class=\"wp-image-8907 size-full\" src=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image-11.png\" alt=\"Dimensionality Reduction\" width=\"329\" height=\"249\" srcset=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image-11.png 329w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image-11-150x114.png 150w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image-11-300x227.png 300w\" sizes=\"auto, (max-width: 329px) 100vw, 329px\" \/><\/a><p id=\"caption-attachment-8907\" class=\"wp-caption-text\">PCA &#8211; Dimensionality Reduction<\/p><\/div>\n<\/div>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">The second principal component must be orthogonal to the first principal component. I For two-dimensional dataset, there can be only two principal components. Below is a snapshot of the data and its first and second principal components. Applying PCA to your dataset loses its meaning.<\/div>\n<\/div>\n<\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><a href=\"https:\/\/data-flair.training\/blogs\/deep-learning-vs-machine-learning\/\"><strong>Read about Machine Learning Vs Deep Learning<\/strong><\/a><\/div>\n<div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Reduce the Number of Dimensions<\/h3>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Dimensionality reduction has several advantages from a machine learning point of view.<\/li>\n<li>\u00a0Since your model has fewer degrees of freedom, the likelihood of overfitting is lower. The model will generalize more <span class=\"adverb\">easily<\/span> to new data.<\/li>\n<li>\u00a0If we are using feature selection the reduction will promote the important variables. Also, it helps in improving the interpretability of your model.<\/li>\n<li>Most of features extraction techniques <span class=\"passivevoice\">are unsupervised<\/span>. You can train your autoencoder or fit your PCA on unlabeled data. This can be helpful if you have a lot of unlabeled data and labeling is time-consuming and expensive.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Features Selection in Reduction<\/h3>\n<\/div>\n<div class=\"\">\n<p>Most, important is to reduce dimensionality. Also, is to remove some dimensions and to select the more suitable variables for the problem.<\/p>\n<p>Here are some ways to select variables:<\/p>\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Greedy algorithms which add and remove variables until some criterion <span class=\"passivevoice\">is met<\/span>.<\/li>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Shrinking and penalization methods, which will add cost for having too many variables. For instance, L1 regularization will cut some variables\u2019 coefficient to zero. Regularization limits the space where the coefficients can live in.<\/li>\n<li>As we have to select model on particular criteria. That need to take the number of dimensions into accounts. Such as the adjusted R\u00b2, AIC or BIC. Contrary to regularization, the model is not trained to optimize these criteria.<\/li>\n<li>Filtering of variables using correlation, VIF or some \u201cdistance measure\u201d between the features.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><\/div>\n<div class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><strong><a href=\"https:\/\/data-flair.training\/blogs\/artificial-neural-network\/\">Let&#8217;s discuss Artificial Neural Network in Machine Learning<\/a><\/strong><\/div>\n<\/div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Advantages of Dimensionality Reduction<\/h3>\n<ul>\n<li>Dimensionality Reduction helps in data compression, and hence reduced storage space.<\/li>\n<li>It reduces computation time.<\/li>\n<li>It also helps remove redundant features, if any.<\/li>\n<li>Dimensionality Reduction helps in data compressing and reducing the storage space required<\/li>\n<li>It fastens the time required for performing same computations.<\/li>\n<li>If there present fewer dimensions then it leads to less computing. Also, dimensions can allow usage of algorithms unfit for a large number of dimensions.<\/li>\n<li>It takes care of multicollinearity that improves the model performance. It removes redundant features. For example, there is no point in storing a value in two different units (meters and inches).<\/li>\n<li>Reducing the dimensions of data to 2D or 3D may allow us to plot and visualize it <span class=\"adverb\">precisely<\/span>. You can then observe patterns more <span class=\"adverb\">clearly<\/span>. Below you can see that, how a 3D data <span class=\"passivevoice\">is converted<\/span> into 2D. First, it has identified the 2D plane then represented the points on these two new axes z1 and z2.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<div id=\"attachment_8908\" style=\"width: 437px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image7.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-8908\" class=\"wp-image-8908 size-full\" src=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image7.png\" alt=\"Dimensionality Reduction\" width=\"427\" height=\"249\" srcset=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image7.png 427w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image7-150x87.png 150w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/image7-300x175.png 300w\" sizes=\"auto, (max-width: 427px) 100vw, 427px\" \/><\/a><p id=\"caption-attachment-8908\" class=\"wp-caption-text\">Advantages of Dimensionality Reduction<\/p><\/div>\n<ul>\n<li>It is helpful in noise removal also and as a result of that, we can improve the performance of models.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Disadvantages of Dimensionality Reduction<\/h3>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Basically, it may lead to some amount of data loss.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Although, PCA tends to find linear correlations between variables, which is sometimes undesirable.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Also, PCA fails in cases where mean and covariance are not enough to define datasets.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<ul>\n<li class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Further, we may not know how many principal components to keep- in practice, some thumb rules <span class=\"passivevoice\">are applied<\/span>.<\/li>\n<\/ul>\n<\/div>\n<div class=\"\">\n<p class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\"><a href=\"https:\/\/data-flair.training\/blogs\/svm-support-vector-machine-tutorial\/\"><strong>Let&#8217;s see SVM- Support Vector Machine\u00a0<\/strong><\/a><\/p>\n<\/div>\n<p>So, this was all about\u00a0Dimensionality Reduction Tutorial. Hope you like our explanation.<\/p>\n<div class=\"\">\n<h3 class=\"public-DraftStyleDefault-block public-DraftStyleDefault-ltr\">Conclusion<\/h3>\n<\/div>\n<div class=\"\">Dimensionality reduction means reducing the number of input variables in a dataset. When data has too many features, it becomes hard to process. Dimensionality reduction removes the extra parts while keeping the important ones.<\/div>\n<div><\/div>\n<div class=\"\">Methods like PCA (Principal Component Analysis) and t-SNE help in this process. PCA finds new variables that hold most of the information. t-SNE helps in visualizing high-dimensional data in 2D or 3D. These methods make data simpler and models faster.<\/div>\n<div><\/div>\n<div class=\"\">It\u2019s very useful in image processing, genomics, and text analysis. Reducing noise and keeping important features leads to better model accuracy and training speed.<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>In this Machine Learning Tutorial, we will study What is Dimensionality Reduction. Also, will cover every related aspect of machine learning- Dimensionality Reduction like components &amp; Methods of Dimensionality Reduction, Principle Component analysis &amp;&#46;&#46;&#46;<\/p>\n","protected":false},"author":5,"featured_media":9048,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[36],"tags":[334,3923,16481,16479],"class_list":["post-8899","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","tag-advantages-of-dimension-reduction","tag-dimension-reduction-techniques","tag-dimensionality-reduction-algorithm","tag-what-is-dimensionality-reduction"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>What is Dimensionality Reduction - Techniques, Methods, Components - DataFlair<\/title>\n<meta name=\"description\" content=\"What is Dimensionality reduction- dimension reduction technique, Methods &amp; importance of dimension reduction, Advantages &amp; Disadvantages of Dimensionality\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What is Dimensionality Reduction - Techniques, Methods, Components - DataFlair\" \/>\n<meta property=\"og:description\" content=\"What is Dimensionality reduction- dimension reduction technique, Methods &amp; importance of dimension reduction, Advantages &amp; Disadvantages of Dimensionality\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-02-21T12:09:14+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-27T14:35:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Machine-Learning-\u2013-Dimensionality-Reduction-01.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"What is Dimensionality Reduction - Techniques, Methods, Components - DataFlair","description":"What is Dimensionality reduction- dimension reduction technique, Methods & importance of dimension reduction, Advantages & Disadvantages of Dimensionality","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/","og_locale":"en_US","og_type":"article","og_title":"What is Dimensionality Reduction - Techniques, Methods, Components - DataFlair","og_description":"What is Dimensionality reduction- dimension reduction technique, Methods & importance of dimension reduction, Advantages & Disadvantages of Dimensionality","og_url":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-02-21T12:09:14+00:00","article_modified_time":"2025-07-27T14:35:47+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Machine-Learning-\u2013-Dimensionality-Reduction-01.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/7f83c342f5d1632d6f7b4b0b0f447823"},"headline":"What is Dimensionality Reduction &#8211; Techniques, Methods, Components","datePublished":"2018-02-21T12:09:14+00:00","dateModified":"2025-07-27T14:35:47+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/"},"wordCount":1902,"commentCount":0,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Machine-Learning-\u2013-Dimensionality-Reduction-01.jpg","keywords":["Advantages of dimension reduction","dimension reduction techniques","Dimensionality Reduction Algorithm","What is Dimensionality Reduction"],"articleSection":["Machine Learning Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/","url":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/","name":"What is Dimensionality Reduction - Techniques, Methods, Components - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Machine-Learning-\u2013-Dimensionality-Reduction-01.jpg","datePublished":"2018-02-21T12:09:14+00:00","dateModified":"2025-07-27T14:35:47+00:00","description":"What is Dimensionality reduction- dimension reduction technique, Methods & importance of dimension reduction, Advantages & Disadvantages of Dimensionality","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Machine-Learning-\u2013-Dimensionality-Reduction-01.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Machine-Learning-\u2013-Dimensionality-Reduction-01.jpg","width":1200,"height":628,"caption":"What is Dimensionality Reduction"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/dimensionality-reduction-tutorial\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Machine Learning Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/machine-learning\/"},{"@type":"ListItem","position":3,"name":"What is Dimensionality Reduction &#8211; Techniques, Methods, Components"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/7f83c342f5d1632d6f7b4b0b0f447823","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/4cf3a74600d131330b8c481d519afd1574093ed89f6d3396a95393ad223eb7cd?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/4cf3a74600d131330b8c481d519afd1574093ed89f6d3396a95393ad223eb7cd?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4cf3a74600d131330b8c481d519afd1574093ed89f6d3396a95393ad223eb7cd?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"DataFlair Team creates expert-level guides on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our goal is to empower learners with easy-to-understand content. Explore our resources for career growth and practical learning.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam1\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/8899","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=8899"}],"version-history":[{"count":8,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/8899\/revisions"}],"predecessor-version":[{"id":146247,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/8899\/revisions\/146247"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/9048"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=8899"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=8899"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=8899"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}