{"id":5661,"date":"2018-01-17T08:56:55","date_gmt":"2018-01-17T08:56:55","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=5661"},"modified":"2018-09-17T18:02:30","modified_gmt":"2018-09-17T12:32:30","slug":"spark-machine-learning-with-r","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/","title":{"rendered":"Spark Machine Learning with R: An Introductory Guide"},"content":{"rendered":"

\u00a01. Objective<\/h2>\n

Today, in this Spark tutorial, we will learn several SparkR<\/strong><\/a> Machine Learning algorithms supported by Spark<\/strong><\/a>. Such as Classification, Regression, Tree, Clustering, Collaborative Filtering, Frequent Pattern Mining, Statistics, and Model persistence. we will learn all these in detail. Moreover, we will learn a\u00a0few examples to understand Spark Machine Learning with R in a better way.<\/span><\/p>\n

So, let’s start Spark machine Learning with R.<\/p>\n

\"Spark<\/a>

Spark Machine Learning with R: An Introductory Guide<\/p><\/div>\n

2. Spark Machine Learning with R<\/h2>\n

The following Spark machine learning<\/a> algorithms using R supports currently are,<\/span>
\n<\/span>
\n<\/span>a. Machine Learning\u00a0<\/span>Classification
\nspark.logit: Logistic Regression<\/b>
\nspark.mlp: Multilayer Perceptron (MLP)<\/b>
\n<\/b>spark.naiveBayes: Naive Bayes<\/b>
\n<\/b>spark.svmLinear: Linear Support Vector Machine<\/b><\/p>\n

b. Machine Learning\u00a0<\/span>Regression<\/h3>\n

spark.survreg: Accelerated Failure Time (AFT) Survival Model<\/b>
\n<\/b>spark.glm or glm: Generalized Linear Model (GLM)<\/b>
\n<\/b>spark.isoreg: Isotonic Regression<\/b><\/p>\n

Let’s\u00a0have a look at Apache Spark Machine Learning\u00a0Algorithm<\/a><\/strong><\/p>\n

c.\u00a0Machine Learning\u00a0<\/span>Tree<\/h3>\n

spark.gbt: Gradient Boosted Trees for Regression and Classification<\/b>
\n<\/b>spark.randomForest: Random Forest for Regression and Classification<\/b><\/b><\/p>\n

d. Machine Learning\u00a0<\/span>Clustering<\/h3>\n

spark.bisectingKmeans: Bisecting k-means<\/b>
\n<\/b>spark.gaussianMixture: Gaussian Mixture Model (GMM)<\/b>
\n<\/b>spark.kmeans: K-Means<\/b>
\n<\/b>spark.lda: Latent Dirichlet Allocation (LDA)<\/b><\/p>\n

e. Machine Learning\u00a0<\/span>Collaborative Filtering<\/h3>\n

spark.als: Alternating Least Squares (ALS)<\/b>
\n<\/b>Frequent Pattern Mining<\/b>
\n<\/b>spark.fpGrowth : FP-growth<\/b><\/p>\n

f. Statistical\u00a0Machine Learning<\/span><\/h3>\n

spark.kstest: Kolmogorov-Smirnov Test<\/b>
\nBasically, SparkR uses MLlib to train the model. Moreover, it supports a subset of the available R<\/a> formula operators. For example, model fitting, including \u2018~\u2019, \u2018.\u2019, \u2018:\u2019, \u2018+\u2019, and \u2018-\u2018.<\/span><\/p>\n

g. Model persistence in\u00a0Machine Learning<\/span><\/h3>\n

Here, below example shows how to save\/load an MLlib model by SparkR.<\/span>
\nFor example,<\/span>
\ntraining <- read.df(“data\/mllib\/sample_multiclass_classification_data.txt”, source = “libsvm”)<\/b>
\n<\/b># Fit a generalized linear model of family “gaussian” with spark.glm<\/b>
\n<\/b>df_list <- randomSplit(training, c(7,3), 2)<\/b>
\n<\/b>gaussianDF <- df_list[[1]]<\/b>
\n<\/b>gaussianTestDF <- df_list[[2]]<\/b>
\n<\/b>gaussianGLM <- spark.glm(gaussianDF, label ~ features, family = “gaussian”)<\/b><\/p>\n

Let’s discuss Data Types in Spark Machine Learning
\n<\/a><\/strong><\/b>
\n# Save and then load a fitted MLlib model<\/b>
\n<\/b>modelPath <- tempfile(pattern = “ml”, fileext = “.tmp”)<\/b>
\n<\/b>write.ml(gaussianGLM, modelPath)<\/b>
\n<\/b>gaussianGLM2 <- read.ml(modelPath)<\/b>
\n<\/b>
\n<\/b># Check model summary<\/b>
\n<\/b>summary(gaussianGLM2)<\/b>
\n<\/b>
\n<\/b># Check model prediction<\/b>
\n<\/b>gaussianPredictions <- predict(gaussianGLM2, gaussianTestDF)<\/b>
\n<\/b>head(gaussianPredictions)<\/b>
\n<\/b>
\n<\/b>unlink(modelPath)<\/b><\/p>\n

3. Conclusion<\/h2>\n

As a result, we have seen all the\u00a0Spark machine learning with R. Also, we have seen various examples to learn machine learning algorithm using spark R well. However, if you feel for any query, feel free to ask in the comment section.<\/p>\n

See also –\u00a0<\/strong><\/p>\n

RDD Lineage in Spark<\/a><\/strong>
\nFor Reference<\/a>.<\/strong>

\u00a01. Objective Today, in this Spark tutorial, we will learn several SparkR Machine Learning algorithms supported by Spark. Such as Classification, Regression, Tree, Clustering, Collaborative Filtering, Frequent Pattern Mining, Statistics, and Model persistence. we...<\/p>\n","protected":false},"author":6,"featured_media":6295,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10],"tags":[8445,8461,8466,13083,13170],"class_list":["post-5661","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-spark","tag-machine-learning-classification","tag-machine-learning-regression","tag-machine-learning-tree","tag-spark-machine-learning-with-r","tag-sparkr-machine-learning"],"yoast_head":"\nSpark Machine Learning with R: An Introductory Guide - DataFlair<\/title>\n<meta name=\"description\" content=\"SparkR Machine Learning algorithms-Spark Machine Learning with R examples,Machine Learning Classification,Machine Learning Regression,Machine Learning Tree\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Spark Machine Learning with R: An Introductory Guide - DataFlair\" \/>\n<meta property=\"og:description\" content=\"SparkR Machine Learning algorithms-Spark Machine Learning with R examples,Machine Learning Classification,Machine Learning Regression,Machine Learning Tree\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-01-17T08:56:55+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-09-17T12:32:30+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/01\/Machine-Learning-Algorithms-in-SparkR-01-1-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Spark Machine Learning with R: An Introductory Guide - DataFlair","description":"SparkR Machine Learning algorithms-Spark Machine Learning with R examples,Machine Learning Classification,Machine Learning Regression,Machine Learning Tree","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/","og_locale":"en_US","og_type":"article","og_title":"Spark Machine Learning with R: An Introductory Guide - DataFlair","og_description":"SparkR Machine Learning algorithms-Spark Machine Learning with R examples,Machine Learning Classification,Machine Learning Regression,Machine Learning Tree","og_url":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-01-17T08:56:55+00:00","article_modified_time":"2018-09-17T12:32:30+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/01\/Machine-Learning-Algorithms-in-SparkR-01-1-1.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89"},"headline":"Spark Machine Learning with R: An Introductory Guide","datePublished":"2018-01-17T08:56:55+00:00","dateModified":"2018-09-17T12:32:30+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/"},"wordCount":437,"commentCount":0,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/01\/Machine-Learning-Algorithms-in-SparkR-01-1-1.jpg","keywords":["Machine Learning Classification","Machine Learning Regression","Machine Learning Tree","Spark Machine Learning with R","SparkR Machine Learning"],"articleSection":["Apache Spark Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/","url":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/","name":"Spark Machine Learning with R: An Introductory Guide - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/01\/Machine-Learning-Algorithms-in-SparkR-01-1-1.jpg","datePublished":"2018-01-17T08:56:55+00:00","dateModified":"2018-09-17T12:32:30+00:00","description":"SparkR Machine Learning algorithms-Spark Machine Learning with R examples,Machine Learning Classification,Machine Learning Regression,Machine Learning Tree","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/01\/Machine-Learning-Algorithms-in-SparkR-01-1-1.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/01\/Machine-Learning-Algorithms-in-SparkR-01-1-1.jpg","width":1200,"height":628,"caption":"Spark Machine Learning with R"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/spark-machine-learning-with-r\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Apache Spark Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/spark\/"},{"@type":"ListItem","position":3,"name":"Spark Machine Learning with R: An Introductory Guide"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"The DataFlair Team provides industry-driven content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our expert educators focus on delivering value-packed, easy-to-follow resources for tech enthusiasts and professionals.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam2\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/5661","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=5661"}],"version-history":[{"count":4,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/5661\/revisions"}],"predecessor-version":[{"id":34464,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/5661\/revisions\/34464"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/6295"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=5661"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=5661"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=5661"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}