Spark Machine Learning with R: An Introductory Guide
Stay updated with latest technology trends
Join DataFlair on Telegram!!
Today, in this Spark tutorial, we will learn several SparkR Machine Learning algorithms supported by Spark. Such as Classification, Regression, Tree, Clustering, Collaborative Filtering, Frequent Pattern Mining, Statistics, and Model persistence. we will learn all these in detail. Moreover, we will learn a few examples to understand Spark Machine Learning with R in a better way.
So, let’s start Spark machine Learning with R.
2. Spark Machine Learning with R
The following Spark machine learning algorithms using R supports currently are,
a. Machine Learning Classification
spark.logit: Logistic Regression
spark.mlp: Multilayer Perceptron (MLP)
spark.naiveBayes: Naive Bayes
spark.svmLinear: Linear Support Vector Machine
b. Machine Learning Regression
spark.survreg: Accelerated Failure Time (AFT) Survival Model
spark.glm or glm: Generalized Linear Model (GLM)
spark.isoreg: Isotonic Regression
c. Machine Learning Tree
spark.gbt: Gradient Boosted Trees for Regression and Classification
spark.randomForest: Random Forest for Regression and Classification
d. Machine Learning Clustering
spark.bisectingKmeans: Bisecting k-means
spark.gaussianMixture: Gaussian Mixture Model (GMM)
spark.lda: Latent Dirichlet Allocation (LDA)
e. Machine Learning Collaborative Filtering
spark.als: Alternating Least Squares (ALS)
Frequent Pattern Mining
spark.fpGrowth : FP-growth
f. Statistical Machine Learning
spark.kstest: Kolmogorov-Smirnov Test
Basically, SparkR uses MLlib to train the model. Moreover, it supports a subset of the available R formula operators. For example, model fitting, including ‘~’, ‘.’, ‘:’, ‘+’, and ‘-‘.
g. Model persistence in Machine Learning
Here, below example shows how to save/load an MLlib model by SparkR.
training <- read.df(“data/mllib/sample_multiclass_classification_data.txt”, source = “libsvm”)
# Fit a generalized linear model of family “gaussian” with spark.glm
df_list <- randomSplit(training, c(7,3), 2)
gaussianDF <- df_list[]
gaussianTestDF <- df_list[]
gaussianGLM <- spark.glm(gaussianDF, label ~ features, family = “gaussian”)
Let’s discuss Data Types in Spark Machine Learning
# Save and then load a fitted MLlib model
modelPath <- tempfile(pattern = “ml”, fileext = “.tmp”)
gaussianGLM2 <- read.ml(modelPath)
# Check model summary
# Check model prediction
gaussianPredictions <- predict(gaussianGLM2, gaussianTestDF)
As a result, we have seen all the Spark machine learning with R. Also, we have seen various examples to learn machine learning algorithm using spark R well. However, if you feel for any query, feel free to ask in the comment section.
See also –