{"id":1093,"date":"2016-08-23T14:52:14","date_gmt":"2016-08-23T14:52:14","guid":{"rendered":"http:\/\/data-flair.training\/blogs\/?p=1093"},"modified":"2019-12-11T12:42:14","modified_gmt":"2019-12-11T07:12:14","slug":"install-apache-spark-multi-node-cluster","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/","title":{"rendered":"Install Apache Spark on Multi-Node Cluster"},"content":{"rendered":"<h2>1. Objective<\/h2>\n<p>This Spark tutorial explains how to install <a href=\"http:\/\/data-flair.training\/blogs\/apache-spark-tutorial\/\"><strong>Apache Spark<\/strong><\/a> on a multi-node cluster. This guide provides step by step instructions to deploy and configure Apache Spark on the real multi-node cluster.\u00a0Once the setup and installation\u00a0are done you can play with Spark and process data.<\/p>\n<p><a href=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-73643 size-full\" src=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2.jpg\" alt=\"apache spark installation on multi node cluster\" width=\"802\" height=\"420\" srcset=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2.jpg 802w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2-150x79.jpg 150w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2-300x157.jpg 300w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2-768x402.jpg 768w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2-520x272.jpg 520w\" sizes=\"auto, (max-width: 802px) 100vw, 802px\" \/><\/a><\/p>\n<h2>2. Steps to install Apache Spark on multi-node cluster<\/h2>\n<p>Follow the steps given below to easily install Apache Spark on a multi-node cluster.<\/p>\n<h3>i. Recommended Platform<\/h3>\n<ul>\n<li><strong>OS &#8211;<\/strong>\u00a0Linux is supported as a development and deployment\u00a0platform. You can use Ubuntu 14.04 \/ 16.04 or later (you can also use other Linux flavors like CentOS, Redhat, etc.).\u00a0Windows is supported as a dev platform. (If you are new to Linux follow this <strong><a href=\"http:\/\/data-flair.training\/blogs\/frequent-linux-commands-beginners-tutorial\/\">Linux commands manual<\/a><\/strong>).<\/li>\n<li><strong>Spark &#8211;<\/strong>\u00a0Apache Spark 2.x<\/li>\n<\/ul>\n<p>For Apache Spark Installation On Multi-Node Cluster, we will be needing multiple nodes, either you can use <a href=\"http:\/\/aws.amazon.com\/\">Amazon AWS<\/a> or follow this\u00a0<a href=\"http:\/\/data-flair.training\/blogs\/install-ubuntu-vmware-player\/\">guide to setup virtual platform using VMWare player<\/a>.<\/p>\n<h3>ii. Install Spark on Master<\/h3>\n<h4>a. Prerequisites<\/h4>\n<p><strong>Add Entries in hosts file<\/strong><\/p>\n<p>Edit hosts file<br \/>\n[php]sudo nano \/etc\/hosts[\/php]<br \/>\nNow add entries of master and slaves<br \/>\n[php]MASTER-IP master<br \/>\nSLAVE01-IP slave01<br \/>\nSLAVE02-IP slave02[\/php]<br \/>\n<em>(NOTE: In place of MASTER-IP, SLAVE01-IP, SLAVE02-IP put the value of the corresponding IP)<\/em><\/p>\n<p><strong>Install Java 7 (Recommended Oracle Java)<\/strong><\/p>\n<p>[php]sudo apt-get install python-software-properties<br \/>\nsudo add-apt-repository ppa:webupd8team\/java<br \/>\nsudo apt-get update<br \/>\nsudo apt-get install oracle-java7-installer[\/php]<\/p>\n<p><strong>Install Scala<\/strong><\/p>\n<p>[php]sudo apt-get install scala[\/php]<\/p>\n<p><strong>Configure SSH<\/strong><\/p>\n<p>Install Open SSH Server-Client<\/p>\n<p>[php]sudo apt-get install openssh-server openssh-client[\/php]<\/p>\n<p><strong>Generate Key Pairs<\/strong><\/p>\n<p>[php]ssh-keygen -t rsa -P &#8220;&#8221;[\/php]<\/p>\n<p><strong>Configure passwordless SSH<\/strong><\/p>\n<p>Copy the content of .ssh\/id_rsa.pub (of master) to .ssh\/authorized_keys (of all the slaves as well as master)<\/p>\n<p><strong>Check by SSH to all the Slaves<\/strong><\/p>\n<p>[php]ssh slave01<br \/>\nssh slave02[\/php]<\/p>\n<h4>b. Install Spark<\/h4>\n<p><strong>Download Spark<\/strong><\/p>\n<p>You can download the latest version of spark from <strong><a href=\"http:\/\/spark.apache.org\/downloads.html\">http:\/\/spark.apache.org\/downloads.html<\/a>.<\/strong><\/p>\n<p><strong>Untar Tarball<\/strong><\/p>\n<p>tar xzf spark-2.0.0-bin-hadoop2.6.tgz<br \/>\n<em>(Note: All the scripts, jars, and configuration files are available in newly created directory \u201cspark-2.0.0-bin-hadoop2.6\u201d)<\/em><\/p>\n<p><strong>Setup Configuration<\/strong><\/p>\n<p>Edit .bashrc<\/p>\n<p>Now edit .bashrc file located in user\u2019s home directory and add following environment variables:<br \/>\n[php]export JAVA_HOME=&lt;path-of-Java-installation&gt; (eg: \/usr\/lib\/jvm\/java-7-oracle\/)<br \/>\nexport SPARK_HOME=&lt;path-to-the-root-of-your-spark-installation&gt; (eg: \/home\/dataflair\/spark-2.0.0-bin-hadoop2.6\/)<br \/>\nexport PATH=$PATH:$SPARK_HOME\/bin[\/php]<br \/>\n<em>(Note: After above step restart the Terminal\/Putty so that all the environment variables will come into effect)<\/em><\/p>\n<p>Edit spark-env.sh<\/p>\n<p>Now edit configuration file spark-env.sh (in $SPARK_HOME\/conf\/) and set following parameters:<br \/>\nNote: Create a copy of template of spark-env.sh and rename it:<br \/>\n[php]cp spark-env.sh.template spark-env.sh[\/php]<br \/>\n[php]export JAVA_HOME=&lt;path-of-Java-installation&gt; (eg: \/usr\/lib\/jvm\/java-7-oracle\/)<br \/>\nexport SPARK_WORKER_CORES=8[\/php]<\/p>\n<p>Add Salves<\/p>\n<p>Create configuration file slaves (in $SPARK_HOME\/conf\/) and add following entries:<br \/>\n[php]slave01<br \/>\nslave02[\/php]<br \/>\n<strong>\u201cApache Spark has been\u00a0installed successfully\u00a0on Master, now deploy Spark on all the Slaves\u201d<\/strong><\/p>\n<h3>iii. Install Spark On Slaves<\/h3>\n<h4>a. Setup Prerequisites on all the slaves<\/h4>\n<p>Run following steps on all the slaves (or worker nodes):<\/p>\n<ul>\n<li>&#8220;1.1. Add Entries in hosts file&#8221;<\/li>\n<li>&#8220;1.2. Install Java 7&#8221;<\/li>\n<li>&#8220;1.3. Install Scala&#8221;<\/li>\n<\/ul>\n<h4>b. Copy setups from master to all the slaves<\/h4>\n<p><strong>Create tarball of configured setup<\/strong><\/p>\n<p>[php]tar czf spark.tar.gz spark-2.0.0-bin-hadoop2.6[\/php]<br \/>\n<em>NOTE: Run this command on Master<\/em><\/p>\n<p><strong>Copy the configured tarball on all the slaves<\/strong><\/p>\n<p>[php]scp spark.tar.gz slave01:~[\/php]<br \/>\n<em>NOTE: Run this command on Master<\/em><br \/>\n[php]scp spark.tar.gz slave02:~[\/php]<br \/>\n<em>NOTE: Run this command on Master<\/em><\/p>\n<h4>c. Un-tar configured spark setup on all the slaves<\/h4>\n<p>[php]tar xzf spark.tar.gz[\/php]<br \/>\nNOTE: Run this command on all the slaves<br \/>\n<strong>\u201cCongratulations Apache Spark has been installed\u00a0on all the Slaves. Now Start the daemons on the Cluster\u201d<\/strong><\/p>\n<h3>iv. Start Spark\u00a0Cluster<\/h3>\n<h4>a. Start Spark Services<\/h4>\n<p>[php]sbin\/start-all.sh[\/php]<br \/>\n<em>Note: Run this command on Master<\/em><\/p>\n<h4>b. Check whether services have been started<\/h4>\n<p><strong>Check daemons on Master<\/strong><\/p>\n<p>[php]jps<br \/>\nMaster[\/php]<\/p>\n<p><strong>Check daemons on Slaves<\/strong><\/p>\n<p>[php]jps<br \/>\nWorker[\/php]<\/p>\n<h3>v. Spark Web UI<\/h3>\n<h4>a. Spark Master UI<\/h4>\n<p>Browse the Spark UI to know about worker nodes, running application, cluster resources.<br \/>\nhttp:\/\/MASTER-IP:8080\/<\/p>\n<h4>b. Spark application UI<\/h4>\n<p>http:\/\/MASTER-IP:4040\/<\/p>\n<h3>vi. Stop the Cluster<\/h3>\n<h4>a. Stop Spark Services<\/h4>\n<p>Once all the applications have finished, you can stop the spark services (master and slaves daemons) running on the cluster<br \/>\n[php]sbin\/stop-all.sh[\/php]<br \/>\n<em>Note: Run this command on Master<\/em><br \/>\nAfter Apache Spark installation, I recommend learning Spark<strong><a href=\"http:\/\/data-flair.training\/blogs\/apache-spark-rdd-tutorial\/\"> RDD<\/a><\/strong>, <strong><a href=\"http:\/\/data-flair.training\/blogs\/apache-spark-sql-dataframe-tutorial\/\">DataFrame<\/a><\/strong>, and <strong><a href=\"http:\/\/data-flair.training\/blogs\/apache-spark-dataset-tutorial\/\">Dataset<\/a>. <\/strong>You can proceed further with\u00a0<strong><a href=\"http:\/\/data-flair.training\/blogs\/apache-spark-shell-commands-beginners-tutorial\/\">Spark shell commands<\/a><\/strong> to play with Spark.<\/p>\n<p>So, this was all in how to Install Apache Spark. Hope you like our explanation.<\/p>\n<h2>3. Conclusion &#8211; Install Apache Spark<\/h2>\n<p>After installing the Apache Spark on the multi-node cluster you are now ready to work with Spark platform. Now you can play with the data, <a href=\"http:\/\/data-flair.training\/blogs\/how-to-create-rdds-in-apache-spark\/\"><strong>create an RDD<\/strong><\/a>, perform operations on those RDDs over multiple nodes and much more.<br \/>\nIf you have any query to install Apache Spark, so, feel free to share with us. We will be happy to solve them.<br \/>\n<strong>See Also-<\/strong><\/p>\n<ul>\n<li><strong><a href=\"http:\/\/data-flair.training\/blogs\/sparkcontext-in-apache-spark-tutorial\/\">SparkContext in Apache Spark<\/a><\/strong><\/li>\n<li><strong><a href=\"http:\/\/data-flair.training\/blogs\/apache-spark-rdd-transformations-actions\/\">RDD Transformations and Actions APIs<\/a><\/strong><\/li>\n<\/ul>\n<p><a href=\"http:\/\/spark.apache.org\/\"><strong>Reference for Spark<\/strong><\/a><span hidden class=\"__iawmlf-post-loop-links\" data-iawmlf-links=\"[{&quot;id&quot;:2387,&quot;href&quot;:&quot;http:\\\/\\\/aws.amazon.com&quot;,&quot;archived_href&quot;:&quot;http:\\\/\\\/web-wp.archive.org\\\/web\\\/20251209215753\\\/https:\\\/\\\/aws.amazon.com\\\/&quot;,&quot;redirect_href&quot;:&quot;&quot;,&quot;checks&quot;:[{&quot;date&quot;:&quot;2025-12-11 06:24:32&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2025-12-14 12:45:48&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2025-12-17 14:56:38&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2025-12-20 17:30:49&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2025-12-24 08:52:46&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2025-12-28 17:12:52&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2025-12-31 20:28:51&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-01-04 18:20:14&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-01-08 13:48:25&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-01-12 10:09:18&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-01-15 19:08:30&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-01-19 04:11:02&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-01-25 15:24:08&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-01-29 03:15:36&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-02-03 02:13:21&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-02-06 17:18:37&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-02-10 12:22:25&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-02-15 16:28:22&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-02-19 01:33:30&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-02-22 13:56:00&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-02-26 05:32:42&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-03-01 18:03:10&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-03-07 11:03:29&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-03-10 16:43:03&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-03-14 14:58:07&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-03-17 19:00:09&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-03-21 02:09:10&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-03-25 04:38:15&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-03-29 15:47:11&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-04-06 19:18:04&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-04-10 08:11:20&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-04-14 06:19:49&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-04-18 14:37:45&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-04-22 16:20:27&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-04-27 23:13:26&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-05-04 14:31:29&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-05-07 15:23:22&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-05-11 13:49:27&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-05-14 22:34:24&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-05-20 14:31:23&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-05-26 04:38:03&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-06-02 11:37:02&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-06-09 09:05:25&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-06-15 06:14:35&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-06-27 03:16:50&quot;,&quot;http_code&quot;:404},{&quot;date&quot;:&quot;2026-07-01 04:48:03&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-07-06 20:31:24&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-07-11 06:58:27&quot;,&quot;http_code&quot;:200},{&quot;date&quot;:&quot;2026-07-15 01:10:53&quot;,&quot;http_code&quot;:200}],&quot;broken&quot;:false,&quot;last_checked&quot;:{&quot;date&quot;:&quot;2026-07-15 01:10:53&quot;,&quot;http_code&quot;:200},&quot;process&quot;:&quot;done&quot;},{&quot;id&quot;:2388,&quot;href&quot;:&quot;http:\\\/\\\/spark.apache.org\\\/downloads.html&quot;,&quot;archived_href&quot;:&quot;http:\\\/\\\/web-wp.archive.org\\\/web\\\/20251002061320\\\/https:\\\/\\\/spark.apache.org\\\/downloads.html&quot;,&quot;redirect_href&quot;:&quot;&quot;,&quot;checks&quot;:[{&quot;date&quot;:&quot;2025-12-11 06:24:36&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-14 12:45:47&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-17 14:56:44&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-20 17:30:50&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-24 01:45:37&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-28 14:19:37&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-31 16:26:03&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-03 20:08:44&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-08 10:52:02&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-12 10:09:31&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-16 05:10:09&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-20 15:18:12&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-25 15:24:13&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-28 19:02:19&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-01 16:32:36&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-04 20:39:53&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-09 05:16:29&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-15 16:28:54&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-19 01:33:29&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-22 13:56:19&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-26 05:32:42&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-01 16:50:25&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-07 11:04:11&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-10 16:43:08&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-14 14:57:14&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-17 19:00:12&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-21 04:25:41&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-25 04:38:44&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-29 22:34:50&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-03 01:24:04&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-06 08:37:51&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-09 21:32:41&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-14 06:19:50&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-17 23:59:38&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-21 13:40:26&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-25 20:16:56&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-29 12:41:17&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-04 19:02:08&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-11 11:33:11&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-14 12:36:10&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-20 13:41:50&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-24 23:44:28&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-28 06:34:34&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-01 10:54:32&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-04 16:54:11&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-09 09:05:53&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-15 06:15:05&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-20 04:24:25&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-27 10:21:55&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-30 17:41:35&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-03 18:45:31&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-07 22:48:03&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-11 06:58:27&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-15 01:10:51&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-19 02:31:09&quot;,&quot;http_code&quot;:206}],&quot;broken&quot;:false,&quot;last_checked&quot;:{&quot;date&quot;:&quot;2026-07-19 02:31:09&quot;,&quot;http_code&quot;:206},&quot;process&quot;:&quot;done&quot;},{&quot;id&quot;:2354,&quot;href&quot;:&quot;http:\\\/\\\/spark.apache.org&quot;,&quot;archived_href&quot;:&quot;http:\\\/\\\/web-wp.archive.org\\\/web\\\/20251009215151\\\/https:\\\/\\\/spark.apache.org\\\/&quot;,&quot;redirect_href&quot;:&quot;&quot;,&quot;checks&quot;:[{&quot;date&quot;:&quot;2025-12-11 04:17:36&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-14 07:11:19&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-17 07:55:29&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-20 14:34:27&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-23 15:49:42&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-26 15:59:57&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2025-12-30 07:08:03&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-02 07:19:25&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-05 08:37:45&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-08 09:28:47&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-11 11:37:40&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-14 12:46:43&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-17 20:26:14&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-20 20:31:00&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-24 06:20:15&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-27 06:26:56&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-01-30 07:17:36&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-02 07:26:54&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-05 10:18:07&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-08 12:50:55&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-11 14:05:53&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-14 15:00:31&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-18 00:17:52&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-21 06:52:12&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-24 08:35:32&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-02-27 08:54:29&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-02 09:01:11&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-05 09:57:46&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-08 12:27:51&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-11 12:42:39&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-14 23:54:40&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-18 03:00:10&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-21 06:08:58&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-24 07:13:58&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-27 09:23:48&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-03-30 11:37:48&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-02 13:11:14&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-05 14:53:20&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-08 19:36:36&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-11 23:42:38&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-15 01:00:01&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-18 06:16:05&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-21 07:55:15&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-24 09:26:05&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-27 11:00:27&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-04-30 12:57:25&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-03 13:36:16&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-06 19:54:59&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-10 07:47:43&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-13 09:22:32&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-16 16:11:08&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-19 16:22:48&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-22 17:30:06&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-05-25 20:07:42&quot;,&quot;http_code&quot;:503},{&quot;date&quot;:&quot;2026-05-29 03:42:28&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-01 10:44:16&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-04 10:50:12&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-07 10:53:31&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-10 11:32:26&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-13 13:26:22&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-17 00:18:56&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-20 03:02:15&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-23 10:46:05&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-26 12:51:38&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-06-29 12:51:51&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-02 13:46:38&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-05 16:50:32&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-09 05:30:27&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-12 14:20:23&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-15 20:00:46&quot;,&quot;http_code&quot;:206},{&quot;date&quot;:&quot;2026-07-19 06:45:58&quot;,&quot;http_code&quot;:206}],&quot;broken&quot;:false,&quot;last_checked&quot;:{&quot;date&quot;:&quot;2026-07-19 06:45:58&quot;,&quot;http_code&quot;:206},&quot;process&quot;:&quot;done&quot;}]\"><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>1. Objective This Spark tutorial explains how to install Apache Spark on a multi-node cluster. This guide provides step by step instructions to deploy and configure Apache Spark on the real multi-node cluster.\u00a0Once the&#46;&#46;&#46;<\/p>\n","protected":false},"author":6,"featured_media":73643,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10],"tags":[896,16622,6796,6797,8176,13039,13088,13089,13113,13128],"class_list":["post-1093","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-spark","tag-apache-spark","tag-install-apache-spark","tag-install-spark","tag-install-spark-on-cluster","tag-learn-spark","tag-spark-cluster","tag-spark-multi-node","tag-spark-multi-node-cluster","tag-spark-setup","tag-spark-standalone-mode"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Install Apache Spark on Multi-Node Cluster - DataFlair<\/title>\n<meta name=\"description\" content=\"Apache Spark Installation on Multi-Node Cluster-learn how to install Apache Spark,install &amp; configure on ubuntu.Run Spark application after installing Spark\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Install Apache Spark on Multi-Node Cluster - DataFlair\" \/>\n<meta property=\"og:description\" content=\"Apache Spark Installation on Multi-Node Cluster-learn how to install Apache Spark,install &amp; configure on ubuntu.Run Spark application after installing Spark\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2016-08-23T14:52:14+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-12-11T07:12:14+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"802\" \/>\n\t<meta property=\"og:image:height\" content=\"420\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Install Apache Spark on Multi-Node Cluster - DataFlair","description":"Apache Spark Installation on Multi-Node Cluster-learn how to install Apache Spark,install & configure on ubuntu.Run Spark application after installing Spark","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/","og_locale":"en_US","og_type":"article","og_title":"Install Apache Spark on Multi-Node Cluster - DataFlair","og_description":"Apache Spark Installation on Multi-Node Cluster-learn how to install Apache Spark,install & configure on ubuntu.Run Spark application after installing Spark","og_url":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2016-08-23T14:52:14+00:00","article_modified_time":"2019-12-11T07:12:14+00:00","og_image":[{"width":802,"height":420,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89"},"headline":"Install Apache Spark on Multi-Node Cluster","datePublished":"2016-08-23T14:52:14+00:00","dateModified":"2019-12-11T07:12:14+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/"},"wordCount":870,"commentCount":33,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2.jpg","keywords":["apache spark","install Apache Spark","install spark","install spark on cluster","learn spark","spark cluster","spark multi-node","spark multi-node cluster","spark setup","spark standalone mode"],"articleSection":["Apache Spark Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/","url":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/","name":"Install Apache Spark on Multi-Node Cluster - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2.jpg","datePublished":"2016-08-23T14:52:14+00:00","dateModified":"2019-12-11T07:12:14+00:00","description":"Apache Spark Installation on Multi-Node Cluster-learn how to install Apache Spark,install & configure on ubuntu.Run Spark application after installing Spark","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/08\/apache-spark-installation-on-multi-node-cluster-2.jpg","width":802,"height":420,"caption":"apache spark installation on multi node cluster"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/install-apache-spark-multi-node-cluster\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Apache Spark Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/spark\/"},{"@type":"ListItem","position":3,"name":"Install Apache Spark on Multi-Node Cluster"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"The DataFlair Team provides industry-driven content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our expert educators focus on delivering value-packed, easy-to-follow resources for tech enthusiasts and professionals.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam2\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/1093","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=1093"}],"version-history":[{"count":6,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/1093\/revisions"}],"predecessor-version":[{"id":73645,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/1093\/revisions\/73645"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/73643"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=1093"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=1093"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=1093"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}