{"id":5883,"date":"2018-01-18T12:06:55","date_gmt":"2018-01-18T12:06:55","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=5883"},"modified":"2019-09-04T11:14:26","modified_gmt":"2019-09-04T05:44:26","slug":"spark-tutorial","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/","title":{"rendered":"Spark Tutorial – Learn Spark Programming"},"content":{"rendered":"

1. Objective – Spark Tutorial<\/h2>\n

In this Spark Tutorial, we will see an overview\u00a0of Spark in Big Data. We will start with an introduction to Apache Spark Programming. Then we will move to know the Spark History. Moreover, we will learn why Spark is needed. Afterward, will cover all fundamental of Spark components. Furthermore, we will learn about Spark\u2019s core abstraction and Spark RDD. For more detailed insights, we will also cover spark features, Spark limitations, and Spark Use cases.\u00a0<\/span><\/p>\n

\"Apache<\/a>

Apache Spark Tutorial – What is Apache Spark?<\/p><\/div>\n

2. Introduction to\u00a0Spark Programming<\/span><\/h2>\n

What is Spark?\u00a0<\/span>Spark Programming is nothing but a general-purpose & lightning fast cluster computing platform<\/em>. In other words, it is an open source, wide range data processing engine<\/em>. That reveals development API\u2019s, which also qualifies data workers to accomplish streaming, machine learning or SQL workloads which demand repeated access to data sets. However, Spark can perform batch processing and stream processing<\/a><\/strong>. Batch processing refers, to the processing of the previously collected job in a single batch. Whereas stream processing means to deal with Spark streaming data. <\/span><\/p>\n

Moreover, it is designed in such a way that it integrates with all the Big data<\/strong><\/a> tools. Like spark can access any Hadoop<\/strong><\/a> data source, also can run on Hadoop clusters. Furthermore, Apache Spark extends Hadoop MapReduce to the next level. That also includes iterative queries and stream processing.
\nOne more common belief about Spark is that it is an extension of Hadoop. Although that is not true. However, Spark is independent of Hadoop since it has its own cluster management<\/strong><\/a> system. Basically, it uses Hadoop for storage purpose only.<\/p>\n

Although, there is one spark\u2019s key feature that it has in-memory cluster computation capability. Also increases the processing speed of an application.<\/p>\n

Basically, Apache Spark offers high-level APIs to users, such as Java<\/strong><\/a>, Scala, Python<\/a>,\u00a0<\/strong>and R. Although, Spark is written in Scala still offers rich APIs in Scala, Java, Python, as well as R. We can say, it is a tool for running spark applications.<\/p>\n

Most importantly, by comparing Spark with Hadoop<\/a>,<\/strong>\u00a0it is 100 times faster than Hadoop In-Memory mode and 10 times faster than Hadoop\u00a0 On-Disk mode.<\/p>\n

3. Spark Tutorial –\u00a0 History<\/h2>\n

At first, in 2009 Apache Spark was introduced in the UC Berkeley R&D Lab, which is now known as AMPLab. Afterward, in 2010 it became open source under BSD license. Further, the spark was donated to Apache Software Foundation, in 2013. Then in 2014, it became top-level Apache project.<\/span><\/p>\n

4. Why Spark?<\/h2>\n
\"Spark<\/a>

Spark Tutorial – Why Spark?<\/p><\/div>\n

As we know, there was no general purpose computing engine in the industry, since<\/span><\/p>\n

    \n
  1. To perform batch processing, we were using Hadoop MapReduce<\/strong><\/a>.<\/span><\/li>\n
  2. Also, to perform stream processing, we were using Apache Storm \/ S4.<\/span><\/li>\n
  3. Moreover, for interactive processing, we were using Apache Impala \/ Apache Tez.<\/span><\/li>\n
  4. To perform graph processing, we were using Neo4j \/ Apache Giraph. <\/span><\/li>\n<\/ol>\n

    Hence there was no powerful engine in the industry, that can process the data both in real-time and batch mode. Also, there was a requirement that one engine can respond in sub-second and perform in-memory processing.<\/span>
    \nTherefore, Apache Spark programming enters, it is a powerful open source engine. Since, it offers real-time stream processing, interactive processing, graph processing, in-memory processing as well as batch processing. Even with very fast speed, ease of use and standard interface. Basically, these features create the difference between Hadoop and Spark. Also makes a huge     comparison between Spark vs Storm.<\/strong><\/a><\/span><\/p>\n

    5. Apache Spark Components<\/h2>\n

    In this Apache Spark Tutorial, we discuss Spark Components. It puts the promise for faster data processing as well as easier development. It is only possible because of its components. All these Spark components resolved the issues that occurred while using Hadoop MapReduce.<\/span><\/p>\n

    Now let\u2019s discuss each Spark Ecosystem Component one by one-<\/span><\/p>\n

    \"Spark<\/a>

    Spark Tutorial – Apache Spark Ecosystem Components<\/p><\/div>\n

    a. Spark Core<\/h3>\n

    Spark Core is a central point of Spark. Basically, it provides an execution platform for all the Spark applications. Moreover, to support a wide array of applications, Spark Provides a \u00a0generalized platform.<\/span><\/p>\n

    b. Spark SQL<\/h3>\n

    On the top of Spark, Spark SQL<\/strong> enables users to run SQL\/HQL queries. We can process structured as well as semi-structured data, by using Spark SQL. Moreover, it offers to run unmodified queries up to 100 times faster on existing deployments. To learn Spark SQL in detail<\/strong><\/a>, follow this link.<\/span><\/p>\n

    c. Spark Streaming<\/h3>\n

    Basically, across live streaming, Spark Streaming enables a powerful interactive and data analytics application. Moreover, the live streams are converted into micro-batches those are executed on top of spark core. Learn Spark Streaming in detail.\u00a0<\/strong><\/a><\/span><\/p>\n

    d. Spark MLlib<\/h3>\n

    Machine learning library delivers both efficiencies as well as the high-quality algorithms. Moreover, it is the hottest choice for a data scientist. Since it is capable of in-memory data processing, that improves the performance of iterative algorithm drastically.<\/span><\/p>\n

    e. Spark GraphX<\/h3>\n

    Basically, Spark GraphX is the graph computation engine built on top of Apache Spark that enables to process graph data at scale.<\/span><\/p>\n

    f. SparkR<\/h3>\n

    Basically, to use Apache Spark from R<\/a><\/strong>. It is R package<\/strong><\/a> that gives light-weight frontend. Moreover, it allows data scientists to analyze large datasets. Also allows running jobs interactively on them from the R shell. Although, the main idea behind SparkR was to explore different techniques to integrate the usability of R with the scalability of Spark. Follow the link to learn SparkR in detail.<\/strong><\/a><\/span><\/p>\n

    To learn about all the components of Spark in detail, follow link\u00a0Apache Spark Ecosystem \u2013 Complete Spark Components Guide<\/strong><\/a><\/p>\n

    6. Resilient Distributed Dataset \u2013 RDD<\/h2>\n

    The key abstraction of Spark is RDD. RDD is an acronym for Resilient Distributed Dataset. It is the fundamental unit of data in Spark. Basically, it is a distributed collection of elements across cluster nodes. Also performs parallel operations. Moreover, Spark RDDs are immutable in nature. Although, it can generate new RDD by transforming existing Spark RDD.Learn about Spark RDDs in detail.<\/strong><\/a><\/p>\n

    a. Ways to create Spark RDD<\/h3>\n

    Basically, there are 3 ways to create Spark RDDs
    \n<\/b><\/p>\n

    i. Parallelized collections<\/b><\/p>\n

    By invoking parallelize method in the driver program, we can create parallelized collections.<\/span>
    \n<\/b><\/p>\n

    ii. External datasets<\/b><\/p>\n

    One can create Spark RDDs, by calling a textFile method. Hence, this method takes URL of the file and reads it as a collection of lines.<\/span>
    \n<\/b><\/p>\n

    iii. Existing RDDs<\/b><\/p>\n

    Moreover, we can create new RDD in spark, by applying transformation operation on existing RDDs.<\/span><\/p>\n

    To learn all three ways to create RDD<\/strong><\/a> in detail, follow the link.<\/p>\n

    b. Spark RDDs operations<\/h3>\n

    There are two types of operations, which Spark RDDs supports:<\/span>
    \n<\/b><\/p>\n

    i. Transformation Operations<\/b><\/p>\n

    It creates a new Spark RDD from the existing one. Moreover, it passes the dataset to the function and returns new dataset.<\/span>
    \n<\/b><\/p>\n

    ii. Action\u00a0Operations <\/b><\/p>\n

    In Apache Spark, Action returns final result to driver program or write it to the external data store.<\/p>\n

    Learn RDD Operations in detail.<\/strong><\/a><\/p>\n

    c. Sparkling Features of Spark RDD<\/h3>\n

    There are various advantages of using RDD. Some of them are<\/span><\/p>\n

    \"Spark<\/a>

    Spark Tutorial – Spark RDD Features<\/p><\/div>\n

    i.\u00a0In-memory computation<\/strong><\/p>\n

    Basically, while storing data in RDD, data is stored in memory for as long as you want to store. It improves the performance by an order of magnitudes by keeping the data\u00a0in memory.<\/span>
    \n<\/b><\/p>\n

    ii. Lazy Evaluation<\/b><\/p>\n

    Spark Lazy Evaluation means the data inside RDDs are not evaluated on the go. Basically, only after an action triggers all the changes or the computation is performed. Therefore, it limits how much work it has to do. learn Lazy Evaluation in detail.\u00a0<\/a><\/strong><\/span>
    \n<\/b><\/p>\n

    iii. Fault Tolerance<\/b><\/p>\n

    If any worker node fails, by using lineage of operations, we can re-compute the lost partition of RDD from the original one. Hence, it is possible to recover lost data easily. Learn Fault Tolerance in detail<\/strong><\/a>.<\/span>
    \n<\/b><\/p>\n

    iv. Immutability<\/b><\/p>\n

    Immutability means once we create an RDD, we can not manipulate it. Moreover, we can create a new RDD by performing any transformation. Also, we achieve consistency through immutability.<\/span>
    \n<\/b><\/p>\n

    v. Persistence<\/b><\/p>\n

    In in-memory, we can store the frequently used RDD. Also, we can retrieve them directly from memory without going to disk. It results in the speed of the execution. Moreover, we can perform multiple operations on the same data. It is only possible by storing the data explicitly in memory by calling persist() or cache() function. <\/span><\/p>\n

    Learn\u00a0Persistence and Caching Mechanism<\/a> <\/strong>in detail.<\/span>
    \n<\/b><\/p>\n

    vi. Partitioning<\/b><\/p>\n

    Basically, RDD partition the records logically. Also, distributes the data across various nodes in the cluster. Moreover, the logical divisions are only for processing and internally it has no division. Hence, it provides parallelism.<\/span>
    \n<\/b><\/p>\n

    vii. Parallel<\/b><\/p>\n

    While we talk about parallel processing, RDD processes the data parallelly over the cluster.<\/span>
    \n<\/b><\/p>\n

    viii. Location-Stickiness<\/b><\/p>\n

    To compute partitions, RDDs are capable of defining placement preference. Moreover, placement preference refers to information about the location of RDD. Although, the DAGScheduler places the partitions in such a way that task is close to data as much as possible. Moreover, it speeds up computation.<\/span>
    \n<\/b><\/p>\n

    ix. Coarse-grained Operation<\/b><\/p>\n

    Generally, we apply coarse-grained transformations to Spark RDD. It means the operation applies to the whole dataset not on the single element in the data set of RDD in Spark.<\/span>
    \n<\/b><\/p>\n

    x. Typed<\/b><\/p>\n

    There are several types of Spark RDD. Such as: RDD [int], RDD [long], RDD [string].<\/span>
    \n<\/b><\/p>\n

    xi. No limitation<\/b><\/p>\n

    There are no limitations to use the number of Spark RDD. We can use any no. of RDDs. Basically, the limit depends on the size of disk and memory.<\/span><\/p>\n

    In this Apache Spark tutorial, we cover most Features of Spark RDD to learn more about RDD Features<\/strong><\/a> follow this link.<\/p>\n

    7. Spark Tutorial – Spark Streaming<\/h2>\n

    While data is arriving continuously in an unbounded sequence is what we call a data stream. Basically, for further processing, Streaming divides continuous flowing input data into discrete units. Moreover, we can say it is a low latency processing and analyzing of streaming data.<\/span><\/p>\n

    In addition, an extension of the core Spark API Streaming was added to Apache Spark in 2013. That offers scalable, fault-tolerant and high-throughput processing of live data streams. Although, here we can do data ingestion from many sources. Such as Kafka, Apache Flume<\/strong><\/a>, Amazon Kinesis or TCP sockets. However, we do processing here by using complex algorithms which are expressed with high-level functions such as map, reduce, join and window.<\/span><\/p>\n

    a. Internal working of Spark Streaming<\/h3>\n

    Let\u2019s understand its internal working. While live input data streams are received. It further divided into batches by Spark streaming, Afterwards, these batches are processed by the Spark engine to generate the final stream of results in batches.<\/span><\/p>\n

    b. Discretized Stream (DStream)<\/h3>\n

    Apache Spark Discretized Stream is the key abstraction of Spark Streaming.\u00a0That is\u00a0what we call Spark DStream. Basically, it represents a stream of data divided into small batches. Moreover, DStreams are built on Spark RDDs, Spark\u2019s core data abstraction. It also allows Streaming to seamlessly integrate with any other Apache Spark components. Such as Spark MLlib and Spark SQL.<\/span><\/p>\n

    Follow this link, to Learn Concept of\u00a0Dstream in detail.\u00a0<\/strong><\/a><\/p>\n

    8. Features of Apache Spark<\/h2>\n

    There are several sparkling Apache Spark features:<\/span><\/p>\n

    \"Apache<\/a>

    Apache Spark Tutorial – Features of Apache Spark<\/p><\/div>\n

    a.\u00a0Swift Processing<\/strong><\/p>\n

    Apache Spark offers high data processing speed. That is about 100x faster in memory and 10x faster on the disk. However, it is only possible by reducing the number of read-write to disk.\u00a0<\/span><\/p>\n

    b. Dynamic in Nature<\/strong><\/p>\n

    Basically, it is possible to develop a parallel application in Spark. Since there are 80 high-level operators available in Apache Spark.<\/span><\/p>\n

    c. In-Memory Computation in Spark<\/strong><\/p>\n

    The increase in processing speed is possible due to in-memory processing<\/strong><\/a>. It enhances the processing speed.<\/span><\/p>\n

    d. Reusability<\/strong><\/p>\n

    We can easily reuse spark code for batch-processing or join stream against historical data. Also to run ad-hoc queries on stream state.<\/span><\/p>\n

    e. Spark Fault Tolerance\u00a0<\/strong><\/p>\n

    Spark offers fault tolerance. It is possible through Spark\u2019s core abstraction-RDD. Basically, to handle the failure of any worker node in the cluster, Spark RDDs are designed. Therefore, the loss of data is reduced to zero.\u00a0<\/span><\/p>\n

    f. Real-Time Stream Processing<\/strong><\/p>\n

    We can do real-time stream processing in Spark. Basically, Hadoop does not support real-time processing. It can only process data which is already present. Hence with Spark Streaming, we can solve this problem.<\/span><\/p>\n

    g. Lazy Evaluation in Spark<\/strong><\/p>\n

    All the transformations we make in Spark RDD are Lazy in nature, that is it does not give the result right away rather a new RDD is formed from the existing one. Thus, this increases the efficiency of the system.<\/span><\/p>\n

    h. Support Multiple Languages<\/strong><\/p>\n

    Spark supports multiple languages. Such as Java, R, Scala<\/strong><\/a>, Python. Hence, it shows dynamicity. Moreover, it also overcomes the limitations of Hadoop<\/strong><\/a>\u00a0since it can only build applications in Java.<\/a><\/strong><\/span><\/p>\n

    i. Support for Sophisticated Analysis<\/strong><\/p>\n

    There are dedicated tools in Apache Spark. Such as for streaming data interactive\/declarative queries, machine learning which add-on to map and reduce.<\/span><\/p>\n

    j. Integrated with Hadoop<\/strong><\/p>\n

    As we know Spark is flexible. It can run independently and also on Hadoop YARN Cluster Manager. Even it can read existing Hadoop data.<\/span><\/p>\n

    k. Spark GraphX<\/strong><\/p>\n

    In Spark, a component for graph and graph-parallel computation, we have GraphX. Basically, it simplifies the graph analytics tasks by the collection of graph algorithm and builders.<\/span><\/p>\n

    l. Cost Efficient<\/strong><\/p>\n

    For Big data problem as in Hadoop, a large amount of storage and the large data center is required during replication. Hence, Spark programming turns out to be a cost-effective solution.<\/span><\/p>\n

    Learn All features of Apache Spark, in detail.\u00a0<\/strong><\/a><\/p>\n

    9. Limitations of Apache Spark Programming<\/h2>\n

    There are many limitations of Apache Spark. Let\u2019s learn all one by one:<\/span><\/p>\n

    \"Spark<\/a>

    Spark Tutorial – Limitations of Apache Spark Programming<\/p><\/div>\n

    a. No Support for Real-time Processing<\/strong><\/p>\n

    Basically, Spark is near real-time processing of live data. In other words, Micro-batch processing takes place in Spark Streaming. Hence we can not say Spark is completely Real-time Processing engine.<\/span><\/p>\n

    b. Problem with Small File<\/strong><\/p>\n

    In RDD, each file is a small partition. It means, there is the large amount of tiny partition within an RDD. Hence, if we want efficiency in our processing, the RDDs should be repartitioned into some manageable format. Basically, that demands extensive shuffling over the network.<\/span><\/p>\n

    c. No File Management System<\/strong><\/p>\n

    A major issue is Spark does not have its own file management system. Basically, it relies on some other platform like Hadoop or another cloud-based platform.<\/span><\/p>\n

    d. Expensive<\/strong><\/p>\n

    While we desire cost-efficient processing of big data, Spark turns out to be very expensive. Since keeping data in memory is quite expensive. However the memory consumption is very high, and it is not handled in a user-friendly manner. Moreover, we require lots of RAM to run in-memory, thus the cost of spark is much higher.<\/span><\/p>\n

    e<\/strong>. Less number of Algorithms<\/strong><\/p>\n

    Spark MLlib<\/strong><\/a> have very less number of available algorithms. For example, Tanimoto distance.<\/span><\/p>\n

    f. Manual Optimization<\/strong><\/p>\n

    It is must that Spark job is manually optimized and is adequate to specific datasets. Moreover, to partition and cache in spark to be correct, it is must to control it manually.<\/span><\/p>\n

    g. Iterative Processing<\/strong><\/p>\n

    Basically, here data iterates in batches. Also, each iteration is scheduled and executed separately.<\/span><\/p>\n

    h. Latency<\/strong><\/p>\n

    On comparing with Flink<\/strong><\/a>, Apache Spark has higher latency.<\/span><\/p>\n

    i. Window Criteria<\/strong><\/p>\n

    Spark only support time-based window criteria not record based window criteria. <\/span>
    \n<\/b><\/p>\n

    Note<\/b>: To overcome these limitations of Spark, we can use Apache Flink \u2013 4G of Big Data<\/strong><\/a>.<\/span><\/p>\n

    Learn All\u00a0Limitations of Apache Spark, in detail. <\/strong><\/a><\/p>\n

    10. Apache Spark Tutorial – Use Cases<\/h2>\n

    There are many industry-specific Apache Spark use cases, let\u2019s discuss them one by one:<\/span><\/p>\n

    \"Spark<\/a>

    Spark Tutorial – Apache Spark Use Cases<\/p><\/div>\n

    a. Spark Use Cases in the Finance Industry<\/strong><\/p>\n

    There are many banks those are using Spark. Basically, it helps to access and analyze many of the parameters in the bank sector like the emails, social media profiles, call recordings, forum, and many more. Further, it helps to make right decisions for several zones.<\/span><\/p>\n

    b. Apache Spark Use Cases in E-Commerce Industry<\/strong><\/p>\n

    Basically, it helps with information about a real-time transaction. Moreover, those are passed to streaming clustering algorithms. <\/span><\/p>\n

    c. Apache Spark Use Cases in Media & Entertainment Industry<\/strong><\/p>\n

    We use Spark to identify patterns from the real-time in-game events. Moreover, it helps to respond in order to harvest lucrative business opportunities. <\/span><\/p>\n

    d. Apache Spark Use Cases in Travel Industry<\/strong><\/p>\n

    Basically, travel industries are using spark rapidly. Moreover, it helps users to plan a perfect trip by speed up the personalized recommendations. Although, its review process of the hotels in a readable format is done by using Spark.<\/span><\/p>\n

    Apache Spark tutorial cover Spark real-time use Cases, there are many more, follow the link to learn all in detail.\u00a0Apache Spark use cases in real time<\/strong><\/a><\/p>\n

    11. Spark Tutorial – Conclusion<\/h2>\n

    As a result, we have seen every aspect of Apache Spark,\u00a0what is Apache spark programming and spark definition, History of Spark, why Spark is needed, Components of Apache Spark, Spark RDD, Features of Spark RDD, Spark Streaming, Features of Apache Spark, Limitations of Apache Spark, Apache Spark use cases. In this tutorial we were trying to cover all spark notes, hope you get desired information in it if you feel to ask any query, feel free to ask in the comment section.<\/span><\/p>\n

    Best Books to Learn Spark<\/strong><\/a>.<\/p>\n

    For reference<\/strong><\/a>

    1. Objective – Spark Tutorial In this Spark Tutorial, we will see an overview\u00a0of Spark in Big Data. We will start with an introduction to Apache Spark Programming. Then we will move to know...<\/p>\n","protected":false},"author":6,"featured_media":9732,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10],"tags":[148,930,935,958,960,2805,4577,4635,5655,6882,8176,8241,13050,13063,13072,13091,13092,13097,13104,13130,13142,15594,15959,16183],"class_list":["post-5883","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-spark","tag-a-complete-guide-on-apache-spark","tag-apache-spark-introduction","tag-apache-spark-notes","tag-apache-spark-tutorial","tag-apache-spark-use-cases","tag-components-of-apache-spark","tag-features-of-apache-spark","tag-features-of-spark-rdd","tag-history-of-apache-spark","tag-internal-working-of-spark","tag-learn-spark","tag-limitations-of-apache-spark","tag-spark-definition","tag-spark-history","tag-spark-introduction","tag-spark-notes","tag-spark-overview","tag-spark-programming","tag-spark-rdd","tag-spark-streaming","tag-spark-tutorial","tag-what-is-apache-spark","tag-what-is-spark","tag-why-spark-is-needed"],"yoast_head":"\nSpark Tutorial - Learn Spark Programming - DataFlair<\/title>\n<meta name=\"description\" content=\"Apache Spark Tutorial-what is spark, Spark overview, spark History, why Spark, Spark Component, Spark RDD, Spark Features, Spark Limitation, Spark use case\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/spark-tutorial\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Spark Tutorial - Learn Spark Programming - DataFlair\" \/>\n<meta property=\"og:description\" content=\"Apache Spark Tutorial-what is spark, Spark overview, spark History, why Spark, Spark Component, Spark RDD, Spark Features, Spark Limitation, Spark use case\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/spark-tutorial\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-01-18T12:06:55+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-09-04T05:44:26+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/03\/Spark-Tutorial-01.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"15 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Spark Tutorial - Learn Spark Programming - DataFlair","description":"Apache Spark Tutorial-what is spark, Spark overview, spark History, why Spark, Spark Component, Spark RDD, Spark Features, Spark Limitation, Spark use case","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/","og_locale":"en_US","og_type":"article","og_title":"Spark Tutorial - Learn Spark Programming - DataFlair","og_description":"Apache Spark Tutorial-what is spark, Spark overview, spark History, why Spark, Spark Component, Spark RDD, Spark Features, Spark Limitation, Spark use case","og_url":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-01-18T12:06:55+00:00","article_modified_time":"2019-09-04T05:44:26+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/03\/Spark-Tutorial-01.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"15 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89"},"headline":"Spark Tutorial – Learn Spark Programming","datePublished":"2018-01-18T12:06:55+00:00","dateModified":"2019-09-04T05:44:26+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/"},"wordCount":2907,"commentCount":31,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/03\/Spark-Tutorial-01.jpg","keywords":["A complete guide on Apache Spark","Apache Spark Introduction","Apache Spark Notes","apache spark tutorial","Apache Spark use cases","Components of Apache Spark","Features of Apache Spark","Features of Spark RDD","History of Apache Spark","internal working of Spark","learn spark","Limitations of Apache Spark","spark definition","spark history","spark introduction","spark notes","spark overview","spark programming","spark rdd","spark streaming","spark tutorial","what is apache spark","what is spark","why Spark is needed"],"articleSection":["Apache Spark Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/spark-tutorial\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/","url":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/","name":"Spark Tutorial - Learn Spark Programming - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/03\/Spark-Tutorial-01.jpg","datePublished":"2018-01-18T12:06:55+00:00","dateModified":"2019-09-04T05:44:26+00:00","description":"Apache Spark Tutorial-what is spark, Spark overview, spark History, why Spark, Spark Component, Spark RDD, Spark Features, Spark Limitation, Spark use case","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/spark-tutorial\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/03\/Spark-Tutorial-01.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/03\/Spark-Tutorial-01.jpg","width":1200,"height":628},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/spark-tutorial\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Apache Spark Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/spark\/"},{"@type":"ListItem","position":3,"name":"Spark Tutorial – Learn Spark Programming"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"The DataFlair Team provides industry-driven content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our expert educators focus on delivering value-packed, easy-to-follow resources for tech enthusiasts and professionals.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam2\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/5883","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=5883"}],"version-history":[{"count":7,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/5883\/revisions"}],"predecessor-version":[{"id":69099,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/5883\/revisions\/69099"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/9732"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=5883"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=5883"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=5883"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}