{"id":13766,"date":"2018-05-05T10:56:47","date_gmt":"2018-05-05T10:56:47","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=13766"},"modified":"2018-05-05T10:56:47","modified_gmt":"2018-05-05T10:56:47","slug":"apache-pig-execution","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/","title":{"rendered":"Learn Apache Pig Execution: Modes and Mechanism"},"content":{"rendered":"

After installing the Apache Pig<\/strong>, here we will discuss the whole concept of Apache Pig Execution. Apart from its introduction, we will include Pig Execution Modes, Execution Mechanisms, and the way to Execute Apache Pig in Batch Mode in depth.\u00a0<\/span><\/p>\n

So, let’s discuss Apache Pig Execution in detail.
\nLet’s learn\u00a0Apache Pig Installation on Ubuntu – A Pig Tutorial<\/a><\/p>\n

Introduction to Apache Pig Execution<\/span><\/h2>\n

As we all know, firstly, the developer creates the scripts further, it goes to the local file system as functions. Also, when the developers submit Pig Script, it contacts with Pig Latin Compiler.
\n<\/span><\/p>\n

Further, the compiler then splits the task and run a series of MR jobs. In that same duration, Pig Compiler retrieves data from the HDFS. Then, after running MR jobs output file again goes to the HDFS<\/strong>.<\/span>
\nLet’s Explore\u00a0Apache Pig Advantages and Disadvantages<\/a><\/p>\n

Apache Pig Execution Modes<\/span><\/h2>\n

Moreover, there are two modes in Apache Pig Execution, in which we can run Apache Pig such as, Local Mode and HDFS mode. Let\u2019s discuss both in detail:<\/span><\/p>\n

a. Local Mode<\/span><\/h3>\n

Basically, in this mode, all the files are installed and run on your localhost and local file system. That implies we do not need \u00a0Hadoop<\/strong> or HDFS any more. Also, we can say we generally use this mode for testing purpose.<\/span><\/p>\n

In other words, the pig implements on single JVM and accesses the file system, in this mode. Especially, for dealing with the small data sets, Local mode is better. In the same duration, the parallel mapper execution is impossible. <\/span>However, the previous version of the Hadoop is not thread-safe.<\/span><\/p>\n

At the same place, the user can offer \u2013x local to get into Pig local mode of execution. Hence, Pig always looks for the local file system path while loading data.<\/span><\/p>\n

b. MapReduce Mode<\/span><\/h3>\n

Basically, while we load or process the data that exists in the Hadoop File System (HDFS) using Apache Pig, is MapReduce\u00a0<\/strong>mode. <\/span><\/p>\n

Also, while we execute the Pig Latin statements to process the data, a MapReduce job is invoked in the back-end to perform a particular operation on the data that exists in the HDFS, in this mode.<\/span><\/p>\n

To be more specific, in this mode, a user could have proper Hadoop cluster setup and installations. By default, Apache pig installs as in MR mode. In addition, Pig translates the queries into MapReduce jobs and runs on top of Hadoop cluster. Hence, this mode as a MapReduce runs on a distributed cluster.<\/span><\/p>\n

Apache Pig Execution Mechanisms<\/span><\/h2>\n

There are three ways, in which Apache Pig scripts can be executed such as interactive mode, batch mode, and embedded mode.<\/span><\/p>\n

a. Interactive Mode (Grunt shell) <\/span><\/h3>\n

By using the Grunt shell, we can run Apache Pig in interactive mode. By using Dump operator, we can enter the Pig Latin statements and get the output, in this shell.\u00a0 <\/span><\/p>\n

b. Batch Mode (Script) <\/span><\/h3>\n

Also, by writing the Pig Latin script in a single file with the .pig extension, we can run Apache Pig in Batch mode.<\/span><\/p>\n

c. Embedded Mode (UDF) <\/span><\/h3>\n

By using User Defined Functions in our script, Pig offers the provision of defining our own functions (User Defined Functions) in programming languages such as Java.<\/span>
\nRead Introduction to Apache Pig Architecture <\/a><\/p>\n

Invoking the Grunt Shell<\/span><\/h2>\n

By using the \u2212x option, we can invoke the Grunt shell in the desired mode (local\/MapReduce).<\/span>
\nCommand<\/strong><\/span>
\n\u00a0 1. Local mode<\/span><\/strong><\/p>\n

$ .\/pig \u2013x local<\/pre>\n
\u00a0 2. MapReduce mode<\/strong><\/span><\/p>\n
$ .\/pig -x mapreduce<\/pre>\n
These commands give us the Grunt shell prompt.<\/span><\/p>\n
grunt><\/pre>\n
Moreover, \u00a0using \u2018ctrl + d\u2019, we can exit the Grunt shell.<\/span>
\nAlso, we can execute a Pig script by directly entering the Pig Latin statements in it, after invoking the Grunt shell.<\/span><\/p>\n
grunt> customers = LOAD 'customers.txt' USING PigStorage(',');<\/pre>\n
Executing Apache Pig in Batch Mode<\/span><\/h2>\n
Further, using the \u2013x command, we can write an entire Pig Latin script in a file and execute it. Let\u2019s assume we have a Pig script in a file named sample_script.pig. <\/span><\/p>\n
Sample_script.pig\nstudent = LOAD 'hdfs:\/\/localhost:9000\/pig_data\/student.txt' USING\n  PigStorage(',') as (id:int,name:chararray,city:chararray);\nDump student;<\/pre>\n
Then, we can execute the script in the above file.<\/span>
\n\u00a0 \u00a01. Local mode<\/span><\/strong><\/p>\n
$ pig -x local Sample_script.pig<\/pre>\n
\u00a0\u00a0 2.MapReduce mode <\/strong><\/span><\/p>\n
$ pig -x MapReduce Sample_script.pig<\/pre>\n
So, this was all about Apache Pig Execution. Hope you like our explanation.<\/p>\n
Conclusion – Pig Execution<\/span><\/h2>\n
As a result, we have seen the whole way of Execution in Apache Pig. Moreover, we discussed Apache Pig Execution Modes, Execution Mechanisms in detail. Still, if you have any doubt, feel free to ask in the comment section.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"
After installing the Apache Pig, here we will discuss the whole concept of Apache Pig Execution. Apart from its introduction, we will include Pig Execution Modes, Execution Mechanisms, and the way to Execute Apache...<\/p>\n","protected":false},"author":7,"featured_media":35443,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[40],"tags":[869,871,874,16663,7166,16664,16665,9501],"class_list":["post-13766","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-pig","tag-apache-pig-execution","tag-apache-pig-execution-modes","tag-apache-pig-in-batch-mode","tag-execution-in-pig","tag-invoking-the-grunt-shell","tag-local-mode","tag-mapreduce-mode","tag-pig-execution"],"yoast_head":"\nLearn Apache Pig Execution: Modes and Mechanism - DataFlair<\/title>\n<meta name=\"description\" content=\"Apache Pig Execution, Pig Execution Modes: Local Mode, MapReduce Mode, Apache Pig Execution mechanisms, Invoking Grunt shell, Executing Pig in Batch mode\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Learn Apache Pig Execution: Modes and Mechanism - DataFlair\" \/>\n<meta property=\"og:description\" content=\"Apache Pig Execution, Pig Execution Modes: Local Mode, MapReduce Mode, Apache Pig Execution mechanisms, Invoking Grunt shell, Executing Pig in Batch mode\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-05-05T10:56:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Pig-Execution-01-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Learn Apache Pig Execution: Modes and Mechanism - DataFlair","description":"Apache Pig Execution, Pig Execution Modes: Local Mode, MapReduce Mode, Apache Pig Execution mechanisms, Invoking Grunt shell, Executing Pig in Batch mode","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/","og_locale":"en_US","og_type":"article","og_title":"Learn Apache Pig Execution: Modes and Mechanism - DataFlair","og_description":"Apache Pig Execution, Pig Execution Modes: Local Mode, MapReduce Mode, Apache Pig Execution mechanisms, Invoking Grunt shell, Executing Pig in Batch mode","og_url":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-05-05T10:56:47+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Pig-Execution-01-1.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/beb0cab24b7aa54423a3b50e669a9dcd"},"headline":"Learn Apache Pig Execution: Modes and Mechanism","datePublished":"2018-05-05T10:56:47+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/"},"wordCount":721,"commentCount":0,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Pig-Execution-01-1.jpg","keywords":["Apache Pig Execution","Apache Pig Execution Modes","Apache Pig in Batch Mode","Execution in Pig","Invoking the Grunt Shell","Local mode","MapReduce Mode","Pig Execution"],"articleSection":["Pig Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/apache-pig-execution\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/","url":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/","name":"Learn Apache Pig Execution: Modes and Mechanism - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Pig-Execution-01-1.jpg","datePublished":"2018-05-05T10:56:47+00:00","description":"Apache Pig Execution, Pig Execution Modes: Local Mode, MapReduce Mode, Apache Pig Execution mechanisms, Invoking Grunt shell, Executing Pig in Batch mode","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/apache-pig-execution\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Pig-Execution-01-1.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Pig-Execution-01-1.jpg","width":1200,"height":628,"caption":"Apache Pig Execution"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/apache-pig-execution\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Pig Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/pig\/"},{"@type":"ListItem","position":3,"name":"Learn Apache Pig Execution: Modes and Mechanism"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/beb0cab24b7aa54423a3b50e669a9dcd","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/c322416204232f4dd97ef3901b0a499a5d34d7ba7fe333f4bfe53a907873d293?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/c322416204232f4dd97ef3901b0a499a5d34d7ba7fe333f4bfe53a907873d293?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c322416204232f4dd97ef3901b0a499a5d34d7ba7fe333f4bfe53a907873d293?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"DataFlair Team specializes in creating clear, actionable content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Backed by industry expertise, we make learning easy and career-oriented for beginners and pros alike.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam3\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/13766","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=13766"}],"version-history":[{"count":0,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/13766\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/35443"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=13766"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=13766"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=13766"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}