{"id":7261,"date":"2018-02-05T07:09:03","date_gmt":"2018-02-05T07:09:03","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=7261"},"modified":"2021-12-07T13:08:41","modified_gmt":"2021-12-07T07:38:41","slug":"sqoop-codegen","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/","title":{"rendered":"Sqoop Codegen – Arguments & Commands in Codegen with Example"},"content":{"rendered":"

A tool which generates Java classes<\/strong><\/a> that encapsulate and interpret imported records is what we call Sqoop Codegen. However, there is much more to know about Sqoop Codegen in Sqoop<\/strong><\/a>. So in this article, we will learn each and every aspect of Codegen. in Sqoop. Apart from the introduction, we will also learn Codegen in Sqoop’s purpose as well as its syntax.<\/span><\/p>\n

Introduction to Sqoop Codegen and its Purpose<\/h3>\n

A tool which generates Java classes that encapsulate and interpret imported records is what we call Sqoop-Codegen. Moreover, as part of the Sqoop import process<\/strong><\/a>, the Java<\/strong> definition<\/strong><\/a> of a record is instantiated. However, we can also perform it separately. Likewise, we can recreate a java source if it gets lost. Also, we can create new versions of a class which uses different delimiters between fields, and so on.<\/span><\/p>\n

Refer this link to know about Sqoop Export<\/a><\/strong>.<\/p>\n

Codegen Syntax in Sqoop<\/h3>\n

$ sqoop codegen (generic-args) (codegen-args)<\/b>
\n$ sqoop-codegen (generic-args) (codegen-args)<\/b>
\nHowever, the Codegen in sqoop arguments can be entered in any order with respect to one another but the Hadoop<\/strong><\/a> generic arguments must precede any Sqoop Codegen arguments only.<\/span><\/p>\n

Sqoop Codegen Arguments<\/h3>\n

a. Common arguments\u00a0in Sqoop Codegen<\/h4>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Argument<\/b><\/td>\nDescription<\/b><\/td>\n<\/tr>\n
–connect <jdbc-uri><\/span><\/td>\nSpecify JDBC connect string<\/span><\/td>\n<\/tr>\n
–connection-manager <class-name><\/span><\/td>\nSpecify connection manager class to use<\/span><\/td>\n<\/tr>\n
–driver <class-name><\/span><\/td>\nManually specify JDBC driver class to use<\/span><\/td>\n<\/tr>\n
–hadoop-mapred-home <dir><\/span><\/td>\nOverride $HADOOP_MAPRED_HOME<\/span><\/td>\n<\/tr>\n
–help<\/span><\/td>\nPrint usage instructions<\/span><\/td>\n<\/tr>\n
–password-file<\/span><\/td>\nSet path for a file containing the authentication password<\/span><\/td>\n<\/tr>\n
-P<\/span><\/td>\nRead password from console<\/span><\/td>\n<\/tr>\n
–password <password><\/span><\/td>\nSet authentication password<\/span><\/td>\n<\/tr>\n
–username <username><\/span><\/td>\nSet authentication username<\/span><\/td>\n<\/tr>\n
–verbose<\/span><\/td>\nPrint more information while working<\/span><\/td>\n<\/tr>\n
–connection-param-file <filename><\/span><\/td>\nOptional properties file that provides connection parameters<\/span><\/td>\n<\/tr>\n
–relaxed-isolation<\/span><\/td>\nSet connection transaction isolation to read uncommitted for the mappers.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

b. Code generation arguments\u00a0in Sqoop Codegen<\/h4>\n\n\n\n\n\n\n\n\n\n
Argument<\/strong><\/td>\nDescription<\/strong><\/td>\n<\/tr>\n
–bindir <dir><\/span><\/td>\nOutput directory for compiled objects<\/span><\/td>\n<\/tr>\n
–class-name <name><\/span><\/td>\nSets the generated class name. This overrides –package-name. When combined with –jar-file, sets the input class.<\/span><\/td>\n<\/tr>\n
–jar-file <file><\/span><\/td>\nDisable code generation; use specified jar<\/span><\/td>\n<\/tr>\n
–outdir <dir><\/span><\/td>\nOutput directory for generated code<\/span><\/td>\n<\/tr>\n
–package-name <name><\/span><\/td>\nPut auto-generated classes in this package<\/span><\/td>\n<\/tr>\n
–map-column-java <m><\/span><\/td>\nOverride default mapping from SQL type to Java type for configured columns<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

c. Output line formatting arguments in Sqoop Codegen<\/h4>\n\n\n\n\n\n\n\n\n\n
Argument<\/b><\/td>\nDescription<\/b><\/td>\n<\/tr>\n
–enclosed-by <char><\/span><\/td>\nSets a required field enclosing character<\/span><\/td>\n<\/tr>\n
–escaped-by <char><\/span><\/td>\nSets the escape character<\/span><\/td>\n<\/tr>\n
–fields-terminated-by <char><\/span><\/td>\nSets the field separator character<\/span><\/td>\n<\/tr>\n
–lines-terminated-by <char><\/span><\/td>\nSets the end-of-line character<\/span><\/td>\n<\/tr>\n
–mysql-delimiters<\/span><\/td>\nUses MySQL\u2019s default delimiter set: fields: , lines: \\n escaped-by: \\ optionally-enclosed-by: ‘<\/span><\/td>\n<\/tr>\n
–optionally-enclosed-by <char><\/span><\/td>\nSets a field enclosing character<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

d. Input parsing arguments\u00a0in Sqoop Codegen<\/h4>\n\n\n\n\n\n\n\n\n
Argument<\/b><\/td>\nDescription<\/b><\/td>\n<\/tr>\n
–input-enclosed-by <char><\/span><\/td>\nSets a required field encloser<\/span><\/td>\n<\/tr>\n
–input-escaped-by <char><\/span><\/td>\nSets the input escape character<\/span><\/td>\n<\/tr>\n
–input-fields-terminated-by <char><\/span><\/td>\nSets the input field separator<\/span><\/td>\n<\/tr>\n
–input-lines-terminated-by <char><\/span><\/td>\nSets the input end-of-line character<\/span><\/td>\n<\/tr>\n
–input-optionally-enclosed-by <char><\/span><\/td>\nSets a field enclosing character<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

e. Hive arguments in Sqoop Codegen<\/h4>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Argument<\/b><\/td>\nDescription<\/b><\/td>\n<\/tr>\n
–hive-home <dir><\/span><\/td>\nOverride $HIVE_HOME<\/span><\/td>\n<\/tr>\n
–hive-import<\/span><\/td>\nImport tables into Hive (Uses Hive\u2019s default delimiters if none are set.)<\/span><\/td>\n<\/tr>\n
–hive-overwrite<\/span><\/td>\nOverwrite existing data in the Hive table.<\/span><\/td>\n<\/tr>\n
–create-hive-table<\/span><\/td>\nIf set, then the job will fail if the target hive<\/span><\/td>\n<\/tr>\n
<\/td>\ntable exits. By default this property is false.<\/span><\/td>\n<\/tr>\n
–hive-table <table-name><\/span><\/td>\nSets the table name to use when importing to Hive.<\/span><\/td>\n<\/tr>\n
–hive-drop-import-delims<\/span><\/td>\nDrops \\n, \\r, and \\01 from string fields when importing to Hive.<\/span><\/td>\n<\/tr>\n
–hive-delims-replacement<\/span><\/td>\nReplace \\n, \\r, and \\01 from string fields with user-defined string when importing to Hive.<\/span><\/td>\n<\/tr>\n
–hive-partition-key<\/span><\/td>\nName of a hive field to partition is shared on<\/span><\/td>\n<\/tr>\n
–hive-partition-value <v><\/span><\/td>\nString-value that serves as partition key for this imported into the hive in this job.<\/span><\/td>\n<\/tr>\n
–map-column-hive <map><\/span><\/td>\nOverride default mapping from SQL type to Hive type for configured columns.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Basically, Sqoop generates a file containing the HQL statements to create a table and load data only if Hive<\/strong><\/a> arguments are provided to the code generation tool.<\/span><\/p>\n

Sqoop Codegen Commands<\/h3>\n

All these commands actually define the importance of Codegen tool. Basically, every database table has one DAO class that carry getter and setter methods. <\/span>
\na. Change the directory to \/usr\/local\/hadoop\/sbin<\/span>
\n$ cd \/usr\/local\/hadoop\/sbin<\/b>
\nb. Start all hadoop daemons<\/span>
\n $ start-all.sh<\/b>
\nc. The JPS (Java Virtual Machine Process Status Tool) tool is limited to reporting information on JVMs for which it has the access permissions<\/span>
\n$ jps<\/b>
\nd. Change the directory to \/usr\/local\/sqoop\/bin<\/span>
\n$ cd \/usr\/local\/sqoop\/bin<\/b><\/p>\n

Codegen Example Invocations in Sqoop<\/h3>\n

Here, for the employees table of a corporate database we are recreating the record interpretation code.<\/span>
\n$ sqoop codegen –connect jdbc:mysql:\/\/db.example.com\/corp \\<\/b>
\n \u00a0\u00a0\u00a0–table employees<\/b><\/p>\n

Conclusion<\/h3>\n

As a result, we have seen a complete introduction to Sqoop Codegen. Also, we have studied purpose and syntax of codegen in Sqoop. Likewise, we have learned all the arguments and commands associated with Sqoop Codegen. But if you want to ask any query regarding, feel free to ask in the comment section.<\/p>\n

Learn about Sqoop eval commands<\/a>\u00a0<\/strong>& Sqoop Books<\/a><\/strong>
\nFor reference<\/a><\/strong>

A tool which generates Java classes that encapsulate and interpret imported records is what we call Sqoop Codegen. However, there is much more to know about Sqoop Codegen in Sqoop. So in this article,...<\/p>\n","protected":false},"author":9,"featured_media":7361,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[68],"tags":[2625,4244,7134,13605,13606,13607,13688],"class_list":["post-7261","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sqoop","tag-codegen-syntax","tag-example-invocations","tag-introduction-to-sqoop-codegen","tag-sqoop-codegen-arguments","tag-sqoop-codegen-commands","tag-sqoop-codegen-examples","tag-sqoop-codegen"],"yoast_head":"\nSqoop Codegen - Arguments & Commands in Codegen with Example - DataFlair<\/title>\n<meta name=\"description\" content=\"Sqoop Codegen-What is sqoop codegen and its purpose, codegen syntax in sqoop, sqoop codegen commands and example, sqoop codegen arguements with commands.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Sqoop Codegen - Arguments & Commands in Codegen with Example - DataFlair\" \/>\n<meta property=\"og:description\" content=\"Sqoop Codegen-What is sqoop codegen and its purpose, codegen syntax in sqoop, sqoop codegen commands and example, sqoop codegen arguements with commands.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-02-05T07:09:03+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-12-07T07:38:41+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Introduction-to-sqoop-codegen-01-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Sqoop Codegen - Arguments & Commands in Codegen with Example - DataFlair","description":"Sqoop Codegen-What is sqoop codegen and its purpose, codegen syntax in sqoop, sqoop codegen commands and example, sqoop codegen arguements with commands.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/","og_locale":"en_US","og_type":"article","og_title":"Sqoop Codegen - Arguments & Commands in Codegen with Example - DataFlair","og_description":"Sqoop Codegen-What is sqoop codegen and its purpose, codegen syntax in sqoop, sqoop codegen commands and example, sqoop codegen arguements with commands.","og_url":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-02-05T07:09:03+00:00","article_modified_time":"2021-12-07T07:38:41+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Introduction-to-sqoop-codegen-01-1.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/6e11a88eea98179e861a0933f8959f8f"},"headline":"Sqoop Codegen – Arguments & Commands in Codegen with Example","datePublished":"2018-02-05T07:09:03+00:00","dateModified":"2021-12-07T07:38:41+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/"},"wordCount":863,"commentCount":0,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Introduction-to-sqoop-codegen-01-1.jpg","keywords":["Codegen Syntax","Example Invocations","Introduction to Sqoop-Codegen","Sqoop Codegen Arguments","Sqoop Codegen Commands","Sqoop codegen examples","Sqoop-Codegen"],"articleSection":["Sqoop Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/sqoop-codegen\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/","url":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/","name":"Sqoop Codegen - Arguments & Commands in Codegen with Example - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Introduction-to-sqoop-codegen-01-1.jpg","datePublished":"2018-02-05T07:09:03+00:00","dateModified":"2021-12-07T07:38:41+00:00","description":"Sqoop Codegen-What is sqoop codegen and its purpose, codegen syntax in sqoop, sqoop codegen commands and example, sqoop codegen arguements with commands.","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/sqoop-codegen\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Introduction-to-sqoop-codegen-01-1.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/Introduction-to-sqoop-codegen-01-1.jpg","width":1200,"height":628,"caption":"What is Sqoop Codegen"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/sqoop-codegen\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Sqoop Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/sqoop\/"},{"@type":"ListItem","position":3,"name":"Sqoop Codegen – Arguments & Commands in Codegen with Example"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/6e11a88eea98179e861a0933f8959f8f","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/54363d8635c383dcd7e3be0ef00dbeef4217c45bf323d27bc2c92366b724df08?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/54363d8635c383dcd7e3be0ef00dbeef4217c45bf323d27bc2c92366b724df08?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/54363d8635c383dcd7e3be0ef00dbeef4217c45bf323d27bc2c92366b724df08?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"At DataFlair, our team of experts is committed to offering high-quality tutorials on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. We simplify learning to help you advance your career in the tech world.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam5\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/7261","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=7261"}],"version-history":[{"count":5,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/7261\/revisions"}],"predecessor-version":[{"id":105344,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/7261\/revisions\/105344"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/7361"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=7261"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=7261"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=7261"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}