{"id":53953,"date":"2019-04-06T15:35:13","date_gmt":"2019-04-06T10:05:13","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=53953"},"modified":"2020-02-05T17:45:38","modified_gmt":"2020-02-05T12:15:38","slug":"hadoop-copyfromlocal-command","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/","title":{"rendered":"3 Easy Steps to Execute Hadoop copyFromLocal Command"},"content":{"rendered":"

Hadoop file system shell commands have a similar structure to Unix commands. People working with Unix shell command find it easy to adapt to Hadoop Shell commands<\/strong><\/a>. These commands interact with HDFS and other file systems supported by Hadoop. For example, local file system, S3 file system and so on. Today, we will explore Hadoop copyFromLocal Command and its use.<\/p>\n

Hadoop copyFromLocal<\/h2>\n

We use this command in Hadoop to copy the file from the local file system to Hadoop Distributed File System (HDFS)<\/strong>. Here is one restriction with this command and that is, the source file can reside only in the local file system.<\/p>\n

coprFromLocal has an optional parameter \u2013f which gets used to replace the files that already exist in the system. This is useful when we have to copy the same file again or update it. By default, the system will throw the error if we try to copy a file in the same directory in which it already exists. One way to update a file is to delete the file and copy it again and another way is to use \u2013f.<\/p>\n

Have a look at most frequently used Hadoop Commands<\/a><\/strong><\/p>\n

Shell Command<\/h2>\n

We can invoke the Hadoop file system by the following command:-<\/p>\n

hadoop fs <args><\/pre>\n
When the command gets executed the output is sent to stdout and errors to stderr. In most cases, both are the console screen.<\/p>\n
We can also use the below method to invoke fs commands which is a synonym to hadoop fs:-<\/p>\n
hdfs dfs -<command> <args><\/pre>\n
Below statement shows the usage of copyFromLocal command:-<\/p>\n
hdfs dfs \u2013copyFromLocal <local-source> URI<\/pre>\n
We can write the command with \u2013f option to overwrite the file if it is already present.<\/p>\n
hdfs dfs \u2013copyFromLocal \u2013f <local-source> URI<\/pre>\n
Recommended Reading – Hadoop Distributed Cache<\/a><\/strong><\/p>\n
Steps to Execute copyFromLocal\u00a0Command<\/h2>\n
We have to perform the following steps to perform copyFromLocal command:-<\/p>\n
1. Make directory<\/h4>\n
hdfs dfs \u2013mkdir \/user\/copy_from_local_example<\/pre>\n
The above command is used to create a directory in HDFS.<\/p>\n
hdfs dfs \u2013ls \/user<\/pre>\n
The above command is to check if the directory is created in HDFS.<\/p>\n
2. Copying the local file into the directory in HDFS<\/h4>\n
hdfs dfs \u2013copyFromLocal testfile.txt \/user\/copy_from_local_example<\/pre>\n
The above command copies the file testfile.txt from local filesystem to hdfs directory.<\/p>\n
hdfs dfs \u2013ls \/user\/copy_from_local_example<\/pre>\n
The above command is to check the creation of testfile.txt in hdfs directory \/user\/copy_from_local_example<\/p>\n
\"Hadoop<\/a><\/p>\n
Do you know about Hadoop Automatic Failover?<\/strong><\/a><\/p>\n
3. Overwriting the existing file in HDFS<\/h4>\n
copyFromLocal command does not by default overwrite the existing files. If we try to copy the file with the same name in the same directory then we will get an error. We can see it from the below screenshot.<\/p>\n
\"Hadoop<\/a><\/p>\n
We have to use \u2013f option of copyFromLocal file to overwrite the file.<\/p>\n
hdfs dfs \u2013copyFromLocal \u2013f testfile.txt \/user\/copy_from_local_example<\/pre>\n
The above command will replace the existing file. To check whether the file has been successfully copied we use ls command. From the below screenshot we can see that file is indeed copied from the timestamp of 22:25 as compared to timestamp of 22:22 when the file was first created.<\/p>\n
\"Hadoop<\/a><\/p>\n
Conclusion<\/h2>\n
Thus, copyFromLocal is one of the important commands of the Hadoop<\/a><\/strong> FS shell. We can use this command to load the input file of the MapReduce job<\/strong><\/a> from the local file system to HDFS. In this article, we have taken an example to understand copyFromLocal command. And how to go about overwriting the already existing file. Still, if you have any doubt, feel free to ask through comments.
Hadoop file system shell commands have a similar structure to Unix commands. People working with Unix shell command find it easy to adapt to Hadoop Shell commands. These commands interact with HDFS and other...<\/p>\n","protected":false},"author":6,"featured_media":53997,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[22],"tags":[5231,19491],"class_list":["post-53953","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hadoop","tag-hadoop-commands","tag-hadoop-copyfromlocal"],"yoast_head":"\n3 Easy Steps to Execute Hadoop copyFromLocal Command - DataFlair<\/title>\n<meta name=\"description\" content=\"What is Hadoop copyFromLocal and why to use it. Explore the steps to perform the copyFromLocal Command. Here you will see the Hadoop Shell Command\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"3 Easy Steps to Execute Hadoop copyFromLocal Command - DataFlair\" \/>\n<meta property=\"og:description\" content=\"What is Hadoop copyFromLocal and why to use it. Explore the steps to perform the copyFromLocal Command. Here you will see the Hadoop Shell Command\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2019-04-06T10:05:13+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-02-05T12:15:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2019\/04\/Hadoop-copyFromLocal.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"3 Easy Steps to Execute Hadoop copyFromLocal Command - DataFlair","description":"What is Hadoop copyFromLocal and why to use it. Explore the steps to perform the copyFromLocal Command. Here you will see the Hadoop Shell Command","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/","og_locale":"en_US","og_type":"article","og_title":"3 Easy Steps to Execute Hadoop copyFromLocal Command - DataFlair","og_description":"What is Hadoop copyFromLocal and why to use it. Explore the steps to perform the copyFromLocal Command. Here you will see the Hadoop Shell Command","og_url":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2019-04-06T10:05:13+00:00","article_modified_time":"2020-02-05T12:15:38+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2019\/04\/Hadoop-copyFromLocal.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89"},"headline":"3 Easy Steps to Execute Hadoop copyFromLocal Command","datePublished":"2019-04-06T10:05:13+00:00","dateModified":"2020-02-05T12:15:38+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/"},"wordCount":555,"commentCount":0,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2019\/04\/Hadoop-copyFromLocal.jpg","keywords":["hadoop commands","Hadoop copyFromLocal"],"articleSection":["Hadoop Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/","url":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/","name":"3 Easy Steps to Execute Hadoop copyFromLocal Command - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2019\/04\/Hadoop-copyFromLocal.jpg","datePublished":"2019-04-06T10:05:13+00:00","dateModified":"2020-02-05T12:15:38+00:00","description":"What is Hadoop copyFromLocal and why to use it. Explore the steps to perform the copyFromLocal Command. Here you will see the Hadoop Shell Command","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2019\/04\/Hadoop-copyFromLocal.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2019\/04\/Hadoop-copyFromLocal.jpg","width":1200,"height":628,"caption":"Hadoop copyFromLocal"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/hadoop-copyfromlocal-command\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Hadoop Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/hadoop\/"},{"@type":"ListItem","position":3,"name":"3 Easy Steps to Execute Hadoop copyFromLocal Command"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"The DataFlair Team provides industry-driven content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our expert educators focus on delivering value-packed, easy-to-follow resources for tech enthusiasts and professionals.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam2\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/53953","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=53953"}],"version-history":[{"count":5,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/53953\/revisions"}],"predecessor-version":[{"id":75497,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/53953\/revisions\/75497"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/53997"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=53953"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=53953"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=53953"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}