Sqoop Installation – Steps to Install Sqoop 1.4.5 on Linux
Keeping you updated with latest technology trends, Join DataFlair on Telegram
1. Sqoop Installation – Objective
In this Sqoop Tutorial, we study “Sqoop Installation” explains all the steps to install Sqoop on Linux. Since we know Sqoop is Hadoop’s sub-project. Thus, it can only work on a Linux operating system. Hence, at first, we will see the installation of some prerequisites to run Sqoop 1.4.5 Installation. So, for that, we will start with steps to verify Java Installation. At last, we will see steps to install Hadoop. Then we will learn the steps for Sqoop Installation on your Linux system.
So, let’s start Sqoop installation.
2. Java Installation
Before Sqoop installation – version 1.4.5 on Linux, at very first we need to have Java installed on our system.
In order to install Java on our system, we need to follow various steps given below.
By visiting the following link, download Java (JDK <latest version> – X64.tar.gz).
Hence, jdk-7u71-linux-x64.tar.gz will be downloaded on our system.
Basically, we find the downloaded Java file in the downloads folder. So, by using the following commands verify it and extract the jdk-7u71-linux-x64.gz file.
$ cd Downloads/
$ tar zxf jdk-7u71-linux-x64.gz
Then, we have to move it to the location “/usr/local/”, to make Java available to all the users. So, open root, and type the following commands.
# mv jdk1.7.0_71 /usr/local/java
# exitStep IV:
Afterward, add the following commands to ~/.bashrc file in order to set up PATH and JAVA_HOME variables.
Now apply all the changes to the current running system.
$ source ~/.bashrc
Now to configure Java alternatives use the following commands
# alternatives –install /usr/bin/java java usr/local/java/bin/java 2
# alternatives –install /usr/bin/javac javac usr/local/java/bin/javac 2
# alternatives –install /usr/bin/jar jar usr/local/java/bin/jar 2
# alternatives –set java usr/local/java/bin/java
# alternatives –set javac usr/local/java/bin/javac
# alternatives –set jar usr/local/java/bin/jar
So, by using the following command, let’s verify Java installation.
$ java –version
However, we get to see the following response, if Java is already installed on your system
java version “1.7.0_71”
Java(TM) SE Runtime Environment (build 1.7.0_71-b13)
Java HotSpot(TM) Client VM (build 25.0-b02, mixed mode)
If these professionals can make a switch to Big Data, so can you:
Java → Big Data Consultant, JDA
PeopleSoft → Big Data Architect, Hexaware
3. Hadoop Installation
It is very important that before Sqoop installation, Hadoop is installed on our system. Follow these steps if Hadoop is not installed on your system:
a. Download Hadoop
By using the following commands, download and extract Hadoop 2.4.1 from Apache Software Foundation.
# cd /usr/local
# wget http://apache.claz.org/hadoop/common/hadoop-2.4.1/
# tar xzf hadoop-2.4.1.tar.gz
# mv hadoop-2.4.1/* to hadoop/
So, let’s verify the Hadoop installation by using following commands
$ hadoop version
We will get the following response if Hadoop is already installed on your system
Subversion https://svn.apache.org/repos/asf/hadoop/common -r 1529768
Compiled by hortonmu on 2013-10-07T06:28Z
Compiled with protoc 2.5.0
From source with checksum 79e53ce7994d1628b240f09af91e1af4
Hence, in this way, we can download Hadoop.
4. Sqoop Download – Sqoop Installation
Basically, we can download the latest version of Sqoop from here: Download sqoop
For this Sqoop Installation tutorial, we are using version 1.4.5, that is,
Step 1- Sqoop Installation
However, to extract the Sqoop tarball and move it to “/usr/lib/sqoop” directory we use the following command.
$tar -xvf sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz
# mv sqoop-1.4.4.bin__hadoop-2.0.4-alpha /usr/lib/sqoop
Step 2- Configuring bashrc
Also, by appending the following lines to ~/.bashrc file we have to set up the Sqoop environment
export SQOOP_HOME=/usr/lib/sqoop export PATH=$PATH:$SQOOP_HOME/bin
Now, to execute ~/.bashrc file we use the following command.
$ source ~/.bashrc
Step 3: Configuring Sqoop
While, we need to edit the sqoop-env.sh file, that is placed in the $SQOOP_HOME/conf directory, in order to configure Sqoop with Hadoop. Now, using the following command redirect to Sqoop config directory and copy the template file
$ cd $SQOOP_HOME/conf
$ mv sqoop-env-template.sh sqoop-env.sh
Also, open sqoop-env.sh and edit the following lines
Step 4: Download and Configure MySQL-connector-java
From the following link, we can download MySQL-connector-java-5.1.30.tar.gz file.
In adddition, to extract MySQL-connector-java tarball and move MySQL-connector-java-5.1.30-bin.jar to /usr/lib/sqoop/lib directory we use the following command.
$ tar -zxf mysql-connector-java-5.1.30.tar.gz
# cd mysql-connector-java-5.1.30
# mv mysql-connector-java-5.1.30-bin.jar /usr/lib/sqoop/lib
Step 5: Verifying Sqoop
At last, to verify the Sqoop version we use the following command.
$ cd $SQOOP_HOME/bin
14/12/17 14:52:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
Sqoop 1.4.5 git commit id 5b34accaca7de251fc91161733f906af2eddbe83
Compiled by abe on Fri Aug 1 11:19:26 PDT 2014
Hence, in this way Sqoop installation is complete.
Hence, in this Sqoop installation tutorial, we study how Sqoop download & install Sqoop by verifying Java Installation. At last, how to Download Hadoop and Verifying Hadoop Installation. Furthermore, if you feel any query regarding Sqoop installation, feel free to ask in the comment section.
See Also- Sqoop Eval & Sqoop Codegen