sqoop installation guide lecturer : prof. kyungbaek kim presenter : zubair amjad

SQOOP INSTALLATION GUIDE

Lecturer : Prof. Kyungbaek KimPresenter : Zubair Amjad

Requirements• Hadoop• Sqoop• MySQL Server• JDBC driver

Environment• Ubuntu 12.04• Hadoop 1.0.3

• Sqoop 1.4.4

Hadoop Fully Distributed Mode

Sqoop Installation• Download the sqoop-1.4.4.bin_hadoop-1.0.0.tar.gz file

• ww.apache.org/dyn/closer.cgl/sqoop/1.4.4• Unzip the tar file

• sudo tar -zxvf sqoop-1.4.4.bin_hadoop-1.0.0.tar.gz• Move sqoop-1.4.4.bin_hadoop-1.0.0 to sqoop

• sudo mv sqoop 1.4.4.bin_hadoop-1.0.0 sqoop• Create a directory sqoop in usr/lib

• sudo mkdir /usr/lib/sqoop• Go to the folder sqoop and copy files to /usr/lib/sqoop

• sudo mv ./* /usr/lib/sqoop• Go to root directory and open bashrc file

• sudo gedit ~/.bashrc• Add SQOOP_HOME variable

• export SQOOP_HOME=/usr/lib/sqoop • export PATH=$PATH:$SQOOP_HOME/bin

• Check sqoop installation• sqoop version

MySQL Server• Install mysql server

• sudo apt-get install mysql-server• Give appropriate username and password

• Download mysql-connector-java-5.1.28-bin.jar and copy to /usr/lib/sqoop/lib• cp mysql-connnectpr-java-5.1.28-bin.jar /usr/lib/sqoop/lib/

• Login to mysql server• mysql -u root –p

Create Database• Create a database

• create database temp_db;• show databases;

• Add a table in the database• use temp_db;• CREATE TABLE employee (id INT, name VARCHAR(20), email

VARCHAR(20));• Insert some rows using INSERT INTO

• Grant privileges to root user• grant all privileges on *.* to root@IP IDENTIFIED BY ‘root’

WITH GRANT OPTION;

Create Database

Create Table

Insert rows in table

Grant Permissions

Sqoop Import• Import the table from temp_db to HDFS

• sqoop import -connect jdbc:mysql://localhost:3306/temp_db -username root -P -table employee –target-dir /sqoopOut –m 1

Table Import

HDFS

Sqoop Export• Export data from HDFS to database table

• Create a table in data base• CREATE TABLE stud1 (id INT, name VARCHAR(20));

• Export data in file stud to database• sqoop export --connect jdbc:mysql://192.168.34.130/temp_db --

username root -P --table stud1 -m 1 --export-dir /sqoopexport/stud

Export

Exported Table

Thank you