enterprise data science - what it takes to build?
DESCRIPTION
Enterprise data science is not just creating dashboard, reports, ad-hoc query, models and/or algorithms, it’s beyond all - Take a look at our approach to enterprise data sciences, it’ very complex and it’s very difficult to implement as it’s involved integrating data across enterprise business function regardless of data source, format and structure There are many instances where people talk about enterprise data sciences (Oracle 12C, HADOOP, SAP) but “have you seen enterprise data sciences in a real system as a live demo”, in most cases the answers is “no” but now there is an opportunity to review enterprise data sciences with CloneSkills. I would say confidently say that there is no one in the world who integrated “Oracle 12C” and SAP HANA with HADOOP for real-time data integration except CloneSkills technical architect Mr. KarthikTRANSCRIPT
Enterprise data science learning solution
A practical approach to big data learning
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Objective
� Educate various key components that’s are typically used to deliver enterprise data sciences
� Demonstrate the steps to move data between Oracle 12C and HADOOP using Sqoop
� Review data flow between SAP HANA and HADOOP using smart data access
CloneSkills, Inc.(916)-296-0228
Our Enterprise Data Science Platform
HADOOP Distribution
SAP HANA Oracle 12C
Social | Forum | Blog | Web
File | Text
Analytics
What’s involved in building enterprise data science?
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
CloneSkills, Inc.(916)-296-0228
Our enterprise data science platform components - Our lab(CSLAB)
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
� SAP HANA
� SAP BOBJ
� Oracle 12C
� Oracle ODI
Enterprise Components
� HDFS
� HBase
� Hive
� Impala
� Pig
� Search
� Shell
� Mapreduce
� Sqoop
� OOIZE
� ZOOKEEPER
� Hue
� Dashboard
� Editor
HADOOP Components
CloneSkills, Inc.(916)-296-0228
Our (CSLAB) On demand Lab Infrastructure
__________________________________
� SAP HANA� SAP BOBJ� Oracle 12C� Oracle ODI� HADOOP
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Node 1
Node 2
Node 3
Node 4
Node 5
Node 6
Our enterprise data science platform technical components
CloneSkills, Inc.(916)-296-0228
Our three (3) node
HADOOP cluster
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Our enterprise data science platform - HADOOP infrastructure
CloneSkills, Inc.(916)-296-0228
Our HADOOP core
components
________________� Hive� Impala� Pig� Search� Hbase� Shell� Mapreduce� Sqoop� Hue� HDFS� OOIZE� ZOOKEEPER
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Our enterprise data science platform - HADOOP components
CloneSkills, Inc.(916)-296-0228
Our HADOOP core
components
________________
� Hive
� Impala
� Pig
� Search
� Hbase
� Shell
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Our enterprise data science platform - Hue components
CloneSkills, Inc.(916)-296-0228
Our Oracle 12 C
Infrastructure
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Our enterprise data science platform - Oracle
CloneSkills, Inc.(916)-296-0228
Our Oracle 12 C
Infrastructure
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Our enterprise data science platform - Oracle
CloneSkills, Inc.(916)-296-0228
Our Oracle ODI (
Oracle Data
Integrator)
Infrastructure
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Our enterprise data science platform - Oracle data integrator (ODI)
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
SAP HANA
_______________
Smart Data Access
Connects SAP HANA
and HADOOP
Our enterprise data science platform – SAP HANA
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
SAP HANA
_______________
Smart Data Access
Connects SAP HANA
and HADOOP
Our enterprise data science platform - SAP HANA and HADOOP integration
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
HADOOP Distribution
Oracle 12C Sqoop
Import
Export
Steps to move data between Oracle and HADOOP using Sqoop
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Oracle table and it’s
data
Review Oracle table – EMPLOYEE_JP
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Sqoop Job
Sqoop job creation
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Sqoop Job
____________
Create connection to
Oracle
Sqoop job creation - Create connection to Oracle
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Sqoop Job
____________
Oracle source table
details
Sqoop job creation - Configure source table
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Sqoop Job
____________
Oracle source table
and column details
Sqoop job creation - Configure source table and the primary key of the table
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Sqoop Job
____________
Destination in
HADOOP ( HDFS
output files)
Sqoop job creation - Configure data target , HDFS files (output files)
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Sqoop Job
____________
Job extraction log
Run Sqoop job - review job log
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Sqoop Job
____________
HDFS destination
files
Sqoop job output - HDFS output file, destination files
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Sqoop Job
____________
Oracle data in
HADOOP - preview
Sqoop job output - Oracle data in HADOOP HDFS files
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Sqoop Job
____________
Data has been
imported from Oracle
to HADOOP
Sqoop Job
____________
We can also export
data from HADOOP
and then load them
into Oracle
Sqoop job output - Data has been moved from Oracle to HADOOP
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Our enterprise data
sciences use case
CloneSkills, Inc.(916)-296-0228
Learn to lead big data - Enterprise data science a practical approach
CloneSkills, Inc.
http://www.CloneSkills.com
Architect : Karthik Rajamanickam
Stay tuned, more to come Thank You !
CloneSkills, Inc.(916)-296-0228