strata + hadoop world: jump into the data lake with hadoop-scale data integration

10
Jump into the Data Lake with Hadoop-Scale Data Integration Dr. Greg Benson Chief Scientist, SnapLogic Professor, University of San Francisco

Upload: snaplogic-inc

Post on 18-Jul-2015

566 views

Category:

Technology


0 download

TRANSCRIPT

Jump into the Data Lake with Hadoop-Scale Data Integration!

Dr. Greg BensonChief Scientist, SnapLogic

Professor, University of San Francisco

SnapLogic’s Vision: !Unified Integration Platform as a Service (iPaaS) !

The SnapLogic Designer !

Elastic Integration, Hadoop-Scale !

•  Cloud to Cloud•  Cloud to Ground!•  Groud to Groud!

•  Elastic: Scales in the cloud or on premise.

Metadata

Data

SnapLogic Key Technologies !•  SaaS model for Integration: iPaaS •  Modern HTML5-based user ���

interface•  No programming required•  Intelligent connectivity: Snaps•  High-performance pipeline ���

execution engine: Snaplex

•  Hybrid execution: ���cloud or ground•  Streaming and accumulating ���

(batch) support•  JSON native data processing•  Pipelines as APIs•  Integration automation

•  Hadooplex, SnapReduce, and SnapSpark

The Data Lake: !Replacing the EDW?!

Hadooplex: Snaplex YARN Application

= Snaplex Container

•  SnapLogic is a first-class citizen in Hadoop

•  Multiplex Hadoop Cluster for integration, data staging, and data prep.

•  Scale out Snaplex processes via Resource Manager

•  Kerberos Authentication

•  Certified by Cloudera and Hortonworks

SnapReduce: Pipelines Generate MapReduce

MAP MAP MAP MAP

REDUCE MAP MAP REDUCE

SnapReduceCompiler

Map Reduce

•  A checkbox option to SnapReduce-enable a pipeline

•  Support for SequenceFile, RCFile, document (JSON) processing for MapReduce jobs

YARN

SnapLogic, Hadoop, and the Data Lake !

•  Augment Hadoop ecosystem•  Open up Hadoop to more IT/Business professionals•  Automate data ingest into Hadoop•  Prepare data for Data Scientists and Analytics•  Generate MapReduce and Spark code for pipeline execution•  Deliver data to DBs, BI Tools, and Cloud Apps

Big Data Integration in a Snap!

@SnapLogic

Facebook.com/SnapLogic Plus.google.com/+SnapLogic

•  Helping customers adopt Hadoop

•  Automate your data integration workflows

Learn more at www.SnapLogic.com !!