internet of things: where fast data meets big data: couchbase connect 2015
TRANSCRIPT
INTERNET OF THINGS:WHERE FAST DATA MEETS BIG DATA
Shane Johnson, Couchbase
©2015 Couchbase Inc. 2
Internet of Things
Agenda
The Introduction The Numbers The Challenge More Challenges
The Convergence
The Solution
The Introduction
©2015 Couchbase Inc. 4
Internet of Things
Landscape
IoT
Mobile
M2MWearabl
e
©2015 Couchbase Inc. 5
Internet of Things
Industry / Sector
Automotive Consumer Electronics Energy Fitness Fleet Management Healthcare Home Automation
Industrial Manufacturing Smart Appliances Supply Chain
Management Transportation Utilities
©2015 Couchbase Inc. 6
Internet of Things
The “Things”
Small or Big Medical Implant Railroad
Consumer or Industrial
Television Wind Turbine
Fixed or Mobile Refrigerator Car
Online or Offline Thermostat Offshore Oil Rig
The Numbers
©2015 Couchbase Inc. 8
Internet of Things
Connected “Things”
2015
15 Billion
2020
50 Billion
©2015 Couchbase Inc. 9
Internet of Things
Data Generated
2013
.35ZB(72 Million TB)
2020
4.4ZB(320 Million TB)
The Challenges
©2015 Couchbase Inc. 11
Internet of Things
Internet of Things == Big Data
©2015 Couchbase Inc. 12
Internet of Things
Velocity – Faster Data
Concurrent Connections
Hundreds to Thousands Thousands to Millions
Streaming Data Hundreds per Second
Example: Sensor Thousands per Second
Example: Wind Turbine Millions per Second
Example: Wind Farm
©2015 Couchbase Inc. 13
Internet of Things
Volume – Bigger Data
Gigabytes
TerabytesPetabytes
©2015 Couchbase Inc. 14
Internet of Things
Variety – Different Data
Heterogeneous Refrigerator Thermostat Door Lock Motion Sensor Baby Monitor
Evolving New Hardware New Firmware New Software
More Challenges
©2015 Couchbase Inc. 16
Internet of Things
Global Economy = Global Data Multiple Data Centers
Single Regions Multiple Regions
Cloud Infrastructure Multiple Regions Multiple Availability
Zones
Hybrid Cloud One (DC) to Many
(Cloud) Replicate Out
Many (Cloud) to One (DC)
Replicate In
©2015 Couchbase Inc. 17
Internet of Things
Never Ending Data
Storage Not Infinite Tiered
Current Historical
Network Availability Not Guaranteed
Unresponsive Interrupted Down
©2015 Couchbase Inc. 18
Internet of Things
Bidirectional Data
Machines Generate Actions Readings Events
Machines Consume Commands Configuration Events
The Convergence
©2015 Couchbase Inc. 20
Internet of Things
The Pinnacle of Convergence?
The Solution
©2015 Couchbase Inc. 22
The Internet of Things
Architecture
Kafka(Messaging)
Couchbase Server
Spark Core / SQL
(Analysis)
©2015 Couchbase Inc. 23
The Internet of Things
Architecture
CouchbaseServer
Kakfa(Messaging)
Spark Streaming(Analysis)
©2015 Couchbase Inc. 24
The Internet of Things
Architecture
Kafka(Messaging)
Spark Streaming(Analysis)
CouchbaseServer
Hadoop(Data Lake)
©2015 Couchbase Inc. 25
The Internet of Things
CouchbaseMobile
CouchbaseSync Gateway
CouchbaseServer
Architecture
Hadoop / Spark
Connected Device
©2015 Couchbase Inc. 26
The Internet of Things
MobileDevice
MobileDevice
Sync Gateway
MobileDevice
MobileDevice
Sync Gateway
Couchbase
Server
Couchbase
Server
Remote Site
Remote Site
Cloud Region
Cloud Region
Couchbase
Server
Local DC
©2015 Couchbase Inc. 27
The Internet of Things
MobileDevice
MobileDevice
Sync Gateway
MobileDevice
MobileDevice
Sync Gateway
Couchbase
Server
Couchbase
Server
Cloud Region
Cloud Region
Couchbase
Server
Local DC
©2015 Couchbase Inc. 28
The Internet of Things
Summary
Challenges Concurrent Connections Streaming Data GBs to TBs to PBs of
Data Different Data (Evolving) Global Data Never Ending Data Directional Data
Solutions Couchbase Server and
Messaging (Kafka) Stream Processing
(Storm) Hadoop
Couchbase Mobile Embedded Database Automatic
Synchronization Online / Offline
Thank you.