Demystifying Big Data and Visual Data Discovery

Advanced Analytics for Any Data at Real-Time SpeedDan PotterChief Marketing Officer, Datawatch Corporation

How Many Sensors Are In Your Pocket?

2

NASDAQ: DWCH Pioneer in real-time visual data discovery and self-service data preparationGlobal operations and support US, UK, Germany, France, Australia, Singapore, Philippines Extensive global customer base93 of the Fortune 10012 of the 15 largest financial institutionsEmbedded and resold by leading vendorsAbout Datawatch

IoT and Industrial Analytics50 billion devices will be interconnected by 2020 - CiscoBy 2017, over 50% of analytics implementations will make use of event data streams generated from instrumented machines, applications and/or individuals - GartnerReal-time visualization leads to more opportunities, greater output, and lower costs - Aberdeen GroupIoT companies attracted more than $1 billion in venture capital - Forbes

IoT companies attracted more than $1 billion in venture capital in 20134

What is Industrial Big Data?

IoT Platform Requirements

StreamProcessingVisual Data DiscoveryReal-Time Transport

SensorData

OTData

ITData

Visualization for IoT

Data AccessVisual DiscoveryMonitor and AlertsData PreparationPredictive AnalyticsTake Action

6 Requirements for IoT VisualizationData DiscoveryStreaming Data VisualizationTime Series Data Predictive & Advanced AnalyticsData PreparationReal-time Geospatial & Location234561

Time Series ConflationStreaming visualizationAlertsIntegration with CEP and event processingJSON file formatsMQTTScada and Pi Server OSIsoftPredictive R and Pytohn

8

#1 Data Discovery, not just Dashboards

Easy for users to author, customize and shareInteractive exploration & visually filter resultsQuickly identify anomalies and outliers with large or in-motion datasetsRich palette of visualizations for static and time series data

#2 Streaming Data Visualization

Database

Distributed or Hybrid Database

In-MemoryDatabase

Streaming AnalyticsFaster Speed, Faster Insights

Data at RestLimitations of Traditional BI

DatabaseDistributed or Hybrid DatabaseIn-MemoryDatabase

Streaming AnalyticsHow are we doing?

Data at RestStreaming Data Discovery

DatabaseDistributed or Hybrid DatabaseIn-MemoryDatabase

Streaming Analytics

Why?You have a problem!

Streaming Data Discovery

DatabaseDistributed or Hybrid DatabaseIn-MemoryDatabase

Streaming Data

Alert! Steam turbine stress level over threshold How does this compare to intra-day? What is mean time to failure?What is likely to occur?Act! Schedule shut down

Would you cross the street based on yesterdays news

13

Streaming Data VisualizationConnect directly to data in motion Hosted IoT platforms Complex Event Processing & MQData model optimized for both caching and persistence High density visuals with rendering in milliseconds

Monitor

Analyze

Take Action

An in-memory, OLAP-based StreamCube is associated with each graphical display object. The system processes new data as it arrives, selects the subset of important data, recalculates the relevant sections of the model and refreshes the associated parts of the display immediately. The parts of the model and the display that are not affected by the new data are not touched. This is faster and more efficient than conventional data visualization tools that operate on batch-loaded snapshots of data, run less frequently, and then recalculate the model and rebuild the display for each iteration.

14

#3 Time Series DataTraditional BI only looks at buckets of timeDay, week, month, yearSensor data is a continuous and has different requirementsSecond, millisecond, nanosecondTime windowsTime slicesPlaybackComplete situational awarenessNow (streaming)Intra-dayHistoric

#4 Predictive & Advanced AnalyticsEnrich streaming OT data with what is likely to occurPredictive models based on historic data patternsMany use cases in IoT (e.g. predictive maintenance, smart logistics, clinical pattern detection etc.)Leverage commercial and open source solutions

Can be used in combination with time series transforms

16

We Want to Predict the Future for Equipment

Modeled and transformed for analysis

#5 Data PreparationSensor and machine data often in multi-structured formatNeed to transform, enrich and prepare dataAlmost no metadataFor example, wave form visualization from JSON arrays stored in MongoDB and streaming

Log FilesHTML, XML

JSON

PDFs

#6 Real-Time Geospatial & LocationReal-time (stream) plotting Street-level geo maps or custom SVG filesTime-series playback

Healthcare

RetailLogisticsUtilities

6 Requirements for IoT VisualizationVisual Data DiscoveryStreaming Data VisualizationTime Series Data Predictive & Advanced AnalyticsData PreparationReal-time Geospatial & Location

234561

New Analytic Approach Required

Time Series ConflationStreaming visualizationAlertsIntegration with CEP and event processingJSON file formatsMQTTScada and Pi Server OSIsoftPredictive R and Pytohn

20

The Next Wave of Business Transformation

Source: Industrial Analytics: The Next Wave of Business TransformationGartner, March 2014

ROI for Real-Time Data Visualization

26% Growth in New Pipeline

15% Increase in Cash Generated

67% Greater Operational Cost Reduction

Real-Time Data Visualization October 2013

Source: Aberdeen Group, Real-Time Data Visualization, October 2013. 22

Customer Use CaseFortune 500 oil & gas exploration and production companyMoving to real-time streaming visualizationFrom 24 hour latency moving data overnight from OSIsoft Pi to Oracle Warehouse feeding dashboardsTo real-time, streaming data discovery connecting directly to OSIsoft Pi server Initial goal is to reduce steam cost by 3-5% ($M) in year 1Pi Server

Smart Meter DashboardStreaming data from electric smart meters

Hospital Emergency Ward Birds Eye View Streaming data from patient monitoring system

Workbook