Demystifying Big Data and Visual Data Discovery

You vs the sensorsSix Requirements for Visualizing the Internet of ThingsDan PotterChief Marketing Officer, Datawatch Corporation

How Many Sensors Are In Your Pocket?

2

Workbook

NASDAQ: DWCH Pioneer in real-time visual data discovery and self-service data preparationGlobal operations and support US, UK, Germany, France, Australia, Singapore, Philippines Extensive global customer base93 of the Fortune 10012 of the 15 largest financial institutionsEmbedded and resold by leading vendorsAbout Datawatch

Datawatch is a new MQ entrant that offers data discovery capabilities for a range of structured, streaming and multi-structured data, both emerging requirements. This makes them particularly well-suited for analytic applications for emerging requirements for the Internet of Things (IoT).

Customers also have an above-average perception of their overall customer experience, particularly in the areas of support and product quality.

New Gartner Magic Quadrant

5

IoT: Industrial & ConsumerIndustrialConsumer

Misconception #1: The IIoT is the same as the consumer Internet of Things (IoT), except its located on a factory floor somewhere.The industrial IoT has far more stringent requirements than the consumer IoT, including the need for no-compromise control, rock-solid security, unfailing reliability even in harsh (extremely hot or cold, dusty, humid, noisy, inconvenient) environments, and the ability to operate with little or no human intervention. Consumer products are built to last years (if that long); industrial products are built to last decades. If a Fitbit or Nest fails, a consumer suffers discomfort or inconvenience; if a train-braking system fails, lives are at stake.Misconception #2:The IIoT is about devices that push and pull status and command information from the networked world.The consumer IoT is geared to interaction with humans. The industrial IoT strives to strip humans out of the equation. Think of operations that must happen too quickly or too reliably, or too frequently, or from too harsh or remote an environment to make it practical to push and pull status and command information to or from any kind of centralized anything, be it an Internet server or the cloud.Misconception #3:The problem is that industrial device owners arent interested in, or actively resist, connecting smart devices together.Unlike most consumer IoT scenarios, which involve digital devices that already have IP support built in or that can be IP enabled easily, typical IIoT scenarios involve pre-IP legacy devices. And unfortunately, IP enablement isnt free. Industrial device owners need a direct economic benefit to justify IP enabling their non-IP devices. Alternatively, they need a way to gain the benefits of IP without giving up their investments in their existing industrial devices that is, without stranding these valuable industrial assets.Bottom line:In other words, yes, the consumer IoTismore innovative, but thats largely because the life cycle of consumer products is far shorter, the products have a higher tolerance for failure, and the products can be far more easily and cheaply disposed and replaced. Strategies that work for the consumer space may not work well in the industrial space.

6

IoT and Industrial Analytics50 billion devices will be interconnected by 2020 - CiscoBy 2017, over 50% of analytics implementations will make use of event data streams generated from instrumented machines, applications and/or individuals - GartnerReal-time visualization leads to more opportunities, greater output, and lower costs - Aberdeen GroupIoT companies attracted more than $1 billion in venture capital in 2013 - Forbes

IoT companies attracted more than $1 billion in venture capital in 20137

TestbedsInnovation to drive new products, processes, servicesTechnology & SecurityArchitectural frameworks, interoperability, privacy & security of Big DataThought LeadershipCommunity to advance innovation, best practices and insights

The goal of the IIC is to improve integration of the physical and digital worlds, to help drive adoption of Industrial Internet applications.

SPEAKER NOTES:Deliver best practices, reference architectures, case studies, and standards requirements to ease deployment of connected technologies;The IIC will identify requirements for open interoperability standards and define common architectures to connect smart devices, machines, people, processes and data.Coordinate the development of common architectures and platforms in order to reduce duplication of effort;Identification and location of sensor devices, Data exchange between sensor devicesControl and integration of collections of heterogeneous devices;Data extraction and storage and Data and predictive analytics Utilize existing and create new industry use cases and test beds for real-world applications;

Influence the global standards development process for Internet and industrial systems; Facilitate open forums to share and exchange real-world ideas, practices, lessons, and insights;Build confidence around new and innovative approaches to security.

8

Savings and growth opportunities across every industry$30Bfuel cost saving in aviation industry

$66Bfuel cost saving in gas powered fleets$63Bproductivityimprovement in healthcare$90Breduction in Cap X in oil & gas exploration and development $27Bproductivityimprovement in rail industry

1% savings from efficient Industrial Internet solutions could save billions in operational costs

Savings and Growth Opportunities Across Every Industry

1% savings from efficient Industrial Internet solutions could save billions in operational costs

9

What is Industrial Big Data?

IoT Platform Requirements

StreamProcessingVisual Data DiscoveryReal-Time Transport

SensorData

OTData

ITData

Visualization for IoT

Data AccessVisual DiscoveryMonitor and AlertsData PreparationPredictive AnalyticsTake Action

6 Requirements for IoT VisualizationData DiscoveryStreaming Data VisualizationTime Series Data Predictive & Advanced AnalyticsComplex File FormatsReal-time Geospatial & Location234561

Time Series ConflationStreaming visualizationAlertsIntegration with CEP and event processingJSON file formatsMQTTScada and Pi Server OSIsoftPredictive R and Pytohn

13

#1 Data Discovery, not just Dashboards

Easy for users to author, customize and shareInteractive exploration & visually filter resultsQuickly identify anomalies and outliers with large or in-motion datasetsRich palette of visualizations for static and time series data

#2 Streaming Data Visualization

Database

Distributed or Hybrid Database

In-MemoryDatabase

Streaming AnalyticsFaster Speed, Faster Insights

Data at RestLimitations of Traditional BI

DatabaseDistributed or Hybrid DatabaseIn-MemoryDatabase

Streaming AnalyticsHow are we doing?

Data at RestStreaming Data Discovery

DatabaseDistributed or Hybrid DatabaseIn-MemoryDatabase

Streaming Analytics

Why?You have a problem!

Streaming Data Discovery

DatabaseDistributed or Hybrid DatabaseIn-MemoryDatabase

Streaming Data

Alert! Steam turbine stress level over threshold How does this compare to intra-day? What is mean time to failure?What is likely to occur?Act! Schedule shut down

Would you cross the street based on yesterdays news

18

Streaming Data VisualizationConnect directly to data in motion Hosted IoT platforms Complex Event Processing & MQData model optimized for both caching and persistence High density visuals with rendering in milliseconds

Monitor

Analyze

Take Action

An in-memory, OLAP-based StreamCube is associated with each graphical display object. The system processes new data as it arrives, selects the subset of important data, recalculates the relevant sections of the model and refreshes the associated parts of the display immediately. The parts of the model and the display that are not affected by the new data are not touched. This is faster and more efficient than conventional data visualization tools that operate on batch-loaded snapshots of data, run less frequently, and then recalculate the model and rebuild the display for each iteration.

19

#3 Time Series DataTraditional BI only looks at buckets of timeDay, week, month, yearSensor data is a continuous and has different requirementsSecond, millisecond, nanosecondTime windowsTime slicesPlaybackComplete situational awarenessNow (streaming)Intra-dayHistoric

#4 Predictive & Advanced AnalyticsEnrich streaming OT data with what is likely to occurPredictive models based on historic data patternsMany use cases in IoT (e.g. predictive maintenance, smart logistics, clinical pattern detection etc.)Leverage commercial and open source solutions

Can be used in combination with time series transforms

21

We Want to Predict the Future for Equipment

Modeled and transformed for analysis

#5 Complex File FormatsSensor and machine data often in multi-structured formatNeed to transform, enrich and prepare dataAlmost no metadataFor example, wave form visualization from JSON arrays stored in MongoDB and streaming

Log FilesHTML, XML

JSON

PDFs

#6 Real-Time Geospatial & LocationReal-time (stream) plotting Street-level geo maps or custom SVG filesTime-series playback

Healthcare

RetailLogisticsUtilities

6 Requirements for IoT VisualizationVisual Data DiscoveryStreaming Data VisualizationTime Series Data Predictive & Advanced AnalyticsComplex File FormatsReal-time Geospatial & Location

234561

New Analytic Approach Required

Time Series ConflationStreaming visualizationAlertsIntegration with CEP and event processingJSON file formatsMQTTScada and Pi Server OSIsoftPredictive R and Pytohn

25

The Next Wave of Business Transformation

Source: Industrial Analytics: The Next Wave of Business TransformationGartner, March 2014

Capital MarketsTransaction Cost AnalysisAnalyze market data at ultra-low latenciesCompliance monitoringRisk and Fraud preventionDetecting multi-party fraudReal time fraud preventionTransportationIntelligent traffic managementAutomotive Telematics

ManufacturingPreventative maintenanceEnergy management

TelcoCDR processingSocial analysisChurn predictionGeomapping

Streaming Analytic ExamplesDefense & Cyber SecurityMultimodal surveillanceSituational awarenessCyber security detectionHealth & Life SciencesICU monitoringEpidemic early warningRemote healthcare monitoringEnergy & UtilitiesProduction surveillanceSmart meter analyticsDown hole sensor monitoring

27Who uses InfoSphere Streams? All industries have a need for InfoSphere Streams. Send clients to the website: http://www-01.ibm.com/software/data/infosphere/stream-computing/industry.html

Government / Law EnforcementAs the world around us gets more instrumented and interconnected, it also continues to become a flatter, smarter and a more dangerous place than ever before. Governments are constantly challenged by the forces of rapid urbanization, increasing strain on natural resources, citizen security as well as evolving threats related to global terrorism. InfoSphere Streams can help governments worldwide to ensure the economic health, welfare and security of their citizens via a plethora of applications such as cyber security, battlefield intelligence, traffic management, natural resource management, crime prevention, etc.

TelecommunicationsThe telecommunications industry has experienced more change in the last decade than in its entire history. Fueled by recent rapid adoption in developing countries, mobile communications have propped up the industry's top line. But now with these markets saturating, the telecom industry faces some serious questions: Where will future growth come from? How will the industry evolve? InfoSphere Streams can help telecommunications companies not just save on their IT spending and predict customer churn but can also uncover newer sources of revenue in a very effective way.

HealthcareIt is no secret that healthcare worldwide is in crisis - high costs, poor or inconsistent quality, and inaccessibility are potentially catastrophic. For example, healthcare in Ontario accounts for 50% of government spending. And in China, 39% of the rural population and 36% of the urban population cannot afford professional medical treatment. While there is no limit to the amount of data continuously being generated in provider organizations, what they lack is a way to analyze and correlate these different types of data in real time - which in many cases could be the difference between life and death. IBM InfoSphere Streams can help healthcare providers in a variety of ways ranging from complex real-time analytics on physiological streams of data in ICU environments.

Energy and UtilitiesTraditional business models for the electric utilities industry are losing relevance. Newer imperatives for the business models include governmental policies related to environmental impact, effect of newer technologies such as smart grid and smart meters and emerging relationships between customers and utility companies which involve more active and real time dialogue. InfoSphere Streams can help utility companies manage their businesses in a more profitable and efficient manner given the major paradigm shifts occurring across the industry.

Financial MarketsFinancial firms are under unrelenting pressure to respond to exploding increases in electronic trading volumes. High performance and low latency are the imperatives. Additionally, regulatory and governing bodies are enforcing newer and incrementally complex regulations on the industry. InfoSphere Streams can address a variety of innovative and emerging challenges in the financial markets industry ranging from market data handling and high frequency trading to risk control and surveillance resulting in higher revenues and lower costs.

Many demos are on IMAZ in the Streams Playbook: http://tinyurl.com/InfoSphereStreamsPlaybook

Additional are on a GSA site: ftp://pokgsa.ibm.com/gsa/pokgsa/home/r/r/rrea12/web/public/

If you install that in the c: drive on your machine, and this presentation page there, the hotlinks will launch these demos/videos

Customer Use CaseFortune 500 oil & gas exploration and production companyMoving to real-time streaming visualizationFrom 24 hour latency moving data overnight from OSIsoft Pi to Oracle Warehouse feeding dashboardsTo real-time, streaming data discovery connecting directly to OSIsoft Pi server Initial goal is to reduce steam cost by 3-5% ($M) in year 1Pi Server

Customer Use CaseSteam Injector Significantly enhances steam injection surveillance and management processIdentify optimization opportunities and troubleshoot problem areas

Steam GeneratorContinuously monitor the health of generators, optimize steam use, improve generator efficiency and identify operational issuesReal-time output qualities, rates and pressures can be monitored for each steam generator and in aggregate

Steam Injector Report:A steam Injector injects steam into reservoir to stimulate production. This Daily Steam Injection Exception report greatly enhances steam injection surveillance and management process. Users perform daily surveillance, look for optimization opportunities and allows operators to troubleshoot problem areas.

Steam Generator Report:A steam generator generates steam that will be used for our injection wells. This report will allow Facilities engineering to check the health of their generators. Provides data necessary for optimizing steam use, improving generator efficiency could result in annual fuel savings. Daily output qualities, rates and pressures can be monitored for each steam generator. This data provides a more accurate means of accessing generator thermal efficiencies and identifying persistent operational issues.

Group Meter Trends:A group meter is a meter that sums up all wells. Using this information we can start calculating real-time production deviation by comparing current projected value throughout the day to an average of Inferred Production Yesterday over a period of time. This would allow operations to make change in the field before they become an issue.

Please do not use this plot as it contains CRC data.

Field Balance:A way to measure the total amount of fluid being injected into reservoir compared to oil and water production of our wells. This will show the mass balance for each field. 29

Hospital Emergency Ward Birds Eye View Streaming data from patient monitoring system

Smart Meter DashboardStreaming data from electric smart meters