Big Data OverviewSTEPHEN S IMPSON

SHARPLYUNCLEAR

V1.1 February 2013

What is Big Data?

3

Major evolution of BI• Builds on technological innovations

from consumer Internet• Data arrives in large volumes at

unpredictable times, in variety of forms, and from different sources

• Different sources somehow related, but complex associations

• May require granular real-time analysis & follow-up actions

• Data structures specified at runtime • Scales linearly – start small• Commodity hardware & open

source software• Economical & practical to keep vast

historical volumes on-line

Big Data IT Business

Big storageBig analytics

Big InsightsBig Returns(?)

Consumer social data

Multiple Core enterprise systems

M2M sensor data

Operational data (for digital forensics)

E-mail & ECM

External data sources

Where Big Data countsFinancial Services

Customer Risk Analysis Surveillance & Fraud Detection Central Data Repository Personalisation & Asset Management Market Risk Modelling Trade Performance Analytics

Retail, Manufacturing & Telco Customer Churn Analysis Brand and Sentiment Analysis POS Transaction Analysis Pricing Models Customer Loyalty Targeted Offers Manufacturing Plant monitoring Sensor analytics & insight

Government & Energy Security Digital Forensics Utilities and Power Grid Smart Meters Biodiversity Indexing Medical Research & Seismic Data Tax evasion

Web & e-Commerce Online Media Mobile Online Gaming Search Quality Recommendations Influence Preventing Network Failures

5

Transparency◦ Information available across all organization’s businesses

Segmenting Populations◦ Social network influencers◦ Client micro-segmentation

Supporting human decisions with algorithms◦ Best-next-product recommendations◦ Optimising preventative maintenance downtime

Enabling experimentation◦ Champion/challenger hypothesis testing

New business models, products & services◦ Core processes “listening” to outside world◦ Make Mobile real-time context-based offers

Out manoeuvring & disrupting the competition◦ Operating at a faster tempo to generate rapidly changing conditions

Big Data creates business value

6

The most successful businesses…

…Collect & analyse information better. Act upon that information faster. Incorporate learning into next iteration

Hindsight Oversight Insight Foresight

Business Intelligence Operational Intelligence

7

Operational intelligence requires agility

Acting quickly offers major competitive advantage

Long cycle times are wasteful:◦ Don’t know what is important

or accurate until test hypothesis◦ Reduces collective ability of

organisation to learn

Speed in learning amplifies effectiveness of everything else

◦ Use agile techniques to reduce scope of each cycle

◦ Use results of cycle to decide what to do next

What’s happening in the market?

The industry watchers say

9

$5B market. 40% CAGR IDC

$50B market by 201758% CAGR wikibon.org

5% market is technologies. The majority is services Forbes

What do clients need?

10

Clients need guidance & new skills

Leadership◦ New capabilities to ask right

questions◦ Provide clear business goals◦ Define success for company

Talent Management◦ Business linkage: what learnt,

and how to use it?◦ To handle large data sets◦ Statistics

12

Technology◦ Unstructured data modelling◦ Hadoop (Big Data storage engine)◦ Analytics & Visualisation

Decision Making◦ Authority & responsibility◦ Co-located with information

collection point

Culture◦ Hunches : what do we think? ◦ What do we know?◦ Weaving into fabric of daily operations

12

Critical success factors Data sources

◦ Creatively sourcing internal & external data◦ Upgrading IT architecture & infrastructure

◦ To ensure easy & fast ingestion, storage and retrieval of data

Prediction & Optimisation Models◦ Focusing on biggest drivers of performance◦ Building or acquiring models that balance complexity with ease of use

Organisational Transformation◦ Simple, understandable tools for people on front lines◦ Updating processes & developing business capabilities to use tools effectively◦ Linking Mobile, Cloud, Big Data and Social

13

What do clients want? Proof of Concepts

◦ Agile: succeed (or fail) quickly and cheaply◦ Successful ones will evolve into much larger programs

Services against the “Big Data skills gap”◦ Provision of skills identified earlier in this presentation

Trusted Advisors◦ Route map: clear & honest balancing of cost, value & limitations◦ Identification and advice on best technology providers

Value propositions◦ With credible references◦ Configurable by appropriately trained internal staff

14

Asset IntelligenceLoad factor, Faults,

Conditions

Situational IntelligenceVoltage & Quality, Post

Fault Demand Side Mgmt., Outages

Forecasting IntelligenceAsset Failure Prediction

Smart interventionsCapacity Operations

Planning

Possible incremental approach in UtilitiesFurther Big Data opportunities across Business divisions:

Supply Chain Management

Revenue Assurance

Personalised Pricing

Consumer mobile

Supply BusinessGeneration Business

AggregatorsDemand Side

Response Operator

2013

2013/4

2016

2018

2020

AssetIntelligence

SituationalIntelligence

ForecastingIntelligence

Proof of Concept

Internal dataset enhancement to

network, premises & integration

Smart Grid Monitoring

Smart Grid & Meter Data

value extraction

Smart Data correlation to external cloud weather/social

media data

Incremental approach can be taken to all verticals