the future of advance analytics

4/1/2013

1

The Future of Advance Analytics

David SmithChief Executive Officer, HBMG [email protected]:InnoTech Main Stage Theater, inside Hall Bg ,

Volume, variety, and velocity are changing us in our companies, government agencies and at home. How do BI, Social/Business media, mobility, Devices and Big Data drive business decisions? The success is driven by the use of advance analytics. Business analytics facilitates realization of business objectives through reporting of data to analyze trends, creating predictive models for forecasting and optimizing business processes for enhanced performance. This is important not only in business but the military and government as well.

This presentation will look at the current and future trends in analytics and how they will impact each of us. Special emphasis will be given to the new trend of embedded analytics. As the world moves faster toward real-time the analytics must move as well. Attendees will leave the session with a understanding of the future directions of advance analytics.

The Future of Advance AnalyticsAnalytics

David SmithCEO [email protected]

4/1/2013

2

Definition for Advanced Analytics

Analysis is the examination process itself where analytics is the supporting technology and associated tools. BI is quite synonymous to analytics in IT context. Advanced Analytics, Business Analytics, Data Analytics, Analytics Software, Analytics Technology are almost always marketing pleonasms (redundantalways marketing pleonasms (redundant expressions) and can be safely substituted by just ‘analytics’


Analysis is a pretty old, well understood term d ti ll “b ki d ”and essentially means “breaking down” or

“decomposition”. More accurately –the process of decomposing complex entity into simpler components for easier comprehension.

4/1/2013

3


Advanced analytics provides algorithms for complex analysis of either structured or unstructured data. It includes sophisticated statistical models, machine learning, neural networks, text analytics, and other advanced data mining techniques. Among its many use cases, it can be deployed to find patterns in data, prediction, optimization, forecasting, and for complex event processing/analysis. Examples include predicting churn, identifying fraud, market basket analysis, or understanding website behavior Advanced analytics does notunderstanding website behavior. Advanced analytics does not include database query and reporting and OLAP cubes.

America gets more than half its economic growth from industries that barely existed

a decade ago—such is the power of innovation, especially in the information

and biotechnology industries.

—The Economist

Copyright, 2011 © HBMG, Inc.

4/1/2013

4

Business Problem

More than half of business and IT executives, 56 percent, report they feel overwhelmed by the amount of data their company manages. Many report they are often delayed in making important decisions as a result of too much information. Surprisingly, 62 percent of C‐level respondents – whose time is considered the respondents whose time is considered themost valuable in most organizations – report being frequently interrupted by irrelevant incoming data.

4/1/2013

5

Delivering business value is hard…

• “Of the work executed: “Many (possibly most) organizations lose as much as 45% of their total revenues due to costs associated with low quality”associated with low quality– Six Sigma

• “Some 75 percent of most large‐scale J2EE projects fail by missing both time and budget projections …”– Mark Driver, Gartner

• “64% of features actually delivered are• 64% of features actually delivered are either rarely or never used” – Jim Johnson, Standish Group

Why?

• Technological innovation is now the most important driver for competitive success

– Many firms earn over one‐third of sales on products developed within last five years

• Product life cycles ( time between product introduction to market and its withdrawal)


– Software 4‐12 months

– Computer hardware 12‐24 months

– Large home appliances 18‐36 months

4/1/2013

6

Business, Knowledge, and Innovation Landscape

• Typically 80% of the key knowledge (and value) is held by 20% of the people – we need to get it to theheld by 20% of the people we need to get it to the right people

• Only 20% of the knowledge in an organization is typically used (the rest being undiscovered or under‐utilized)

80 90% f th d t d i t d ill b• 80‐90% of the products and services today will be obsolete in 10 years – companies need to innovate & invent faster

Copyright 2012@ HBMG Inc.

Tapping into the Data

• Data Storage• Reporting• Analytics

Utilized data• Analytics• Advanced

Analytics

– Computing with big datasets is a

Unutilized data that can be

il bl tg

fundamentally different challenge than doing “big compute” over a small dataset

available to business

4/1/2013

7

Innovation:‘The real voyage of discovery consists

not in seeing new lands, but in seeing with new eyes’

(Source: Marcel Proust)

‘Innovation = creative idea and implementation’

(Source: Glossary of Electronics)

‘Innovation: change that

‘A new method, idea, product, etc’

(Source: Oxford English Dictionary)

creates a new dimension of performance’

(Source: Peter Drucker)

‘An innovation to be effective has to be simple and it has

to be focused’(Source: Peter Drucker)

‘Value innovators look for what customers value in

common’(Source: Kim & Mauborgne)

‘Innovation is the process by which new products or methods of production are introduced, including all the steps from the inventor’s idea to bringing the

new item to market’(Source: Baumol, Economics: Principles & Policy)

(Source: Peter Drucker)

‘Firms need to manage steady state innovation and radical change because continuous improvement is no longer

enough’(Source: Tom Peters)

“Big Data” and it’s close relatives “Cloud Computing”, “Social Media” and

"Mobile"

are the new frontier of innovation.

Driven by Advance Analytics

4/1/2013

8

Big Data and It’s Brothers

VolumeVolume

Variety

Velocity

………..

Volume

Volume is increasing at incredible rates. h l h h dWith more people using high speed

internet connections than ever, plus these people becoming more proficient at creating content and just more people in general contributing information are combined forces thatinformation are combined forces that are causing this tremendous increase in Volume.

4/1/2013

9

Variety

Next in breaking down Big Data into easily digestible bite‐size chunks is the concept of Variety. Take your personal experience and think about how much information youexperience and think about how much information you create and contribute in your daily routine. Your voicemails, your e‐mails, your file shares, your TV viewing habits, your Facebook updates, your LinkedIn activity, your credit card transactions, etc.

Whether you consciously think about it or not the VarietyWhether you consciously think about it or not the Variety of information you personally create on a daily basis which is being collected and analyzed is simply overwhelming.

Velocity

The speed at which data enters organizations these days is absolutely amazing. With mega internet bandwidth nearly being common place anymore in conjunction with the proliferation of mobile devices, this simply gives people more opportunity than ever to contribute content to storage systems. contribute content to storage systems.

4/1/2013

10

Worldwide digital content will double in 18 months, and every 18 months thereafter.

VELOCITY

IDC

CRM Data

GP

S

Demand

Sp

ee

d

VelocityO

ppor

tuni

ties

Customer

Inventory

Em

ails

Tw

eets

Planning

Mobile

Instant MessagesVOLUME VARIETY

Transactions

Se

rvice C

alls

Sales Orders

ThingsIn 2005, humankind created 150 exabytes of information. In 2011, over 1,200 exabytes was created.

VOLUME80% of enterprise data will be unstructured, spanning traditional and non traditional sources.

Gartner

The Economist

But I Believe there are Four V4

4/1/2013

11

• Volume：Gigabyte(109), Terabyte(1012), Petabyte(1015), Exabyte(1018), Zettabytes(1021)

• Variety: Structured,semi‐structured, unstructured; Text, image, audio, video, record

• Velocity(Dynamic, sometimes time‐varying)

• BUT needs to add and create Value!• BUT needs to add and create Value!

Trends driving data management

– The volume of data has never been greater and is growing exponentially

– The value of data has never been better understood– The value of data has never been better understood

– The capabilities for processing data have never been better

• Higher processor performance and density are enabling advanced processing on commodity hardware

• Software enhancements designed to make best use of i f d l bl hit tprocessing performance and scalable architecture

• Advanced and in‐database analytics bring processing to the data, reducing latency and improving efficiency

– The data deluge problem is also a big data opportunity

22

4/1/2013

12

Copyright, 2011 © HBMG, Inc.From http://geekandpoke.typepad.com

Next Generation

• Cloud Computing (e.g. “Blue Cloud”)

– The “network becomes the computer”

D b d i ??????– Dumb devices ??????

• Pervasive Computing– Monitoring and tracking almost anything

– The Internet is Free

• Continuous Services

• The Cloud + Pervasive Computing– Smart buildingsSmart buildings

– Sensor nets

– “Invisible computing” or “ubiquitous computing”

– Semantic Interoperability

– Ad hoc networking


4/1/2013

13

Advance Analytics as a strategic asset

“The future belongs to companies and people that turn data into products.”

Mike Loukides, O’Reilly

25


“85% of eBay’s analytic workload is new and unknown. We are architected for the unknown.”

Oliver Ratzesberger, eBay

• Data exploration – data as the new oil The exploration for data, rather than the exploration of data

Uncovering pockets of untapped data

Processing the whole data set, without sampling

eBay’s Singularity platform combines transactional data with behavioral data, enabled identification of top sellers, driving increased revenue from those sellers

26

4/1/2013

14


“Groupon will not be the first or last organization to compete and win on the power of data It’s happeningcompete and win on the power of data. It s happening everywhere.”

Reid Hoffman and James SlavetGreylock Partners

Data harnessing – data as renewable energy

H i t ll i d t t Harnessing naturally occurring data streams

Like harnessing raw energy to be converted into usable energy

Conversion of raw data into usable data

27

Facebook

28

4/1/2013

15

BIG DATAREAL TIMEPREDICTIVEENABLED BYENABLED BYADVANCE ANALYTICS

As the world gets smarter, infrastructure demands will grow

Smart traffic

systems

Smart energy grids

Smart healthcare

Smart food

systems

Intelligent oil field

technologies Smart retail

Smart water management

Smart regions

Smart weather

Smart countries

Smart supply chains

Smart cities

4/1/2013

16

Data

Desktop

Knowledge

4/1/2013

17


The Nature of Communications Has Been Changing...

People toThings

People toPeople

Things toThings

People toThings

People toPeople

Things toThings

TOLL

TOLL

TOLL

TOLL

ThingsPeople ThingsThingsPeople Things


15.62 Gallons27.33 DollarsThank You!



4/1/2013

18

Mainframe/Terminal

Client Server

Ubiquitous

COSTS

Mainframe

Mini/Terminal

Server/PC

ASP/Appliance

USAGE

Distribution of Computing Power Virtualization Ubiquitous

Improved Access/Convenience

Improved Ease of Use

Large Organizations + Small Organizations + Departments + BYOD Entrepreneurs

Middle Class (PCs) + General Population

Digital Video Adapters

Satellite Radio Receivers

Digital CamerasPDAs

Wireless CamerasWireless TV Monitors

Digital Music Adapters

Networked Storage Centers

Game Consoles

Smart Displays

Smart Phones

Laptop PCs

Desktop PCs

Wireless Gaming Adapters

Movies-on-DemandReceivers

“Fourth Generation”Set-top Boxes

MP3 PlayersDigital Media Receivers Personal Video Recorders

Networked DVD Player Mobile Gaming Devices

802.11 Speakers


4/1/2013

19

The Global Grid


To Every Sensor is a Server

Phone -PDA

Processor

Data Storage

Communications

Rich variety of sensors

Smart Dust

Microphone

Microstorage

(Areal density 100x’s CD)


Embedded Biofluidics Chip

Robot

4/1/2013

20

3,500

4,000

• Mobile

Growth at the Edge of the Network

1,000

1,500

2,000

2,500

3,000

Pet

abyt

es/D

ay G

loba

lMobile

• Device to Device • Sensors • Entertainment• Smart Home• Distributed Industrial• Autos/Trucks• Smart Toys

ConvergedContent

0

500

2003 2004 2005 2006 2007 2008 2009 2010 2011

Year

2012

Traditional Computation


DOD ExampleDOD Example

4/1/2013

21

Counterinsurgency operations are complex increased emphasis on: Information and analysis at lowest levels

Shortened decision making time-scales

Operation Trends

Source: TTI Vanguard Conference - Psydex

g

Wider array of information sources

Continued growth in volume of data, especially informal information with limited structure must transform disparate info to knowledge

Processing power and storage capacity increasing faster than communications capacity must smartly position data and services within networks

Increased use of commercial cellular networks hybrid networks Increased use of commercial cellular networks hybrid networks that exploit and interoperate with commercial wireless comms is key

Enhancing coalition decision making depends on secure communications and information networks must address end-to-end problem of data-to-decision (coalition)

“Recce” P-38 “Recce” P-38

RF-101 Voodoo

Info in War Revolution

Technology—Information—Organization

B-17 Spotter Corps

0-2 Bird Dog

0-2 Bird DogB-52

75 Days75 Days

14 Days14 Days

<10 Minutes<10 Minutes

N b f W N b f S

RSI

RI

S ISR?

42

Evolution of Technology, Information, and Culture Enabled Move from Evolution of Technology, Information, and Culture Enabled Move from Segregation of Ops and Intel to Integration of Ops and Intel…Segregation of Ops and Intel to Integration of Ops and Intel…

Number of WeaponsRequired to Target

Number of SensorsRequired to Target

19431943 20092009

4/1/2013

22

National

StrategicStrategicStrategic

OperationalOperationalOperational

AIR

AIR

SP

AC

SP

AC

CY

BE

CY

BE

StrategicStrategic

OperationalOperational

StrategicStrategic


21st Century Challenges:Precision and Information Synergy

Tactical


TacticalTacticalTactical

DESERTSTORM

1991 & Prior

1999ALLIEDFORCE

2001ENDURING FREEDOM

2009AF/PAK & Iraq

TODAY

INFO AGEINFO AGEWARFAREWARFARE

TOMORROWTOMORROWRRC

EC

E

ER

ER

TacticalTactical

pp

TacticalTactical

Ops …+ Intel

I + S&R GLOBAL INTEGRATED

ISRSegregatedIntel & Ops

NTI

+ Intel INTEGRATED ISR

Intel & Ops

Kandahar Runway

Pod RecceReal‐TimeFusion

OPS

INTEL

Multi‐DomainFusion

More CollectorsMore CollectorsBetter SensorsBetter Sensors

Dimensions of ISR…“More of Everything”

More DataMore Data

•• More StorageMore Storage•• More CommsMore Comms•• More ToolsMore Tools•• More AnalystsMore Analysts•• More LinguistsMore Linguists

Better Intel

…All on an Operationally Responsive Timeline…All on an Operationally Responsive Timeline

4/1/2013

23

Sensor Data Volume

How do we handle all this data?How do we handle all this data?“Rebalancing Collection & PED may be Necessary”“Rebalancing Collection & PED may be Necessary”

Persistent Surveillance Data Rates

4/1/2013

24

General Norton A. Schwartz

Air Force’s chief of staff

“If i id f l h“If automation can provide a cue for our people that would make better use of their time, that would help us significantly,” NY Times.

Lieutenant General David Deptula

Former Deputy Chief of Staff for ISR, Headquarters, US Air p y f f ff f qForce

“We’re going to find ourselves in the not too distant future swimming in sensors and drowning in data”

Advance Analytics

• Advanced Analytics and Big Data are two of the most active areas of innovation in the Tech sector

• legacy infrastructures and government policies are increasingly at odds with the realities of the analytic landscape

• Certain forms of analysis is no longer possible within an encrypted environment. Rules that require data to be encrypted, both while in transit and at rest, also introduce performance penalties that make it difficult if not impossible to process large datasets in an acceptable timeframe

4/1/2013

25

Today's Cycle

Where is Real Time?

Advance Analytics

• The time to use the output is increasingly getting shorter – Real Time is becoming very common

Li it d il bl h d f i• Limited available human resources, and performance is often unreliable due to human fatigue and distraction. Therefore, automated real‐time sensor processing techniques are required to reliably detect and discriminate targets of interest

• Limited automated processing and tagging tools

• – Still NOT enough

4/1/2013

26

Advance Analytics

• The time to use the output is increasingly getting shorter – Real Time is becoming very common

• Limited available human resources, and performance is often punreliable due to human fatigue and distraction. Therefore, automated real‐time sensor processing techniques are required to reliably detect and discriminate targets of interest – Still NOT enough

• Need to move to thesensor/collector

• Needs to be embedded in the the sensor

Autonomous Systems

Agents coordinate and negotiate to achieve

common goals

Agents dynamically adaptto and learn abouttheir environment

Adaptive

Autonomous Mobile Interoperate

CooperativeSocial

PersonalityIntelligentAgents

InformationAgents

HBMG Inc. Copyright 2012

Agents are goal directedand act on theirown performing

tasks on your behalf

Agents moveto where they

are needed

Agents interoperatewith humans, other, legacy systems, and information sources

4/1/2013

27

Autonomic Networks

Self-configuring : Adapt automatically to the dynamically changing

Self-healing:Discover, diagnose, and react todynamically changing

environments of link and node failures.

Self-optimizing: Monitor and tune resources automatically during an attack to minimize its attack during and in the

and react todisruptions from catastrophes and attacks.

Self-protecting:Anticipate, detect, identify, and protect against attacks from anywhere (safety )

SelfSelf--HealingHealing

SelfSelf--ProtectingProtecting

SelfSelf--OptimizingOptimizing

SelfSelf--ConfiguringConfiguring

HBMG Inc. Copyright 2012

attack during and in the aftermath.

anywhere (safety.)

Numbers

• How many data in the world?

– 800 Terabytes, 2000

– 160 Exabytes, 2006

– 500 Exabytes(Internet), 2009

– 2.7 Zettabytes, 2012

– 35 Zettabytes by 2020

• How many data generated ONE day?

– 7 TB, Twitter

– 10 TB, Facebook

Big data: The next frontier for innovation, competition, and productivity

McKinsey Global Institute 2011

4/1/2013

28

1 illi1million transactions during this presentation

If You Liked ____, You’ll Love ___ !

4/1/2013

29

1billi1 billion clicks during this presentation

4/1/2013

30

Gartner Hype Cycle 2012

4/1/2013

31

2012 Business Intelligence, Analytics and Information Management Survey from InformationWeek Reports

A few insights from the report:

•58% of those surveyed are “very interested” in advanced analyticsy y y•Advanced analytics is the No. 1 leading-edge technology•Cloud analytics systems are hot because they are easier on the pocketbook; yet 63% of users have privacy concerns•Data pros just can’t get good data – data quality still ranks as the top barrier to adopting BI products throughout the company•25% of those surveyed are mobilizing their data analytics with dashboards and data visualizations40% f d t t li t t b th bi d t•40% of data pros are struggling to stay above the big data wave

Conclusion

Data is one the major factors driving infrastructure computing

The growing volume of data is a problem, but it is also an opportunity

Don’t worry about ‘big data,’ worry about your datay g , y y

Take a Total Data approach to data management

• Think pragmatically about data storage and analysis

• Attempt to capture and analyze any data that might be relevant, regardless of where it resides

‘Datastructure’ will become increasingly valuable, not only as a source of data but also as a source of intelligence

The rise of the ‘data cloud’ and the PaaS data layer will encourage a more flexible approach to data management and analytics

The companies that win will be those that think about data as a strategic asset and implement the technology to monetize it

4/1/2013

32

Conclusion

The World is moving to Real Time

Advanced Analytics is the Keyy y

Advanced Analytics Must be embedded in the collectors and sensors

• Think about where the data comes from

• Attempt to capture and analyze any data that might be relevant, regardless of where it resides

• Realize collaboration is the key in Advance Analytics just as it is in Business

If we don’t change ourIf we don t change our direction, we’ll end up exactly

where we are headed.

—Ancient Chinese Proverb

4/1/2013

33

In Parting: Be Paranoid

•“Sooner or later, something fundamental in your business world

•“Sooner or later, something fundamental in your business worldfundamental in your business world

will change.”

• Andrew S. Grove, Founder, Intel“Only the Paranoid Survive”

fundamental in your business world will change.”

• Andrew S. Grove, Founder, Intel“Only the Paranoid Survive”Only the Paranoid SurviveOnly the Paranoid Survive

Copyright @2008 HBMG Inc.