big trends in big data

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 1

Big Trends in Big DataTimo ElliottVP, Global Innovation Evangelist


Analytics Takes Over The World…


88%

How Do Executives Make Decisions?

Aspect Consulting, 1997

12%Hard Facts

Gut Feel

90%

10%Hard Facts

Gut Feel

Economist Intelligence Unit, 2014

Why the worst-practice shaded 3D donut charts? JUST TO ANNOY DATA VISUALIZATION EXPERTS!


Plus Ça Change…

Petabytes

DataScientists

+ IoT

Big Data


Big Data On The Gartner Hype Cycle

Source: Gartner, August 2014

Innovation trigger

Peak of Inflated Expectations

Trough of Disillusionment

Slope of Enlightenment

Plateau of Productivity

Big Data


Benefits from Big Data Initiatives

#5 Identified new product opportunities (6%)

#4 More reliable decision making (9%)

#3 Improved operational efficiency (11%)

#2 Identified new business opportunities (31%)

#1 “DON’T KNOW” (51%)

Source: Information Difference Research Study Dec 2013: “Big Data Revealed”


Use AnalyticsToday

NeedAnalytics by 2020

Gartner, 2014

The Opportunity

Inability to see, understand, and optimize new opportunities

Inaccessible dataand technology

Insights remain hidden

Complexity, cost, confusionSilos of approaches and

analytic technologies

75%

10%

Slow decision making lacking future view

Rear view mirrorBI mentality


Predicting Prescriptive Analytics

Descriptive:What happened?

Diagnostic:Why did it happen?

Predictive:What will happen?

Prescriptive:How can we make it happen?

Taking Analytics To The Next Level

Hindsight Insight Foresight

DATA SCIENCE

QUIZ.

These numbers were found in two tax declarations. One is entirely made up. Which one?

EUR

127,-2.863,-

10.983,-694,-

29.309,-32,-

843,-119.846,-18.744,-

1.946,-275,-

EUR

937,-82.654,-18.465,-

725,-98.832,-

7.363,-4.538,-

38,-8.327,-

482,-2.945,-

1 2 3 4 5 6 7 8 9

30.1%

17.6%

12.5%

9.7%7.9%

6.7%5.8% 5.1% 4.6%

Benford’s LawDistribution of the first digit of real-world sets of numbers that uniformly span several orders of magnitude

DATA SCIENCE

QUIZ.

These numbers were found in two tax declarations. One is entirely made up. Which one?

EUR

127,-2.863,-

10.983,-694,-

29.309,-32,-

843,-119.846,-18.744,-

1.946,-275,-

EUR

937,-82.654,-18.465,-

725,-98.832,-

7.363,-4.538,-

38,-8.327,-

482,-2.945,-

Benford's Law, also called the First-Digit Law

13© 2015 SAP SE or an SAP affiliate company. All rights reserved.

1999 to 2009

“Greece shows the largest deviation from Benford’s law with respect to all measures. [And] the suspicion of manipulating data has officially been confirmed by the European Commission.”

Fact and Fiction in EU-GovernmentalEconomic Data, 2011


Transport For London


Big Data Discovery


Big Data Discovery =

Big Data

Data Discovery

Data Science

Gartner Strategic Planning Assumption: By 2017, Big Data Discovery Will Evolve Into a Distinct Market Category


Big Data Discovery

• Volume, velocity, or variety of data

• Potential business impact

• Difficult to implement• Potentially expensive• Lack of skills available

• Ease of use• Agility and flexibility• Time-to-results• Installed user base

• Complexity of analysis

• Potential impact• Range of tools• Smart algorithms• Difficult to implement• Slow and complex• Narrow focus of

analysis

• Limited depth of information exploration

• Low complexity of analysis

BIGDATA

DATASCIENCE

DATADISCOVERY


Big Data Discovery

• Simpler to use than data science

• Accessible to a wider range of users

• Broad range of data manipulation features

• Able to handle new types of data sources

• With adequate performance for big data

BIG DATA

DISCOVERY


Potential impact per user

Potential user base

The Rise of the Citizen Data Scientist?

Business analyst

Data scientist

Citizen data scientist


Citizen Data Scientists


The Opportunity

New Business Opportunities

Traditional Analytics

Data Value

Volume / Variety / Velocity of Data

“Big Data Discovery”

Data Discovery

Big Data

Data Science


SAP’s Opportunity

Big Data

Discovery

SAP HANA (+ Hadoop etc.)

SAP Predictive Analytics 2.0

SAP Lumira


The Dashboards of Yesterday And

Tomorrow


“Intricate calculations of sales by territories will appear as if by magic in the digital age ahead”


Decision Cockpits

Boardroom Redefined

Source:In-Memory Data Management: An Inflection Point for Enterprise Applications. Hasso Plattner Alexander Zeier


The New Multi-Polar World of Analytics


In-Memory Maturity

Data from 245M customers/week, 11,000 stores under 71 banners in 27 countries and e-commerce websites in 11 countries with $482.2 Bn sales and 2.2M employees.• Full in-memory system• 250 Bn rows of data• 94% of queries run < 2s• >1,000 concurrent users even

under heavy loads • Data load throughput >20 million

records/hour

Suja ChandrasekaranCTO of Walmart Technology

e.g. Wal-Mart’s Data Café (“Collaborative Analytics Facilities for Enterprise”)


The End of the Hadoop Honeymoon?

“Despite considerable hype and reported successes for early adopters, only 18% have plans to invest in Hadoop over the next two years…. in fact, there are fewer who plan to begin in the next two years than already have.”

Nick Heudecker, research director at Gartner.

SAP HANA Platform

Bringing Enterprise Data to Hadoop and Hadoop Data to The Enterprise

SPATIAL PROCESSING

ANALYTICS, TEXT, GRAPH, PREDICTIVE

ENGINES

CONSUME

COMPUTE

STORAGE

SOURCE

INGEST

Application Development Environment

Transformations & Cleansing

Smart Data IntegrationSmart Data Quality

StreamProcessing

Smart Data Streaming

STREAM PROCESSING

LogsTextOLTP Social Machine GeoERP SensorStore & forward

Mobile applications and BI

Smart Data Access

Virtual Tables

User Defined Functions

101010010101101001110

Dynamic Tiering

Aged datain Disk

In-Memory

Data model& data

Calculation engine

Fastcomputing

Column Storage

High performance analytics

Series Data Storage

Store time-series data

Reporting &Dashboards

High Performance Applications

Data Exploration& Visualization

Adhoc & OLAP Analytics

PredictiveAnalysis

Business Planning & Forecasting Lumira / BI

But there is more work to do…

Hadoop / NoSQL

MapReduce

YARN

HDFS


The New Multi-Polar World of Big Data Architectures

Data Warehouse

Third-Party Data Feeds

Hadoop,MongoDB,Spark, etc Personal

Data / BI

Where does data arrive?When does it need to move?Where does modeling happen?What can users do themselves?What governance is required?

Big Data Architectures got complicated

What we want — consistent, seamless solution

Hybrid Transaction/

Analytical Processing


Apache Atlas


Culture: Suits AND Hoodies

Source: Gartner


Conclusion

Dashboards of Tomorrow

Multi-Polar Analytics

Prescriptive Analytics

Big Data Discovery


“Judge a man by his questions rather than his answers.”Voltaire

“Status Quo is, you know, latin for “the mess we’re in”Ronald Reagan

“Any intelligent fool can make things bigger and more complex. It takes a touch of genius and a lot of courage to move in the opposite direction.”

E.F. Schumacher

Thank You!Timo ElliottVP, Global innovation Evangelist

[email protected] @timoelliott