big trends in big data

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 1

Big Trends in Big DataTimo ElliottVP, Global Innovation Evangelist

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 2

Analytics Takes Over The World…

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 3

88%

How Do Executives Make Decisions?

Aspect Consulting, 1997

12%Hard Facts

Gut Feel

90%

10%Hard Facts

Gut Feel

Economist Intelligence Unit, 2014

Why the worst-practice shaded 3D donut charts? JUST TO ANNOY DATA VISUALIZATION EXPERTS!

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 4

Plus Ça Change…

Petabytes

DataScientists

+ IoT

Big Data

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 5

Big Data On The Gartner Hype Cycle

Source: Gartner, August 2014

Innovation trigger

Peak of Inflated Expectations

Trough of Disillusionment

Slope of Enlightenment

Plateau of Productivity

Big Data

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 6

Benefits from Big Data Initiatives

#5 Identified new product opportunities (6%)

#4 More reliable decision making (9%)

#3 Improved operational efficiency (11%)

#2 Identified new business opportunities (31%)

#1 “DON’T KNOW” (51%)

Source: Information Difference Research Study Dec 2013: “Big Data Revealed”

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 7

Use AnalyticsToday

NeedAnalytics by 2020

Gartner, 2014

The Opportunity

Inability to see, understand, and optimize new opportunities

Inaccessible dataand technology

Insights remain hidden

Complexity, cost, confusionSilos of approaches and

analytic technologies

75%

10%

Slow decision making lacking future view

Rear view mirrorBI mentality

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 8

Predicting Prescriptive Analytics

Descriptive:What happened?

Diagnostic:Why did it happen?

Predictive:What will happen?

Prescriptive:How can we make it happen?

Taking Analytics To The Next Level

Hindsight Insight Foresight

DATA SCIENCE

QUIZ.

These numbers were found in two tax declarations. One is entirely made up. Which one?

EUR

127,-2.863,-

10.983,-694,-

29.309,-32,-

843,-119.846,-18.744,-

1.946,-275,-

EUR

937,-82.654,-18.465,-

725,-98.832,-

7.363,-4.538,-

38,-8.327,-

482,-2.945,-

1 2 3 4 5 6 7 8 9

30.1%

17.6%

12.5%

9.7%7.9%

6.7%5.8% 5.1% 4.6%

Benford’s LawDistribution of the first digit of real-world sets of numbers that uniformly span several orders of magnitude

DATA SCIENCE

QUIZ.

These numbers were found in two tax declarations. One is entirely made up. Which one?

EUR

127,-2.863,-

10.983,-694,-

29.309,-32,-

843,-119.846,-18.744,-

1.946,-275,-

EUR

937,-82.654,-18.465,-

725,-98.832,-

7.363,-4.538,-

38,-8.327,-

482,-2.945,-

Benford's Law, also called the First-Digit Law

13© 2015 SAP SE or an SAP affiliate company. All rights reserved.

1999 to 2009

“Greece shows the largest deviation from Benford’s law with respect to all measures. [And] the suspicion of manipulating data has officially been confirmed by the European Commission.”

Fact and Fiction in EU-GovernmentalEconomic Data, 2011

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 14

Transport For London

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 15

Big Data Discovery

16© 2015 SAP SE or an SAP affiliate company. All rights reserved.

Big Data Discovery =

Big Data

Data Discovery

Data Science

Gartner Strategic Planning Assumption: By 2017, Big Data Discovery Will Evolve Into a Distinct Market Category

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 17

Big Data Discovery

• Volume, velocity, or variety of data

• Potential business impact

• Difficult to implement• Potentially expensive• Lack of skills available

• Ease of use• Agility and flexibility• Time-to-results• Installed user base

• Complexity of analysis

• Potential impact• Range of tools• Smart algorithms• Difficult to implement• Slow and complex• Narrow focus of

analysis

• Limited depth of information exploration

• Low complexity of analysis

BIGDATA

DATASCIENCE

DATADISCOVERY

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 18

Big Data Discovery

• Simpler to use than data science

• Accessible to a wider range of users

• Broad range of data manipulation features

• Able to handle new types of data sources

• With adequate performance for big data

BIG DATA

DISCOVERY

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 19

Potential impact per user

Potential user base

The Rise of the Citizen Data Scientist?

Business analyst

Data scientist

Citizen data scientist

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 20

Citizen Data Scientists

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 21

The Opportunity

New Business Opportunities

Traditional Analytics

Data Value

Volume / Variety / Velocity of Data

“Big Data Discovery”

Data Discovery

Big Data

Data Science

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 22

SAP’s Opportunity

Big Data

Discovery

SAP HANA (+ Hadoop etc.)

SAP Predictive Analytics 2.0

SAP Lumira

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 23

The Dashboards of Yesterday And

Tomorrow

24© 2015 SAP SE or an SAP affiliate company. All rights reserved.

“Intricate calculations of sales by territories will appear as if by magic in the digital age ahead”

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 25

Decision Cockpits

Boardroom Redefined

Source:In-Memory Data Management: An Inflection Point for Enterprise Applications. Hasso Plattner Alexander Zeier

27© 2015 SAP SE or an SAP affiliate company. All rights reserved.

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 28

The New Multi-Polar World of Analytics

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 29

In-Memory Maturity

Data from 245M customers/week, 11,000 stores under 71 banners in 27 countries and e-commerce websites in 11 countries with $482.2 Bn sales and 2.2M employees.• Full in-memory system• 250 Bn rows of data• 94% of queries run < 2s• >1,000 concurrent users even

under heavy loads • Data load throughput >20 million

records/hour

Suja ChandrasekaranCTO of Walmart Technology

e.g. Wal-Mart’s Data Café (“Collaborative Analytics Facilities for Enterprise”)

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 30

The End of the Hadoop Honeymoon?

“Despite considerable hype and reported successes for early adopters, only 18% have plans to invest in Hadoop over the next two years…. in fact, there are fewer who plan to begin in the next two years than already have.”

Nick Heudecker, research director at Gartner.

SAP HANA Platform

Bringing Enterprise Data to Hadoop and Hadoop Data to The Enterprise

SPATIAL PROCESSING

ANALYTICS, TEXT, GRAPH, PREDICTIVE

ENGINES

CONSUME

COMPUTE

STORAGE

SOURCE

INGEST

Application Development Environment

Transformations & Cleansing

Smart Data IntegrationSmart Data Quality

StreamProcessing

Smart Data Streaming

STREAM PROCESSING

LogsTextOLTP Social Machine GeoERP SensorStore & forward

Mobile applications and BI

Smart Data Access

Virtual Tables

User Defined Functions

101010010101101001110

Dynamic Tiering

Aged datain Disk

In-Memory

Data model& data

Calculation engine

Fastcomputing

Column Storage

High performance analytics

Series Data Storage

Store time-series data

Reporting &Dashboards

High Performance Applications

Data Exploration& Visualization

Adhoc & OLAP Analytics

PredictiveAnalysis

Business Planning & Forecasting Lumira / BI

But there is more work to do…

Hadoop / NoSQL

MapReduce

YARN

HDFS

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 32

The New Multi-Polar World of Big Data Architectures

Data Warehouse

Third-Party Data Feeds

Hadoop,MongoDB,Spark, etc Personal

Data / BI

Where does data arrive?When does it need to move?Where does modeling happen?What can users do themselves?What governance is required?

Big Data Architectures got complicated

What we want — consistent, seamless solution

Hybrid Transaction/

Analytical Processing

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 33

Apache Atlas

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 34

Culture: Suits AND Hoodies

Source: Gartner

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 35

Conclusion

Dashboards of Tomorrow

Multi-Polar Analytics

Prescriptive Analytics

Big Data Discovery

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 36

“Judge a man by his questions rather than his answers.”Voltaire

“Status Quo is, you know, latin for “the mess we’re in”Ronald Reagan

“Any intelligent fool can make things bigger and more complex. It takes a touch of genius and a lot of courage to move in the opposite direction.”

E.F. Schumacher

Thank You!Timo ElliottVP, Global innovation Evangelist

Timo.Elliott@sap.com @timoelliott