guest lecture on big data in business,
TRANSCRIPT
![Page 1: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/1.jpg)
1© Copyright 2013 Pivotal. All rights reserved. 1.
@krishdpi
The Foundation for ChangeBig Data in Business
10/10/2016SJSU
![Page 2: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/2.jpg)
2© Copyright 2013 Pivotal. All rights reserved. 2.
@krishdpi
[email protected]@krishdpihttp://www.linkedin.com/in/kriss
SK(Saravana Krishnamurthy)
![Page 3: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/3.jpg)
3© Copyright 2013 Pivotal. All rights reserved. 3
@krishdpi
What?
Why?
How?(Use Cases)
Next?
![Page 4: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/4.jpg)
4© Copyright 2013 Pivotal. All rights reserved. 4
@krishdpi
What?
Why?
How?(Use Cases)
Next?
![Page 5: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/5.jpg)
5© Copyright 2013 Pivotal. All rights reserved. 5.
@krishdpi
What is “Big Data”
“Big Data” refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. This definition is intentionally subjective and incorporates a moving definition of how big a dataset needs to be in order to be considered big data—i.e., we don’t define big data in terms of being larger than a certain number of terabytes (thousands of gigabytes).
-McKinsey Global Institute, May 2011
![Page 6: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/6.jpg)
6© Copyright 2013 Pivotal. All rights reserved. 6.
@krishdpi
!!!
!!!
!!!
!!!
!!!“Big Data Is Less About Size, And More About Freedom”
―Techcrunch
!!!
!!!“Findings: ‘Big Data’ Is More Extreme Than Volume”
― Gartner “Big Data! It’s Real, It’s Real-time, and It’s Already Changing Your World” ―IDC
“Total data: ‘bigger’ than big data” ― 451 Group
THE ERA OF
BIG DATA
IS HERE
![Page 7: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/7.jpg)
7© Copyright 2013 Pivotal. All rights reserved. 7.
@krishdpi
Data VolumeGrowing 44x
2020: 35.2 Zettabytes
2010:1.2
Zettabytes
The Digital Universe 2010 - 2020
Source: IDC Digital Universe Study, sponsored by EMC, May 2010
Terabyte < Petabyte < Exabyte < Zettabyte < Yottabyte < Xenottabyte < Shilentnobyte < Domegemegrottebyte
![Page 8: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/8.jpg)
8© Copyright 2013 Pivotal. All rights reserved. 8.
@krishdpi
Growth of Data• 2015: Youtube users were uploading 400 hours worth of video every minute works out to be 1PB raw capacity every day
– 2 TB = 500HDD– 500 * 365 = 182,000HDD/Year– $18m /Year in HDD alone.
• Number of Internet-connected devices will reach 33bn by 2020 – Strategy Analytics
• By 2015, Mobile data traffic is predicted to be 75 Exabytes annually – Cisco• Healthcare (as of 2011) is calculated at 150 Exabytes – SAS
![Page 9: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/9.jpg)
9© Copyright 2013 Pivotal. All rights reserved. 9
@krishdpi
What?
Why?
How?(Use Cases)
Next?
![Page 10: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/10.jpg)
10© Copyright 2013 Pivotal. All rights reserved. 10
@krishdpi
![Page 11: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/11.jpg)
11© Copyright 2013 Pivotal. All rights reserved. 11
@krishdpi
![Page 12: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/12.jpg)
12© Copyright 2013 Pivotal. All rights reserved. 12.
@krishdpi
Economics Have Changed the Game
![Page 13: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/13.jpg)
13© Copyright 2013 Pivotal. All rights reserved. 13.
@krishdpi
Big Data Analytics: Big Data Analytics: The Path to The Path to
Business ValueBusiness Value
IN THE BIG DATA ERA: ANALYTICS ARE THE KEY TO SUCCESS
![Page 14: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/14.jpg)
14© Copyright 2013 Pivotal. All rights reserved. 14
@krishdpi
15 Exa Bytes(15,000,000,000,000,000,000 bytes)
![Page 15: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/15.jpg)
15© Copyright 2013 Pivotal. All rights reserved. 15.
@krishdpi
John Reese: “I never understood why people put all their information on those sites. Used to make our job a lot easier at the CIA.”
Harold Finch: “Of course. That's why I created them.”
John Reese: “You're telling me you invented online social networking, Finch?”
Harold Finch: “The Machine needed more information. People's social graph, their associations. The government had been trying to figure it out for years. Turns out most people were happy to volunteer it. Business wound up being quite profitable, too.”
![Page 16: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/16.jpg)
16© Copyright 2013 Pivotal. All rights reserved. 16.
@krishdpi
Analytics TermsAnalytics
The practice of applying aggregations, statistics and models to large datasets to solve problems in business and industry
Business intelligenceAnother term for analytics, but often used to refer specifically to reporting, OLAP and other descriptive statistics
Machine learningAlgorithms that allow computers to learn behaviors from data(Ex. Facebook PYMK, Spam Filters, Speech/Face recognition, self-driving cars)
![Page 17: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/17.jpg)
17© Copyright 2013 Pivotal. All rights reserved. 17
@krishdpi
Transactions(OLTP)
Analytics(OLAP)
Learning(AI/DL)
Order an under cabinet kitchen water filtering system(Jan 1, 2016)
Dear Jon Doe, The water carbon filter needs to be replacedevery 6 months. Please click here if you would like us to send one tomorrow.(July 1, 2016)
A drone maps out optimal route using geospatial data, weather data, other traffic in the air, location data to deliver the filter at your door steps. (July 2, 2016)
![Page 18: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/18.jpg)
18© Copyright 2013 Pivotal. All rights reserved. 18.
@krishdpi
Analytics Evolution Desired by CustomerHIGH
FutureLOW Past Time
BUSINESS VALUE
ThenBusiness Intelligence(Descriptive)
NowPredictive Analytics and Data Mining
Artificial Intelligence
![Page 19: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/19.jpg)
19© Copyright 2013 Pivotal. All rights reserved. 19
@krishdpi
What?
Why?
How?(Use Cases)
Next?
![Page 20: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/20.jpg)
20© Copyright 2013 Pivotal. All rights reserved. 20.
@krishdpi
Private/Hybrid Cloud Infrastructure or Appliance
Data Access & Query Layer
Tools & Services
Analytic Productivity Layer
Hadoop
Data Scientist
Data Engineer
Data Analyst
Bl Analyst
LOB User
DatabaseData Platform Admin
DA
TA S
CIE
NC
E T
EA
M
Visualization Layer
CxO/Decision Maker
![Page 21: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/21.jpg)
21© Copyright 2013 Pivotal. All rights reserved. 21.
@krishdpi
Industries Are Broadly Embracing Big Data
Big Data Users
![Page 22: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/22.jpg)
22© Copyright 2013 Pivotal. All rights reserved. 22.
@krishdpi
Big Data Ecosystem Enablers
![Page 23: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/23.jpg)
23© Copyright 2013 Pivotal. All rights reserved. 23
@krishdpi
![Page 24: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/24.jpg)
24© Copyright 2013 Pivotal. All rights reserved. 24.
@krishdpi
ORACLESQL ServerSAP HANATerradataGreenplum
MS ExcelSASBusiness ObjectsPivotal
Platform SupportRedHatWindowsServerPivotalVMware
Modern Big Data Architecture
![Page 25: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/25.jpg)
25© Copyright 2013 Pivotal. All rights reserved. 25.
@krishdpi
Use Cases
![Page 26: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/26.jpg)
26© Copyright 2013 Pivotal. All rights reserved. 26.
@krishdpi
OLTP(Oracle)
OLAP(Teradata, Hadoop)Extract
(Nightly)
Machine Learning
@Verizon Service #Sucks! @verizon too many call drops
Special Offers
![Page 27: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/27.jpg)
27© Copyright 2013 Pivotal. All rights reserved. 27.
@krishdpi
Flight Test
ObjectiveOptimize flight time
ProblemManual diagnostics4 hours test flight is 2 TB400 000 parameters, only widely 4000 used
SolutionRealtime big data analyticsMachine learning
![Page 28: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/28.jpg)
28© Copyright 2013 Pivotal. All rights reserved. 28.
@krishdpi
ObjectiveImprove patient care
ProblemScattered member dataFrequent hospital visit
SolutionCombine behavioral, contextual dataUtilize member history and data scienceProvide accurate diagnostics
![Page 29: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/29.jpg)
29© Copyright 2013 Pivotal. All rights reserved. 29.
@krishdpi
Physical Data Strategy
Extreme OLTP(Cassandra)
Streaming Data
Interactive Data
Operational(DB2, Oracle,
Informix)
Landing(Hadoop)
Repository(Teradata)
OLTP(DB2, Oracle,
Informix)
Repository(lower SLA)(Greenplum)
Batch
BI(Teradata,
Oracle)
General BI
Perf Analytics(Greenplum)
Lab Analytics(Hadoop)
RL 2.0
Analytics Lab
![Page 30: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/30.jpg)
30© Copyright 2013 Pivotal. All rights reserved. 30
@krishdpi
![Page 31: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/31.jpg)
31© Copyright 2013 Pivotal. All rights reserved. 31
@krishdpi
![Page 32: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/32.jpg)
32© Copyright 2013 Pivotal. All rights reserved. 32
@krishdpi
![Page 33: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/33.jpg)
33© Copyright 2013 Pivotal. All rights reserved. 33
@krishdpi
Autonomous Cars
![Page 34: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/34.jpg)
34© Copyright 2013 Pivotal. All rights reserved. 34
@krishdpi
![Page 35: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/35.jpg)
35© Copyright 2013 Pivotal. All rights reserved. 35
@krishdpi
Machine LearningDeep Learning
(Beat Lee Sedol in AlphaGo. 2 followed by 56 zeros possible board positions)
![Page 36: Guest Lecture on Big Data in Business,](https://reader034.vdocuments.us/reader034/viewer/2022051708/589ef9c61a28ab06368b4f65/html5/thumbnails/36.jpg)
36© Copyright 2013 Pivotal. All rights reserved. 36.
@krishdpi
Q & A