big data and analytics - why should we care?
DESCRIPTION
Big Data is Big and it is easy to get lost. If you are interested in a primer on what it is all about and how you can get started on the analytics, this deck will help you scratch the surface.TRANSCRIPT
![Page 1: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/1.jpg)
Big Data & Analytics – So What?
A few answers by Vishwa Kolla
(Prepared for UMass Boston MBA Students)
![Page 2: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/2.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 2
About Vishwa Kolla
Vishwa Kolla | [email protected]
Vishwa KollaSr. Consultant, Advanced Analytics & ModelingDeloitte Consulting, Boston
MBA Carnegie Mellon UniversityMS University of DenverBS BITS Pilani, India
Professional Interests Absolutely love solving a variety of business
problems using advanced, predictive analytical techniques and building decision support systems as a means
My engagements typically involve synthesizing Big Data into actionable insights
Some engagements include: Helping F5 firm solve customer attrition Helping Top 5 professional services firm solve
employee attrition Predicting what will viewers watch and when on
TV for a large Cable company Building demand forecast models Implementing scoring engines & building
simulators
Personal Interests Most recent interest - watching my 4 year old
grow (lot of fun and lot of work) Volunteering for a non-profit organization to help
it grow and shape the direction of its growth Outdoor activities – climbing 14ers (peaks over
14,000 ft. high), skiing Traveling Meeting new people Philosophy – understanding differences between
cultures and reasons why various cultures developed and are as they are currently
Coaching / Mentoring / Teaching / Helping people reach their highest potential
![Page 3: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/3.jpg)
Big Data & Analytics - Why Should We Care? 3
Contents
April 10, 2023
What is Big Data?
Why is Big Data Important?
How does Big Data manifest in our daily lives?
Who is into Big Data?
What skills are required to master Big Data?
How can I get started?
![Page 4: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/4.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 4
What is Big Data?
Source(s): (1) Gartner
Big data is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.
- Gartner
![Page 5: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/5.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 5
What is Big Data?
Source(s): (1) IBM’s Understanding Big Data eBook (2) Intel’s Big Data 101, (3) The Big Data Group (4) YouTube Press statistics
Dawn oftime
2003 2012
5 EB
…
2.7 ZB
2015
10 ZB (E)
Volume of data created Worldwide
1 YB = 10^24 Bytes 1 ZB = 10^21 Bytes 1 EB = 10^18 Bytes 1 PB = 10^15 Bytes 1TB = 10^12 Bytes 1 GB = 10^9 Bytes
Variety of data
Velocity of data
Walmart handles 1M transactions per hour Google processes 24PB of data per day AT&T transfers 30 PB of data per day 90 trillion emails are sent per year World of Warcraft uses 1.3 PB of storage
Facebook when had a user base of 900 M users, had 25 PB of compressed data
400M tweets per day in June ’12 72 hours of video is uploaded to Youtube
every minute
Radio TV News E-Mails Facebook
Posts
Tweets Blogs Photos Videos (user
and paid) RSS feeds
Wikipedia GPS data RFID POS
Scanners …
Volume
Variety
Velocity
Big Data Elements
![Page 6: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/6.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 6
How Big is Big, Really?
Source(s): (1) Mozy.com
![Page 7: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/7.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 7
Big Data & Analytics Ecosystem – It revolves around improving people’s lives
Data Providers (Instrumented & Non-instrumented)
Data Store (Structured & Unstructured)
1Data is generated from a wide variety of sources that are either Instrumented (e.g. POS scanners, Video surveillance cameras) Non-Instrumented (e.g., Facebook posts, Twitter feeds, blogs)
The format of the data is either Structured (e.g. database tables) Un-structured (e.g., E-Mails, Blogs, Photos, Videos)
2
3 Visualization &Analytics
Apps & Devices
4
People
5
Visualization tools are used to better understand inherent patterns The data is processed, transformed and analyzed to create insights More often than not, scoring models are built that auto-generate insights
Desktop / Web / Mobile apps consume these insights E.g., Desktop -> Dashboards, Web -> Movie recommendations, Mobile
(Restaurant recommendations)
Improving people’s lives is almost always the end goal The uses of big data and analytics transcends industries, firms and functions
![Page 8: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/8.jpg)
Big Data & Analytics - Why Should We Care? 8
Contents
April 10, 2023
What is Big Data?
Why is Big Data Important?
How does Big Data manifest in our daily lives?
Who is into Big Data?
What skills are required to master Big Data?
How can I get started?
![Page 9: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/9.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 9
It is not about having a lot of data; it is about USING data effectively
Source(s): Google finance
Value gap as perceived by the market. Effective use of big data amongst other things is an important driver of this gap
![Page 10: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/10.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 10
It is not really about Big Data, but is really about Tiny Data (i.e, INSIGHTS)
Who will this customer watch?
Who is likely to attrite?
Who is likely to respond to an offer?
What should I offer?
What is similar to this customer?
Who should I hire?
What will demand be in 2014?
What does this customer value?
How much should I spend on marketing?
How much stock should I carry?
Given weather patterns, what should I sell?
Which ad will this customer
watch?
What is at the risk of default?
Who is likely to vote for the
democrats?
![Page 11: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/11.jpg)
Big Data & Analytics - Why Should We Care? 11
Contents
April 10, 2023
What is Big Data?
Why is Big Data Important?
How does Big Data manifest in our daily lives?
Who is into Big Data?
What skills are required to master Big Data?
How can I get started?
![Page 12: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/12.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 12
Then and Now – Marketing
Source(s): (1) Big Data Trends by David Feinleib
Then Now
Marketing Leads Campaign Recommendations
![Page 13: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/13.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 13
Then and Now – Selling
Source(s): (1) Big Data Trends by David Feinleib
Then Now
One size fits all Personalization & Targeted Selling
![Page 14: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/14.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 14
Then and Now – IT
Source(s): (1) Big Data Trends by David Feinleib
Then Now
Peruse through log files Interactive Dashboards
![Page 15: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/15.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 15
Then and Now – Customer Service
Source(s): (1) Big Data Trends by David Feinleib
Then Now
Reactive Customer Service Pro-active Customer Service
![Page 16: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/16.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 16
Then and Now – Credibility
Source(s): (1) Big Data Trends by David Feinleib
Then Now
Credit Databases Professional & Social Networks
![Page 17: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/17.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 17
Then and Now – Operations
Source(s): (1) Big Data Trends by David Feinleib
Then Now
Maps Location Based Services
![Page 18: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/18.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 18
Then and Now – Medical Research
Source(s): (1) Big Data Trends by David Feinleib
Then Now
Keyword searches Word Clouds
![Page 19: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/19.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 19
Then and Now – Fitness
Source(s): (1) Big Data Trends by David Feinleib
Then Now
Manual tracking Focus on the goal
![Page 20: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/20.jpg)
Big Data & Analytics - Why Should We Care? 20
Contents
April 10, 2023
What is Big Data?
Why is Big Data Important?
How does Big Data manifest in our daily lives?
Who is into Big Data?
What skills are required to master Big Data?
How can I get started?
![Page 21: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/21.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 21
The Big Data buzz has begun; every one is into it …
WSJ• Teaming up on Big Data• Re-inventing society in the wake of Big Data• Wanted – A few good data scientists• Big Data adds nickels and dimes to Giant Wind Farm• Visa uses Big Data in Fraud detection• How Big Data is changing the Whole Equation of Busine
ss• Moneyball, VC Style (using Big Data)• Big Data, Big Blunders• The New Shape of Big Data• What your CEO is reading – Steam Engines Meet Big D
ata
A few company sites about Big Data• Deloitte’s Big Data site• PWC’s Big Data site• IBM’s Big Data site• Intel’s Big Data site• Microsoft’s Big Data site• Walmart
Books / Articles• IBM’s E-Book• Deloitte E-Book• HBR – The management revolution• HBR – Making Advanced Analytics work for you• HBR – Next best offer• Amazon books
Big Data in Various Industries• Healthcare• Financial Services• Big Data in Insurance• Retail
Big Data in Various Functions• Marketing• Operations• HR• Finance
![Page 22: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/22.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 22
… and they are into it very seriously
![Page 23: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/23.jpg)
Big Data & Analytics - Why Should We Care? 23
Contents
April 10, 2023
What is Big Data?
Why is Big Data Important?
How does Big Data manifest in our daily lives?
Who is into Big Data?
What skills are required to master Big Data?
How can I get started?
![Page 24: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/24.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 24
Skills Required to Master Big Data
Data Providers (Instrumented & Non-instrumented)
Data Store (Structured & Unstructured)
1 Hardware engineering Instrumentation & Design Content generators (FB posts, blogs, videos, photos)
Cloud RDBMS (SQL) NoSQL, Hadoop
2
3 Visualization &Analytics
Apps & Devices
4
People
5
Effective Data visualization techniques Statistical & Probabilistic techniques Analytical methods, tools & processes
Web 2.0 Mobile Apps Device specific - iOS / Andriod Device agnostic – HTML 5.0
Leadership Management Administrative Consulting People
![Page 25: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/25.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 25
Skills Required to Master Big Data & Analytics
Statistical &ProbabilisticTechniques
VisualizationTechniques
Programming & Trouble-shooting
Genuine Curiosity
Industries
Functions
Customer Analytics Profitable growth
opportunities Next best offer Cross-Sell
Fraud Analytics Fraudulent claims Fraudulent transactions
Marketing Analytics Pricing Price & demand
optimization Market Mix
Lifestyle & Life Stage Insurance Premium Pricing Detecting diseases based on
lifestyle
Workforce Analytics Hiring Growing Retaining
Subscription Analytics Credit Score Analytics in the cloud
![Page 26: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/26.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 26
Skills Required to Master Big Data & Analytics – Some Tools to Learn
Source(s): http://www.bigdatalandscape.com/
![Page 27: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/27.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 27
Skills Required to Master Big Data – Example 1 of effective visualization
![Page 28: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/28.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 28
Skills Required to Master Big Data – Example 2 of effective visualization
Source(s): Visual News
![Page 29: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/29.jpg)
Big Data & Analytics - Why Should We Care? 29
Contents
April 10, 2023
What is Big Data?
Why is Big Data Important?
How does Big Data manifest in our daily lives?
What skills are required to master Big Data?
Who is into Big Data?
How can I get started?
![Page 30: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/30.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 30
Navigating Big Data and Analytics is a Journey
Foundation(School)
Learn from the experts
Grow
EstablishMaster ofBig Data
& Analytics
1. Develop your eminence (by publishing your work)
1. Solve the same problem across industries
2. Solve different problems across industries
3. Apply methods across functions
1. Pay attention in Probability & Statistics courses2. Learn at least one programming language thoroughly and a few if
you can 3. Recommended minimum tool sets: R, SAS, Tableau4. Take advanced level analytical courses such as New Product
Introduction, Optimizations, Operations Research, Data-mining, Modeling, Forecasting & Time Series, Simulations
5. Practice solving problems end-to-end to understand the implication of building models and implementing them in real life
1. Learn industry best practices when you get hired into a firm
2. Surround yourself with good people and experts to accelerate your learning
3. Build / implement models under the guidance of an expert
![Page 31: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/31.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 31
Some things to watch out for
1. Big Data is not a panacea
2. Big Data is not everything for everybody
3. Big Data does not have all the answers and is directional at best if done right
4. Big Data & Analytics do not replace human intelligence ; Relying solely on Data & Analytics usually trips one up
5. There are several limitations of using Big Data & Analytics. Some are:
a) Data collection limitation -> Not all data can and is collected. One may have access to a ton of data, but very little can be analyzed and/or is meaningful
b) Data quality limitation -> Garbage in garbage out; this is getting better every day
c) Data transformation limitations -> Raw data is rarely used. It is almost always transformed. There is no perfect transformation
d) Measurement limitation -> Metrics cannot capture the entire picture
e) Modeling limitation -> Not every relationship can be modeled. The models mostly confirm / deny hypotheses. Again, models need to be evaluated for their predictive strength before adoption
f) Interpretation limitation -> One needs to be careful when interpreting results and often misinterpretations of data / metrics / model insights can be dangerous
g) Actionability limitation -> Not all insights are actionable. They may very well be interesting, but one cannot act on most insights
h) Using / Relying on single data source / data point -> Coming to a conclusion based on a single or very few biased data points can often happen
6. At the end of the day, to make Big Data & Analytics work for you, one needs to question the outcomes and insights, reconcile with understanding and use the insights as illumination as opposed to for support
![Page 32: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/32.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 32
Summary
1. Big Data is Big. It is easy to get lost. Know and understand what you are getting into before you leap
2. Make up your mind of where you want to play (i.e., get into the area where your strengths lie)
3. Build a roadmap of where you want to go and how you are going to get there
4. Fill in the skill gaps
5. Surround yourself with good people. You are a sum total of who and what you interact with
6. Have fun and enjoy what you are doing
![Page 33: Big Data and Analytics - Why Should We Care?](https://reader035.vdocuments.us/reader035/viewer/2022062511/54c6386e4a7959bf458b462e/html5/thumbnails/33.jpg)
April 10, 2023Big Data & Analytics - Why Should We Care? 33
Questions?