matt wood, chief data scientist, amazon web services

82
a presentation at the UNITED NATIONS STATISTICAL COMMISSION by DR. MATT WOOD introducing BIG DATA ANALYTICS

Upload: vuthu

Post on 14-Feb-2017

221 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Matt Wood, Chief Data Scientist, Amazon Web Services

a presentation at the UNITED NATIONS STATISTICAL COMMISSION

by

DR. MATT WOOD

introducing

BIG DATA ANALYTICS

Page 2: Matt Wood, Chief Data Scientist, Amazon Web Services

Hello.

Page 3: Matt Wood, Chief Data Scientist, Amazon Web Services

Thank you.

Page 4: Matt Wood, Chief Data Scientist, Amazon Web Services

IData, data everywhere

Page 5: Matt Wood, Chief Data Scientist, Amazon Web Services

I IIData, data everywhere

Data timeline

Page 6: Matt Wood, Chief Data Scientist, Amazon Web Services

I II IIIData

securityData, data everywhere

Data timeline

Page 7: Matt Wood, Chief Data Scientist, Amazon Web Services

I II III IVData

movementData, data everywhere

Data security

Data timeline

Page 8: Matt Wood, Chief Data Scientist, Amazon Web Services

I II III IVData

movementData, data everywhere

Data security

Data timeline

0.Amazon web

Services

Page 9: Matt Wood, Chief Data Scientist, Amazon Web Services

Compute, storage & databases.

Page 10: Matt Wood, Chief Data Scientist, Amazon Web Services

Retail Merchantservices

Web services

Page 11: Matt Wood, Chief Data Scientist, Amazon Web Services

Blinding flash of the obvious.

Page 12: Matt Wood, Chief Data Scientist, Amazon Web Services

Available.

Page 13: Matt Wood, Chief Data Scientist, Amazon Web Services

Low cost.

Page 14: Matt Wood, Chief Data Scientist, Amazon Web Services

Flexible.

Page 15: Matt Wood, Chief Data Scientist, Amazon Web Services

1.3 trillion objects835k peak requests/second

Page 16: Matt Wood, Chief Data Scientist, Amazon Web Services

300 government agencies.1,500 educational institutions.

Page 17: Matt Wood, Chief Data Scientist, Amazon Web Services

Data, data everywhereI

Page 18: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

Page 19: Matt Wood, Chief Data Scientist, Amazon Web Services

Cost of data generation is falling.

Page 20: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

lower cost,increased throughput

Page 21: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

highly constrained

Page 22: Matt Wood, Chief Data Scientist, Amazon Web Services

Gap.

Page 23: Matt Wood, Chief Data Scientist, Amazon Web Services

1990 2000 2010 2020

The Data Analysis Gap

Enterprise Data Data in Warehouse

Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011 IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares

Generated data

Available for analysis

Data volume

Page 24: Matt Wood, Chief Data Scientist, Amazon Web Services

Utility.

Page 25: Matt Wood, Chief Data Scientist, Amazon Web Services

Remove constraints.

Page 26: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

highly constrained

Page 27: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

Page 28: Matt Wood, Chief Data Scientist, Amazon Web Services

Close the gap.

Page 29: Matt Wood, Chief Data Scientist, Amazon Web Services

Technologies and techniques for working productively with data, at any scale.

Page 30: Matt Wood, Chief Data Scientist, Amazon Web Services

Data timelineII

Page 31: Matt Wood, Chief Data Scientist, Amazon Web Services

Lots of data.Lots of users.Lots of uses.

Lots of locations.

Page 32: Matt Wood, Chief Data Scientist, Amazon Web Services

Cost.

Page 33: Matt Wood, Chief Data Scientist, Amazon Web Services

Multipliers.

Page 34: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation challenge.

Page 35: Matt Wood, Chief Data Scientist, Amazon Web Services

Analytics challenge.

Page 36: Matt Wood, Chief Data Scientist, Amazon Web Services

Co-evolution.

Page 37: Matt Wood, Chief Data Scientist, Amazon Web Services

Co-evolution.

software

Page 38: Matt Wood, Chief Data Scientist, Amazon Web Services

Co-evolution.

software

utility computing

Page 39: Matt Wood, Chief Data Scientist, Amazon Web Services

Hadoop.

Page 40: Matt Wood, Chief Data Scientist, Amazon Web Services

Availability challenge.

Page 41: Matt Wood, Chief Data Scientist, Amazon Web Services

Beautiful and unique.

Page 42: Matt Wood, Chief Data Scientist, Amazon Web Services

Snowflake Statistics

Page 43: Matt Wood, Chief Data Scientist, Amazon Web Services

Data has gravity.

Page 44: Matt Wood, Chief Data Scientist, Amazon Web Services

Move data to users.

Page 45: Matt Wood, Chief Data Scientist, Amazon Web Services

Move data to users.X

Page 46: Matt Wood, Chief Data Scientist, Amazon Web Services

Move tools to data.

Page 47: Matt Wood, Chief Data Scientist, Amazon Web Services

Place data where it can be easily consumed.

Page 48: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 49: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 50: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 51: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 52: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 53: Matt Wood, Chief Data Scientist, Amazon Web Services

Reusable environment.

Page 54: Matt Wood, Chief Data Scientist, Amazon Web Services

Always more people outside your team, than within it.

Page 55: Matt Wood, Chief Data Scientist, Amazon Web Services

Technologies and techniques for working productively with data, at any scale.

Page 56: Matt Wood, Chief Data Scientist, Amazon Web Services

Data security.III

Page 57: Matt Wood, Chief Data Scientist, Amazon Web Services

Security is our number one priority.

Page 58: Matt Wood, Chief Data Scientist, Amazon Web Services

Shared responsibility.

Page 59: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 60: Matt Wood, Chief Data Scientist, Amazon Web Services

Choose your region.

Page 61: Matt Wood, Chief Data Scientist, Amazon Web Services

Availability zones.

Page 62: Matt Wood, Chief Data Scientist, Amazon Web Services

ITAR

FIPS 140-2

MPAAISO 27001

SOC 2 ISAE 3402 PCI DSS

HIPAA

FISMA Moderate

Page 63: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 64: Matt Wood, Chief Data Scientist, Amazon Web Services

Virtual Private Cloud.

Page 65: Matt Wood, Chief Data Scientist, Amazon Web Services

Network isolated environment.

Page 66: Matt Wood, Chief Data Scientist, Amazon Web Services

Data movement.IV

Page 67: Matt Wood, Chief Data Scientist, Amazon Web Services

“How do I get my data into the cloud?”

Page 68: Matt Wood, Chief Data Scientist, Amazon Web Services

Generated and stored in the AWS cloud.

Page 69: Matt Wood, Chief Data Scientist, Amazon Web Services

Inbound transfer if free.

Page 70: Matt Wood, Chief Data Scientist, Amazon Web Services

Multipart upload.

Page 71: Matt Wood, Chief Data Scientist, Amazon Web Services

Physical media.

Page 72: Matt Wood, Chief Data Scientist, Amazon Web Services

AWS Direct Connect.

Page 73: Matt Wood, Chief Data Scientist, Amazon Web Services

1Gbps or 10Gbps

Page 74: Matt Wood, Chief Data Scientist, Amazon Web Services

Built in AZ replication.

Page 75: Matt Wood, Chief Data Scientist, Amazon Web Services

Regional replication.

Page 76: Matt Wood, Chief Data Scientist, Amazon Web Services

aws.amazon.com

Page 77: Matt Wood, Chief Data Scientist, Amazon Web Services

IData, data everywhere

Page 78: Matt Wood, Chief Data Scientist, Amazon Web Services

I IIData, data everywhere

Data timeline

Page 79: Matt Wood, Chief Data Scientist, Amazon Web Services

I II IIIData

securityData, data everywhere

Data timeline

Page 80: Matt Wood, Chief Data Scientist, Amazon Web Services

I II III IVData

movementData, data everywhere

Data security

Data timeline

Page 81: Matt Wood, Chief Data Scientist, Amazon Web Services

Thank you.