google cloud platform - kxgoogle cloud platform confidential & proprietary 6 2002 2004 2006 2008...

23
Confidential & Proprietary Google Cloud Platform Antonio Zurlo [email protected]

Upload: others

Post on 29-May-2020

13 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Confidential & Proprietary

Google Cloud Platform

Antonio Zurlo [email protected]

Page 2: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

$29 billion Over 1 billion 5,000 caching points

99.95% uptime

Page 3: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Over one billion users each.

Google Search

Android Chrome Google Maps

Google Play

YouTube Gmail Google Cloud

Page 4: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Google Cloud Platform Regions15 current regions. 4 new regions coming.

3

3

3

3

33

3

3

333

3

4

3

3

3

3

3

3

3

Page 5: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

proprietary & confidential | not for distribution

2012 20132002 2004 2006 2008 2010

Google Research Publications referenced are available here: http://research.google.com/pubs/papers.htmlThe Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, 2009 http://research.google.com/pubs/pub35290.html

GFS

MapReduce

BigTable

Google Research in Infrastructure Technologies

Colossus

Dremel Flume

Megastore

Spanner

Millwheel

PubSub

F1

Page 6: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Confidential & ProprietaryGoogle Cloud Platform 6

2012 20132002 2004 2006 2008 2010

Cloud Storage

DataProc

BigTable Cloud Storage

BigQuery DataFlow

DataStore

Spanner

DataFlow

PubSub

F1

Available as of Sep 2015 at cloud.google.com

Google Research Publications referenced are available here: http://research.google.com/pubs/papers.htmlThe Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, 2009 http://research.google.com/pubs/pub35290.html

Google Products in Infrastructure Technologies

Page 7: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Tensorflow

Page 8: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

8

Google Cloud PlatformBuilding Blocks

Network DataCompute

Page 9: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Confidential & Proprietary

Compute Engine

Page 10: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

10

Google Compute

Highly Configurable Resources

Managed Compute Paradigms

Page 11: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Better EconomicsSUSTAINED USE DISCOUNTSThe more you use, the bigger the discount: automatically get up to 30%-off workloads that run for the duration of the billing month on Compute Engine 1

COMMITTED USE DISCOUNTSSavings of up to 57% without upfront fees and with instance-type flexibility 1

PER SECOND BILLINGYou pay per SECOND, with a one minute minimum.

CUSTOM MACHINE TYPESPick any configuration of CPU, memory, and attached devices. Our customers have saved an average of 19% — and as much as 50% — on top of our already market-leading prices.3

PREEMPTIBLE VM INSTANCESUp to 80%-off2 workloads that can be interrupted, like HTC, Batch and data processing.

1https://cloud.google.com/compute/pricing2https://cloud.google.com/preemptible-vms/3https://cloudplatform.googleblog.com/2016/02/Custom-Machine-Types-goes-GA-saving-you-up-to-50-percent-on-compute-costs.html

Page 12: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Managed Instance Groups

Page 13: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Intel Skylake CPUs

Page 14: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

NVIDIA GPUs

Page 15: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Confidential & Proprietary

Networking

Page 16: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

16

Network/Bandwidth

Google Networking

Network

Efficient SDN network

●●

Performance

●●●

Connectivity

●●●

Page 17: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

17

Page 18: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Confidential & Proprietary

Storing data on GCP

Page 19: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Storage options on GCP

GOOGLE CLOUD STORAGE

PERSISTENT DISK

LOCAL STORAGE

ON-SITE SOLUTIONS

DataStorage

Page 20: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

Block Storage Options

Local SSDFastest, Attached, Ephemeral

Persistent Disk: SSDFast, Persistent, Durable, Remote

Persistent Disk: HDDCheapest, Persistent, Durable, Remote

Targetscenarios

- High-performance scratch space. Frequently accessed data.

Features

- Max Throughput: 800 MB/s read, 400 MB/s write

- Max Throughput: 180 MB/s read, 120 MB/s write

Price / GB

Page 21: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

GCS Object Storage

●●●●

●●●

Page 22: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

22

Kx on GCP

StorageBigQuery

StorageCloud Storage

Pipeline ManagementCloud Composer

Data AnalysisCloud Datalab

ProcessingCloud Dataproc

MLCloud ML

Architecture: Kx > GCP

StreamingCloud Pub/Sub

On premises

Corporate SourcesStorage Compute

Cold StorageNearline Storage

Business Analysis

ETLCloud Dataflow

Transform Data

TensorFlow

External Feeds

embedPy

Page 23: Google Cloud Platform - KxGoogle Cloud Platform Confidential & Proprietary 6 2002 2004 2006 2008 2010 2012 2013 Cloud Storage DataProc BigTable Cloud Storage BigQuery DataFlow

23

Thank You