aws for the retail industry, webinar, september 2012

AWS for the Retail Industry

Ryan Shuttleworth – Technical Evangelist @ryanAWS

Adam Bidwell – eCommerce Manager, Kurt Geiger

Agenda

Amazon Web Services Background

Utility computing & Elasticity

AWS & Retail

Security & compliance

Highly available customer facing systems

Core platforms

Customer analytics

Kurt Geiger – Customer Story

Your feedback is important

Tell us:

What’s good, what’s not

What you want to see at these events

What you want AWS to deliver for

background

Consumer Business

Tens of millions of active customer

accounts

Eight countries: US, UK, Germany,

Japan, France, Canada, China, Italy

Seller Business

Sell on Amazon websites

Use Amazon technology for your own retail website

Leverage Amazon’s massive fulfillment

center network

IT Infrastructure Business

Cloud computing infrastructure for hosting web-scale

solutions

Hundreds of thousands of

registered customers in over 190 countries

Deep experience in building

and operating global web

scale systems

About Amazon Web Services

…get into cloud computing?

How did Amazon…

Over 10 years in the making

Enablement of sellers on Amazon

Internal need for scalable deployment environment

Early forays proved developers were hungry for more

AWS Mission

Enable businesses and developers to use web services* to build scalable,

sophisticated applications.

*What people now call “the cloud”

Each day AWS adds the equivalent server capacity to power Amazon when it was a global, $2.76B

enterprise

(circa 2000)

250.000

500.000

750.000

1000.000

1 Trillion

750k+ peak transactions per second

Objects in S3

Utility computing

On demand Pay as you go

Uniform Available

Utility computing

Compute

Storage

Security Scaling

Database

Networking Monitoring

Messaging

Workflow

Load Balancing

Backup CDN

Uniform Available

Utility computing

On a global footprint

Region

US-WEST (N. California) EU-WEST (Ireland)

ASIA PAC (Tokyo)

ASIA PAC (Singapore)

US-WEST (Oregon)

SOUTH AMERICA (Sao Paulo)

US-EAST (Virginia)

GOV CLOUD

Availability Zone

Edge Locations

Dallas(2)

St.Louis

Jacksonville Los Angeles (2)

Palo Alto

Seattle

Ashburn(2)

Newark

New York (2)

Dublin

London(2)

Amsterdam

Stockholm

Frankfurt(2)

Paris(2)

Singapore(2)

Hong Kong

Sao Paulo

South Bend

San Jose

Osaka Milan

Sydney

Elasticity

Traditional IT

capacity

Elastic capacity

Capacity

Time Your IT needs

On and Off Fast Growth

Variable peaks Predictable peaks

Elastic capacity

On and Off Fast Growth

Predictable peaks Variable peaks

CUSTOMER DISSATISFACTION

Elastic capacity

Elastic cloud capacity

Traditional

IT capacity

Your IT needs

Capacity

Elastic capacity

Fast Growth On and Off

Predictable peaks Variable peaks

Elastic capacity

503 Service Temporarily Unavailable

The server is temporarily unable to service

your request due to maintenance downtime or

capacity problems. Please try again later.

503 Service Temporarily Unavailable

The server is temporarily unable to service

your request due to maintenance downtime or

capacity problems. Please try again later.

From one instance…

…to thousands

And back again…

4/12/2008 4/14/2008 4/15/2008 4/16/2008 4/18/2008 4/19/2008 4/20/2008 4/17/2008 4/13/2008

40 servers to 5000 in 3 days

EC2 scaled to peak of 5000 instances

“Techcrunched”

Launch of Facebook modification

Steady state of ~40 instances

Security you can rely upon

Foundation Services

Compute Storage Database Networking

AWS Global Infrastructure Regions

Availability Zones

Edge Locations Am

Shared responsibility

Foundation Services

Availability Zones

Edge Locations Am

Sarbanes-Oxley (SOX)

ISO 27001 Certification

Payment Card Industry Data Security

Standard (PCI DSS) Level 1 Compliant

SAS70(SOC 1) Type II Audit

FISMA A&As Multiple NIST Low Approvals to Operate (ATO) NIST Moderate, GSA issued ATO FedRAMP

DIACAP MAC III Sensitive IATO Customers have deployed various compliant applications such as HIPAA (healthcare)

Foundation Services

Availability Zones

Edge Locations

Client-side Data Encryption & Data Integrity Authentication

Server-side Encryption (File System and/or Data)

Network Traffic Protection (Encryption/Integrity/Identity)

Platform, Applications, Identity & Access Management

Operating System, Network & Firewall Configuration

Customer Data

AWS and Retail

Customer facing infrastructure

DNS Application Data

Rule 1: Service all web requests

a) Make sure requests get to your ‘front door’

DNS Application Data Request

…then this is irrelevant

Clients can’t resolve you?

“100% Available”

Route53

Feature Details

Global Supported from AWS global edge locations for fast and reliable domain name resolution

Scalable Automatically scales based upon query volumes

Latency based routing Supports resolution of endpoints based upon latency, enabling multi-region application delivery

Integrated Integrates with other AWS services allowing Route 53 to front load balancers, S3 and EC2

Secure Integrates with IAM giving fine grained control over DNS record access

http://aws.amazon.com/route53/sla

a) Make sure requests get to your ‘front door’ b) Make sure you open the door when they arrive

Route53

Region

Elastic Load

Balancer Region

Availability Zone

Route53

a) Make sure requests get to your ‘front door’ b) Make sure you open the door when they arrive

Elastic load balancing Multi-availability zone Multi-region

Region

a) Make sure requests get to your ‘front door’ b) Make sure you open the door when they arrive c) Have the data to form a response

Elastic Load

Balancer Region

Availability Zone

Route53

Region

Elastic Load

Balancer

Route53

Region

Availability Zone

a) Make sure requests get to your ‘front door’ b) Make sure you open the door when they arrive c) Have the data to form a response

Multi-AZ RDS

(Master-slave)

Inter-region replication

Read-replicas

Rule 2: Service requests as fast as possible

a) Choose the fastest route

Region A

Route53

Region B

Request

Region A

Route53

Region B

16ms 92ms

Request

Region A

Route53

Region B

16ms 92ms

Request

Region A

Route53

Region B

Request

Region A DNS entry

a) Choose the fastest route b) Offload your application servers

London

Served from S3

/images/*

Served from EC2

Single CNAME

www.mysite.com

CloudFront World-wide content distribution network

Easily distribute content to end users with low

latency, high data transfer speeds, and no

commitments.

Without CloudFront

EC2 webservers/app servers loaded by user

requests

With CloudFront

Load of user requests pushed into

CloudFront, EC2 cluster can scale

Offload Scale Down

No CDN CDN for

Static

Content

CDN for

Static &

Dynamic

Content

Offload Scale Down

Rule 3: Handle requests at any scale

a) Scale up

Vertical Scaling

From $0.02/hr

Basic unit of compute capacity

Range of CPU, memory & local disk options

14 Instance types available, from micro through cluster

compute to SSD backed

Scale up with Elastic Compute Cloud (EC2)

a) Scale up b) Scale out

Trigger

auto-scaling

policy

as-create-auto-scaling-group MyGroup

--launch-configuration MyConfig

--availability-zones eu-west-1a

--min-size 4

--max-size 200

Auto-scaling Automatic re-sizing of compute clusters based upon demand

Manually

Send an API call or use CLI to launch/terminate instances – Only need

to specify capacity change (+/-)

By Schedule

Scale up/down based on date and time

By Policy

Scale in response to changing conditions, based on user configured real-time

monitoring and alerts

Auto-Rebalance

Instances are automatically launched/terminated to ensure the

application is balanced across multiple Azs

Manually

Send an API call or use CLI to launch/terminate instances – Only need

to specify capacity change (+/-)

By Schedule

Scale up/down based on date and time Preemptive manual scaling of capacity

e.g. before a marketing event add 10 more instances

Regular scaling up and down of instances

e.g. scale from 0 to 2 to process SQS messages every night or double capacity

on a Friday night

By Policy

Scale in response to changing conditions, based on user configured real-time

monitoring and alerts

Auto-Rebalance

Instances are automatically launched/terminated to ensure the

application is balanced across multiple Azs

Dynamic scale based upon custom metrics

e.g. SQS queue depth, Average CPU load, ELB latency

Maintain capacity across availability zones

e.g. Instance availability maintained in event of AZ becoming unavailable

a) Scale up b) Scale out c) Dial it up

Elastic Block Store Provisioned IOPS up to 1000 per EBS

volume

Predictable performance for

demanding workloads such as

databases

DynamoDB Provisioned read/write performance per

Predictable high performance scaled via

console or API

Relational Database Service Database-as-a-Service

No need to install or manage database instances

Scalable and fault tolerant configurations

DynamoDB Provisioned throughput NoSQL database

Fast, predictable performance

Fully distributed, fault tolerant architecture

Use RDS for databases

Use DynamoDB for high performance key-

value DB

Rule 4: Simplify architecture with services

Amazon SQS

Processing

task/processing

trigger

Processing results

Amazon SQS Reliable, highly scalable, queue service

for storing messages as they travel

between instances

Task A

Task B

(Auto-scaling)

Task C

Simple Workflow Reliably coordinate processing steps

across applications

Integrate AWS and non-AWS resources

Manage distributed state in complex

systems

Push inter-process workflows into the cloud with SWF

Reliable message queuing without

additional software

Cloud Search Elastic search engine based upon

Amazon A9 search engine

Fully managed service with

sophisticated feature set

Scales automatically

Document Server

Results

Search Server

Don’t install search software, use CloudSearch

Process large volumes of data cost effectively

with EMR

Elastic MapReduce Elastic Hadoop cluster

Integrates with S3 & DynamoDB

Leverage Hive & Pig analytics scripts

Integrates with instance types such as

“Amazon CloudSearch is a game-changing product that has allowed us to deliver powerful

new search capabilities. Our customers can now find what they are looking for faster and more

easily than ever before…

….We saved many months of re-architecture and development time by going with Amazon

CloudSearch”

Don MacAskill CEO & Chief Geek

SmugMug

10 Million records 44 GB collection more than 2,000 operations per second Order volumes increase substantially during the holidays necessitating elasticity

Core platforms

Certification of SAP BusinessObjects business intelligence solutions and SAP Rapid Deployment Solutions (RDS) on

Linux & Windows Server 2008 R2

Certification of SAP Business All-in-One on Linux & Windows Server 2008 R2

Certified database engines for production SAP deployments: MaxDB, DB2, MS SQL Server 2008 R2

Non production systems

(dev, test, staging)

Backup, archive and recovery

(databases, AMIs)

http://aws.amazon.com/sap/

Production systems

(Analytics, branch etc)

Relational Database Service Database-as-a-Service

No need to install or manage database instances

Scalable and fault tolerant configurations

Feature Details

Platform support Create MySQL, SQL Server and Oracle RDBMS

Preconfigured Get started instantly with sensible default settings

Automated patching Keep your database platform up to date automatically

Backups Automatic backups and point in time recovery and full DB backups

Backups Volumes can be snapshotted for point in time restore

Failover Automated failover to slave hosts in event of a failure

Replication Easily create read-replicas of your data and seamlessly replicate data across availability zones

Disaster recovery in AWS

Pilot light architecture

Build resources around

replicated dataset

Keep ‘pilot light’ on by replicating core

databases

Build AWS resources around dataset and

leave in stopped state

replicated dataset

databases

Scale resources in AWS in

response to a DR event

Start up pool of resources in AWS when

events dictate

Match current production capacity through

auto-scaling polcies

replicated dataset

databases

Scale resources in AWS in

response to a DR event

Start up pool of resources in AWS when

events dictate

Match current production capacity through

auto-scaling policies

Switch-over to system in AWS

Customer analytics

We can collect more

Big Data

There is more

Big Data

And data has gravity…

Big Data

Data App App

http://blog.mccrory.me/2010/12/07/data-gravity-in-the-clouds/

Data has gravity

Compute Storage Big Data

…and inertia at volume…

…easier to move applications to the data

Lorem ipsum dolor sit

amet, consectetur

adipiscing elit. Etiam

quis ligula neque, eget

venenatis sem.

Suspendisse non eros

nulla, at placerat nibh.

Very large dataset seeks strong &

consistent compute for

short term relationship,

possibly longer. GSOH a

plus aws.amazon.com

Personal

Lorem ipsum dolor sit

amet, consectetur

adipiscing elit. Etiam

quis ligula neque, eget

venenatis sem.

Suspendisse non eros

nulla, at placerat nibh.

Cras id lectus mattis est

ullamcorper blandit.

Proin ut nisi vitae enim

vulputate tempor.

Phasellus id commodo

eros. Mauris nec

dignissim turpis. Nunc

Cras id lectus mattis

est ullamcorper

blandit. Proin ut nisi

vitae enim vulputate

tempor. Phasellus id

commodo eros.

Mauris nec dignissim

turpis. Nunc

Bring compute capacity to the data

Cras id lectus mattis

est ullamcorper

blandit. Proin ut nisi

vitae enim vulputate

tempor. Phasellus id

commodo eros.

Mauris nec dignissim

turpis. Nunc

Cloud has the power to process

From one instance…

…to thousands

and back again…

The revolution

have data

can store

have data

can store can analyse

have data

economically

Who is your customer really?

What do people really like?

What is happening socially with your products?

How do people really use your products?

Lesson 1: don’t leave your Amazon account logged in at home

Lesson 2: use the data you have to

drive proactive processes

1 instance for 100 hours =

100 instances for 1 hour

Small instance = $8

Amazon Elastic MapReduce

Elastic MapReduce Managed, elastic Hadoop cluster

Integrates with S3 & DynamoDB

Leverage Hive & Pig analytics scripts

Integrates with instance types such as spot

Feature Details

Scalable Use as many or as few compute instances running Hadoop as you want. Modify the number of instances while your job flow is running

Integrated with other services Works seamlessly with S3 as origin and output. Integrates with DynamoDB

Comprehensive Supports languages such as Hive and Pig for defining analytics, and allows complex definitions in Cascading, Java, Ruby, Perl, Python, PHP, R, or C++

Cost effective Works with Spot instance types

Monitoring Monitor job flows from with the management console

But what is it?

A framework Splits data into pieces Lets processing occur

Gathers the results

Elastic MapReduce

Code Name node

Output S3 + SimpleDB

S3 + DynamoDB

Elastic cluster

HDFS Queries

+ BI Via JDBC, Pig, Hive

Input data

Very large click log (e.g TBs)

Lots of actions by John Smith

Split the log into

many small pieces

Split the log into

many small pieces

Process in an EMR cluster

Split the log into

many small pieces

Aggregate the results

from all the nodes

What John Smith

Split the log into

many small pieces

Aggregate the results

from all the nodes

What John Smith

Very large click log (e.g TBs) Insight in a fraction of the time

1 instance for 100 hours =

100 instances for 1 hour

Small instance = $8

1 instance for 1,000 hours =

1,000 instances for 1 hour

Small instance = $80

Features powered by Amazon Elastic MapReduce:

People Who Viewed this Also Viewed

Review highlights Auto complete as you type on search

Search spelling suggestions Top searches

200 Elastic MapReduce jobs per day Processing 3TB of data

“With AWS, our developers can now do things they couldn’t before…

…Our systems team can focus their energies on other

challenges.”

Dave Marin Search and data-mining engineer

Elastic MapReduce Web log analysis and recommendation algorithms

Adam Bidwell eCommerce Manager

Overview of Kurt Geiger

Kurt Geiger are responsible for the operation of three retail websites: • Kurtgeiger.com • Shoeaholics.com • Ninewest.co.uk In total serving upwards of a half-million page views a day.

Our interest in Amazon, is to host:

• Frontend systems - three Magento installations which the stores are built on.

• Administration systems – backend tasks, such

as product enrichment and reporting. • Testing – load-testing systems, and other

‘sandpit’ tasks • Research/Development – one-off installations for

investigation purposes.

Challenges faced by Kurt Geiger:

• Rapidly changing business needs – fast pace

makes it difficult to predict long-term requirements

• Marketing activity – drives large traffic spikes

Why Amazon?

• Unique model – we’ve used several cloud providers

Amazon offer a wide range of network/server infrastructure and services.

• Self-service – 24/7 help yourself approach, suits us to take what we need when we need it

Future

• Larger capacity architectures

• More API based “pop-up” systems on demand

• Reserved instances - further cost savings

Benefits

• Hourly billing – the cost adapts with our current set

up, no tie-in

• Large capacity – Whether capacity will be there is not a consideration, it just is

• Trusted provider – architecture still requires planning for good reliability, but AWS has robust infrastructure to build on

aws.amazon.com/free

aws for the retail industry, webinar, september 2012

web requestsa

make sure requests

data encryption data

elastic capacity traditional

capacity problems

elastic capacity waste

available customer

equivalent server capacity

Technology

aws march 2016 webinar series building your data lake on aws

aws webcast - aws webinar series for education #3 - discover...

webinar retail focus on retail intro

aws april webinar series - getting started with aws...

aws apac webinar week - running microsoft sql server on aws

continuous delivery with aws lambda - aws april 2016 webinar...

aws apac webinar week - big data on aws. redshift, emr, &...

aws educate - webinar slides

aws october webinar series - introducing aws mobile hub

aws january 2016 webinar series - introduction to docker on...

aws october webinar series - introducing aws import / export...

aws logianalytics engagecx webinar share

improving infrastructure governance on aws - aws june 2016...

aws segment xo group joint webinar

aws july webinar series: getting started with aws device...

aws apac webinar week - understanding aws storage options

aws march 2016 webinar series - getting started with aws...

aws webcast - library storage webinar

aws april 2016 webinar series - getting started with aws

aws may 2016 webinar series - aws services overview