aws summit 2011: customer presentation - nytimes

NYTimes.com

Vadim Jelezniakov

AWS Summit 2011

Next 10 min

The Company

Why Cloud

The App Examples

Progress So Far

What’s Next

The New York TimesThe International Herald Tribune

The Boston Globe15 daily newspapers

50 Web sitesNYTimes.comBoston.comAbout.com

The New York Times

Company

NYTimes.com’s

Stats

In the US:38.1 mil unique users /

month810 mil page views / month

Worldwide:62 mil unique users /

month984 mil page views / month#1

individual newspaper site#2 media

site bloggers link to the most*#4 news

and information Web site

comScore Report, April 2011

*Technorati's index

Why Cloud?

On Demand Availability

Scalability

Rapid Deployment

Why AWS?

Cost-Effective

IaaS = Flexible

Comprehensive

Why Cloud?

On Demand Availability

Scalability

Rapid Deployment

Why AWS?

Cost-Effective

IaaS = Flexible

Comprehensive

Why Cloud?

On Demand Availability

Scalability

Rapid Deployment

Why AWS?

Cost-Effective

IaaS = Flexible

Comprehensive

On Demand Capacity -

TimesMachine

Flexibility - Times Skimmer

Scalability - User Generated

Content (Comments)

Examples

TimesMachine (2007, 2008)

130 years of archives in 36 hours

400K objects, 4T of data

S3, EC2, Hadoop

http://timesmachine.nytimes.com

Skimmer Private Beta (Feb 14th, 2009)

http://www.nytimes.com/skimmer/

11:00 News leaks to TechCrunch, Google Finance11:10 Average load on the instance is 4011:15 Snapshotting the instance, configuring the LB, testing12:00 Four Skimmer instances load balanced, quality is restored1 hour - from prototype to production

“SkimmerGate”

Comments on Articles and Blogs

Rate and Review for Movies,

Theater, Dining and Travel

We like

To be able to scalequickly on demand

We don’t like

calls from Systems teamat 6PM on Fridays

Traffic Spikes - Add Capacity

Load Balancing / Security View

Everyone’s Favorites:

Development Instances!

Gap Analysis

• Load Balancing

• Communication between instances

• Talking back to existing Datacenter

• Scaling up and back

• Securing the instances

Nimbul

Light Cloud Manager

http://github.com/nimbul/nimbul

CloudSource

Simple SVN Deployment

http://github.com/nimbul/cloudsourc

ebased on ServerMattic developed by WordPress

Emissary

Fast AMQP Messaging

http://github.com/nimbul/emissar

y

Tools Developed for the

Cloud

Progress So Far

NYTimes.com EC2

Expansion

Clusters: from 2 to 100. Instances: from 30 to 400.

NYTimes.com EC2

Expansion

EC2 DC in comparison with other DCs

from 5% to 40% (VMs/instances count)

2009 2011

NYTimes.com EC2

Expansion

EC2 contributes ~26% of total operational cost across DCs

(space, power, bandwidth)

What’s Next

Multi-AZ and Multi-Region

SNS, SNS2SQS - in Dev

Multi-AZ MySQL RDS

Need 'smarter' ELBNeed

multi-AZ support in VPC

email: vadimj@nytimes.comtwitter: @vadimj

http://www.nytimes.com

http://www.nytimes.com/skimmer/http://timesmachine.nytimes.com

http://open.nytimes.com

http://github.com/nimbul/

images licensed fromhttp://www.dreamstime.com/