aws at mendeley (london, september 27th 2011)

Dan HarveySystems Architect

Revolutionising the world of research

with Amazon Web Services

Mendeley helps researchers work smarter

Mendeley makes science morecollaborative and transparent

Sync between computersand peers around the world

1 million researchers...

Uploaded 130 million citations

40 million unique

17 million papers

16 TB of data

Document Storage

• Backed on to S3

• Previews for 16TB of PDFs?

PDF Previews

ElasticBeanstalk S3

Process Queue

Load PDFs

Store PNGServe

via Cloud Front

Render to PDF

System Overview

EMREC2

S3

EBS

RDS

VPC

Article Search

• Based on Solr (open source search)• 40GB index• Variable usage

Day  of  the  month

H-pRequests

Solr Layout

SolrSlave Solr

SlaveSolr

Slave

EBS

ElasticLoad Balancer

SolrMaster

SearchQueries

Inside VPC

Outside VPC

• Machine learning on Hadoop

• Personalised article recommendations• Collaborative filtering based

• Running on Elastic Map Reduce

Summary

• Not all or nothing

• Focus on your problemnot “Undifferentiated heavy lifting”

- Werner Vogels

• Learn the building blocks AWS provide

Enjoy what you’ve seen?

We’re hiring!

Senior Java Engineers

chat to me afteror e-mail/tweet

Questions?

@DanHarveydan.harvey@mendeley.com