mongodb on ec2 and ebs
DESCRIPTION
Best practices and recommendations for deploying MongoDB on Amazon EC2 and EBS.TRANSCRIPT
MongoDB on Amazon Web Services
Webinar 3-14-2011
Agenda
• MongoDB deployment components• Amazon EC2 infrastructure• Deployment scenarios
MONGODB COMPONENTS
MongoDB Components
• Actual data• Needs RAM +
Disk IO
• Stores sharding configuration
• Stores small amounts of data
• Infrequently queried/updated by MongoS
• Stateless router• Typically run on App
Servers
• Can run as Arbiter• No data• Just votes to elect
primary
Replica Sets
Primary
Secondary
Secondary
Primary
Secondary
Arbiter
Primary
Secondary
Secondary
Secondary
Secondary
Sharding
MongoD
MongoD
MongoD
MongoD
MongoD
MongoD
MongoD
MongoD
MongoD
MongoD
MongoD
MongoD
ConfigD
ConfigD
ConfigD
MongoS MongoS MongoS
AMAZON EC2
Amazon EC2 Instance Types
32-bit = Don’t Use
ConfigD / Arbiter
32-bit = Don’t Use
Typical MongoD
Big MongoD
High CPU not necessary
OS• Turn off atime• Raise file descriptor limits
cat >> /etc/security/limits.conf << EOF* hard nofile 65536* soft nofile 65536EOF
• DO NOT use large VM pages• Use ext4, xfs• Use RAID
– RAID10 on MongoD– RAID1 on ConfigbDB
• Warning! Known problems with Ubuntu 10.04 & EBS– https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853– https://bugzilla.kernel.org/show_bug.cgi?id=16991
MONGODB COMPONENTS ON EC2
MongoD on EC2
MongoD
RAID 10
EBS Volumes
64-Bit EC2 Instance
LVM orMDADM
MongoS on EC2
32- or 64-Bit EC2 Instance
(Typically your app server)
MongoS
Your AppLanguage Specific Driver
Arbiter on EC2
Arbiter64-Bit EC2 Instance
(Micro Instance is Fine)
Local Instance Storage
ConfigDB on EC2
ConfigDB64-Bit EC2 Instance
(Micro Instance is Fine)
LVM or MDADM
RAID 1
EBS Volumes
DEPLOYMENT SCENARIOS
3-Node Replica Set
MongoD(Primary)
RAID 10
MongoD(Secondary)
RAID 10
MongoD(Secondary)
RAID 10
Your App
Large Large Large
2-Nodes + Arbiter
MongoD(Primary)
RAID 10
MongoD(Secondary)
RAID 10
Arbiter
Your App
Large Large Micro
US WestUS East
3-Node Replica Set Your App
MongoD(Primary)
RAID 10
Large
MongoD(Secondary)
RAID 10
Large
MongoD(Secondary)Priority = 0
RAID 10
Large
App Tier
Config DB
Shard 3Shard 2Shard 1
ShardingAmazon ELB
RUNNING IN EC2
Operational Considerations
• Automating Deployment – Puppet – Chef – Amazon CloudFormation
Backup / Restore
• EBS Snapshots– Also for rapid replica provisioning
• LVM Snapshots• Use secondaries for zero downtime backups
• http://www.mongodb.org/pages/viewpage.action?pageId=19562846
Security GroupsComponent Default PortsMongoD 27017, 28017 (Status page)MongoD Shard Server 27018MongoS 27017ConfigDB 27019
• All nodes listen on all interfaces unless –bind_ip is specified
• Specify alternate port with –port • Don’t mix names & IP’s. Names preferred• Use security groups as access control lists • (e.g. allow 27017 from app servers, but not
everybody)
Final thoughts
• EC2 is a great platform for Mongo
• Lots of production use
• Try it out! Test a big cluster for cheap!
• We’re here to help!
Questions?
• I am– Jared Rosoff ([email protected]) @forjared
• Try it out! – http://www.mongodb.org/downloads– http://www.mongodb.org/display/DOCS/Amazon
+EC2