back to basics webinar 6: production deployment

MongoDB Europe 2016Old Billingsgate, London

15th November

mongodb.com/europe

Register with code JD20 for a 20% discount

Back to Basics 2016 : Webinar 6

Production DeploymentJoe Drumgoole

Director of Developer Advocacy, EMEA@jdrumgoole

V1.1

3

Recap

• Webinar 1 – Introduction to NoSQL– NoSQL Types, MongoDB is a document database

• Webinar 2 – My First Application– Creating databases and collections, CRUD, Indexes and Explain

• Webinar 3 – Schema Design– Dynamic schema, Embedding approaches

• Webinar 4 – GeoSpatial and Text Indexes• Webinar 5 – Introduction to the Aggregation Framework

4

Production Deployment

• The process of taking a development project and making it available to end users

• From a database perspective key concerns:– Durability– Scalability– Data security– General production notes

5

Durability – Replica Sets

Node 1

Node 2 Node 3

6

Replica Set Initialize

Node 1

Node 2 Node 3

Node 1(Primary)

Node 2(Secondary)

Node 3(Secondary)

Replication Replication

Heartbeat

7

Node 2(Secondary)

Node 3(Secondary)

Heartbeat

Primary Election

Node 1(Primary)

Replica Set - Failure

8

Node 1(Primary)

Node 2(Primary)

Node 3(Secondary)

Heartbeat

Replication

Replica Set - Failover

9

Node 2(Primary)

Node 3(Secondary)

Heartbeat

Replication

Node 1(Recovery)

Replication

Replica Set - Recovery

10

Node 2(Primary)

Node 3(Secondary)

Heartbeat

Replication

Node 1(Secondary)

Replication

Replica Set - Recovered

Tunable Consistency

12

Node 1(Primary)

Node 2(Secondary)

Node 3(Secondary)

Application

Driver

Read and write

Strong Consistency

13

Node 1(Primary)

Node 2(Secondary)

Node 3(Secondary)

Application

Driver

Strong Consistency

Read Read

Read and write

14

Write Concern

• Controls the level of acknowledgement a write operation will receive• w:0 : Once the packet leaves the driver we are done• w:1 : Return errors generated by the server• w:majority : Ensure write persisted to a majority of the replica set members

• Journaling • j:false : Writes do not need to persist to the journal• j:true : Writes must persist to the journal

Tunable Consistency: w:0

Tunable Consistency: w:1

Tunable Consistency: w:j

Tunable Consistency: w:majority

Sharding

20

I/O Bottlenecks

Memory Bottlenecks

Sharded Cluster

mongos mongos mongos

Primary

Secondary

Secondary

Primary

Secondary

Secondary

Primary

Secondary

Secondary

Primary

Secondary

Secondary

Primary

Secondary

Secondary

App

Driver

Config Server

23

Setting Up Replica Sets And Sharded Clusters

• Simplest – our management, our servers– MongoDB Atlas

• Slightly More Complicated – cloud management - your servers– MongoDB Cloud Manager

• More Complicated Again – on premise management – your servers– MongoDB Ops Manager

• Hardest– Do it all yourself with scripts and manual steps – see docs for details

24

Data Security – Defense in Depth

• Use SSL for all connections• Setup up users, passwords and access roles for all databases• Use encryption on disk for credit card and/or personally identifiable

information• Turn on audit if you need to to keep track of who is changing what• Make sure your monitoring is enabled for all nodes to identify unusable

information• Don't make your databases globally accessible

25

General Production Notes

• Use the most recent release of MongoDB and keep it patched (64 bit builds only in production)• Use the Wired Tiger storage engine (its our default and will ensure great performance)• Configure the appropriate Write Concern• Configure the connection pool for the server to match the load• Make sure you have sufficient RAM and CPU for the workload you are expecting• Prefer SSDs• Disable NUMA ( Non Uniform Memory Access)• Allocate SWAP space (otherwise on Linux the OOM daemon may kill you)• Review Wired Tiger compression options, none, snappy, zlib (snappy is the default)• If you are using RAID, RAID10 is the recommended configuration• Use the XFS file system if possible for better performance• Use a noop scheduler for virtual devices when using VMs• Synchronise time on all cluster members with NTP• Turn off the "atime" option for the file-system storing database files• Increase the file descriptor limit• Avoid NFS file-systems or file systems that provide underlying replication (SANs)

26

Before You Go Live

• Talk to us• Get a health check• Use our automated tools

Most production problems happen because the customer tried to go it alone

Q&A