strategies for backing up mongodb
DESCRIPTION
For the first time this year, 10gen will be offering a track completely dedicated to Operations at MongoSV, 10gen's annual MongoDB user conference on December 4. Learn more at MongoSV.com Come learn about the different ways to back up your single servers, replica sets, and sharded clustersTRANSCRIPT
![Page 1: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/1.jpg)
Engineering Manager, 10gen
Jeff Yemin
#MongoBoston
Strategies for Backing Up MongoDB
![Page 2: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/2.jpg)
File and Directory Layout• A set of files per database
![Page 3: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/3.jpg)
Insert with write concern of {fsync : true}
![Page 4: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/4.jpg)
Archive the data directory
![Page 5: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/5.jpg)
Restore the data directory
![Page 6: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/6.jpg)
Start mongod on restored data directory
![Page 7: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/7.jpg)
Everything is fine, right?
• No, it's not
• But you can't tell until you look
![Page 8: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/8.jpg)
Try validating the collection• In the shell, run the validate command
![Page 9: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/9.jpg)
How can we get a clean backup?• kill mongod
• fsyncLock / fsyncUnlock
![Page 10: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/10.jpg)
How can we get a clean backup?• mongodump
![Page 11: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/11.jpg)
mongodump
• Snapshot of each collection– Does NOT represent a point in time, even for a
single collection
• Can NOT be combined with fsyncLock– Remember, you can't read…
• You CAN dump directly from data files to get a point in time backup– mongodump –dbpath
• Can be costlier than archiving as FS level
![Page 12: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/12.jpg)
Snaphot Query
1
2
3 4
5
6
7
8 9
![Page 13: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/13.jpg)
How can we get a clean backup?• journaling
![Page 14: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/14.jpg)
Journaling
• Write-ahead log
• Guarantees a consistent view even after a hard crash
• Default behavior as of 2.0
• Journal stored in –dbpath /journal folder
• --journalCommitInterval* (2ms - 300ms)
![Page 15: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/15.jpg)
Journaling implications for backup
• Logical Volume Manager (LVM)
• LVM snapshots to the rescue– lvcreate –size 100M –snapshot –name mdb-snap01 /dev/vg0/mongodb
• No shutdown or fsyncLock needed
• True point in time backup for a single instance
![Page 16: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/16.jpg)
Replica Sets
![Page 17: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/17.jpg)
Backing up a replica set
• Back up a (hidden) secondary– kill mongod– fsyncLock– mongodump– LVM snapshot
![Page 18: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/18.jpg)
Mongodump for replica sets• True point in time
– mongodump –oplog– mongorestore –-oplogreplay
• Snapshot query of each collection, then replay the oplog at the end– Similar to how a new secondary does an initial
sync
![Page 19: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/19.jpg)
Sharded clusters
Shard 1 Shard 2 Shard 3 Shard 4
5
9
1
6
10
2
7
11
3
8
12
4
17
21
13
18
22
14
19
23
15
20
24
16
29
33
25
30
34
26
31
35
27
32
36
28
41
45
37
42
46
38
43
47
39
44
48
40
mongos
balancerconfig
config
config
Chunks!
![Page 20: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/20.jpg)
Backing up a sharded cluster• mongodump through mongos
– (but no –oplog)
• mongorestore through mongos
![Page 21: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/21.jpg)
Backup a Sharded Cluster1. Stop Balancer, and wait till inactive
(state:0) db.settings.update( { _id: "balancer" }, { $set : { stopped: true } } , true )
2. Stop a config server Backup Data– Each shard– Config server (mongodump --db config)
3. Restart config server
4. Resume balancer
![Page 22: Strategies for Backing Up MongoDB](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b6f0e84a79590d5e8b4571/html5/thumbnails/22.jpg)
Engineering Manager, 10gen
Jeff Yemin
#MongoBoston
Thank You