running production mongodb lightning talk
DESCRIPTION
These slides cover data modeling, indexing and DBA tools. Talk given at Meteor London MeetUpTRANSCRIPT
featuring special guest Sean Bean
Let’s talk about production
MongoDBD
ata
ba
se
Ha
pp
ine
ss
Developer Fun
But it doesn’t need to be
this way.
Let’s use House Stark as an
example
It starts with the data model
_id name home p_id
0 Ned Stark Winterfell 0
1 Arya Stark Winterfell 1
2 Bran Stark Winterfell 2
p_id proficiencies
0keeping a cool
head
1pulling off the short
hair look
2 riding piggyback
Arrays in MongoDB
{
_id: ObjectId(“000”),
name: “Ned Stark”,
relatives: [
ObjectId(“001”),
ObjectId(“002”)
]
}
{
_id: ObjectId(“000”),
name: “Ned Stark”,
relatives: [
{
_id: ObjectId(“001”),
name: “Arya Stark”,
…
},
{
_id: ObjectId(“002”),
name: “Bran Stark”,
…
}
]
}
vs.
What are “total normalization”
trade-offs?{
_id: ObjectId(“000”),
name: “Ned Stark”,
relatives: [
ObjectId(“001”),
ObjectId(“002”)
]
}
Here is Ned Stark.
His relatives are:
ObjectId(“001”) ObjectId(“002”)
And “partial denormalization”?
{
_id: ObjectId(“000”),
name: “Ned Stark”,
relatives: [
{
_id: ObjectId(“001”),
name: “Arya Stark”,
img: “../arya.png”
}
{
_id: ObjectId(“002”),
name: “Bran Stark”,
img: “../bran.png”
}
]
}
Here is Ned Stark.
His relatives are:
Arya Stark Bran Stark
Rules on Arrays
High reads, low updates/writes
Array should be bound in size
Don’t need to frequently access
embedded items individually
#1 cause of slow databases:
(lack of) indexes
cursor.explain() tells us if we’re
indexed{
"cursor" : "<Cursor Type and Index>",
"n" : <num>,
"nscannedObjects" : <num>,
"nscanned" : <num>,
"nscannedObjectsAllPlans" : <num>,
"nscannedAllPlans" : <num>,
"scanAndOrder" : <boolean>,
"indexOnly" : <boolean>,
"millis" : <num>,
"indexBounds" : { <index bounds> },
"allPlans" : [
{ "cursor" : "<Cursor Type and Index>",
"n" : <num>,
"nscannedObjects" : <num>,
"nscanned" : <num>,
"indexBounds" : { <index bounds> }
},
...
]
}
Remember our indexing
essentials
NO BasicCursor!!!!!!!!!!!!! NO!
n vs nScanned
scanAndOrder
Tier 1
Tier 2
Tier 3
• mongod logs
• db.currentOp() & db.killOp()
How to save your database
• 2014-10-09T02:26:58.994-0700 [conn367784] query gotDB.people query: {
$query: { isAlive: true }, $orderby: { age: 1 } } planSummary: IXSCAN { isAlive:
1 }, cursorid:1736436984952 ntoreturn:10 ntoskip:0 nscanned:7677
nscannedObjects:7677 scanAndOrder:1 keyUpdates:0 numYields:0
locks(micros) r:115557 nreturned:10 reslen:34036 115ms
• 2014-10-09T21:26:58.994-0700 [conn127156] command gotDB.$cmd
command: mapReduce { $msg: "query not recording (too large)" }
planSummary: COLLSCAN keyUpdates:0 numYields:3843 locks(micros)
W:2840 r:85085260 w:26899 reslen:187 76657ms
There’s lots of good info in the
logs
Use currentOp() to triage slow
operations
db.currentOp().inprog.forEach(
function(op) {
if(op.secs_running > 5) printjson(op);
}
)
Terminate slowness
{
"locks": {
"^myDB": "R"
},
"ns": "myDB.bar",
"op": "query",
"opid": 1344808,
"query": {},
"secs_running": 53,
"waitingForLock": false
}
db.killOp(1344808)
