Update AnatomyThe guide to how an update works

Me = Scott Hernandez● Worked on Morphia (+java), community● Over 3 years with MongoDB● Worked on update re-write in 2.6● Working on replication team now● Lots of experience with many parts

User Components

update({f:1}, {$set: {a: 2}}, {multi:true})● Query● Update Modifiers (or replacement doc)● Options

o multi, upserto writeConcern

New Code (in 2.6)● New update and query code● Replaced all modifier implementations● New validation and error messages● Using new “MutableBSON” document class● New oplog entry generation code

Internal Components● Update Command (-> request)● Update Executor: Runner + Driver● *Modifiers (Inc, Set, Unset, Bit, …)● Validation, Modification, and Replication

Two Paths To Update

Wire Protocol Op

User

Write Commands

Internal Update Request

… don’t forgetFindAndModify ;)

Update Request1. Create

o Validate argso Prepare what we can (reduce lock time)o Record system state (repl, sharding, etc)

2. Send for execution

What is a modifier{$set: {a: 1} }, {$min: {c: “b”} }

● Modifier: “set”, “min”● Field Path: “a”, “c” (+ “b”)● Value: 1, “b”

Implementing a modifierdb/ops/modifier_interface.h:● init ( modExpr, opts ) // validation● prepare ( doc, fieldName, execInfo ) // work● apply ( ) // update document● log ( ) // oplog entry created

Modifier::init● Validate form of update, and args● Parse path● Indicate if positional, and supported

Modifier::prepare● Work with doc, but don’t change it● Validate field, type, operation● Substitute positional path element, if appl.● Do work, stored to the side● Return ExecInfo (no-op, fields affected)

Modifier::apply● Apply prepared work to doc

Modifier::log● Calculate transformation logic for oplog entry● Update idempotent oplog entry

o Seto Unseto Full Replacement

db/ops/modifier_* + tests● add_to_set.h/cpp● bit.h/cpp● compare.h/cpp● current_date.h/cpp● inc.h/cpp● obj_replace.h/cpp● pop.h/cpp

● pull.h/cpp● pull_all.h/cpp● push.h/cpp● rename.h/cpp● set.h/cpp● unset.h/cpp

Update Executor● Finish prep● If no docs + upsert, prepare insert doc● Run driver (mods) against each doc● Ensure, validate, log for replication● Return UpdateResult

Update FlowQuery + Update

Cur

sor ● Init

● Prepare● Apply

{ $inc: {a:1}, { $inc: {b:1}}

● Init● Prepare● Apply

{ $set: {c: {e:1} }}

● Log

● Log

Entry

Oplog

No results, upsert:true

Inse

rt

Entry

Update Flow Notes● Yields based on timing/in-memory● Each document update is atomic● Periodically allows journal to flush, if needed

Questions?Round 1: No, then we have more slides

2.6 Stuff● New modifiers● Improvements● Behavioral Changes

New Modifiers● Mul(tiply) existing field by constant: $mul{a:1} + {$mul:{a:10}} -> {a:10}● Min/Max of existing field w/constant: $min/$max{a:1} + {$min:{a:100}} -> {a:1} (no-op)● Store current date/ts in a field: $currentDate{} + {$currentDate:{ dt: true}} -> {dt: new Date()}

Improvements● $push

o $sort elements (works with numbers/strings/etc)o $sort and/or $slice (doesn’t require $sort for $slice)o $slice off either end (no middle yet)o $position lets you specify where in the arrayo $each, $sort, $slice no longer order dep.

● $bito xor support

Improvements● Error messages with context

o The field 'l' must be an array but is of type NumberLong in document {_id: ObjectId('52a0a61672a5f2f771099b54')}

o Cannot apply $inc to a value of non-integral type. {_id: ObjectId('52a0a61672a5f2f771099b54')} has the field 'd' of non-integral type Date

Improvements● No reordering of fields (except: _id is first)● Better building of doc during insert

o update({a:1, b:1}, {$inc:{c:1}}, 0,1) -> {a:1, b:1, c:1} o now uses parsed query tree to get equalities

● Will now return new _id for all types● Accurate tracking of no-ops, modified docs

Improvements● Validation

o Immutable Fields (_id, shard keys)o Storage restrictions (no $field, “.”, _id field types)o DBRef validation

● Can now query + set _id/shard-key fieldso update({_id:1}, {$set:{_id:1, ...}}) // oko update({_id:1}, {_id:2, ...}) // error

● Cleanup bad data ($unset/$rename etc)

Questions?No, well thanks for coming!