Lessons in Open Source from the MongooseJS ODM

Valeri KarpovSoftware Engineer, MongoDBwww.thecodebarbarian.com

www.slideshare.net/vkarpov15github.com/vkarpov15

@code_barbarian

*

Who Am I?

•NodeJS Engineer at MongoDB

•Maintainer of mongoose ODM

•Former CTO, LevelUp

•Hacker in Residence at BookaLokal

•Author of Professional AngularJS

*

Talk Overview

•How I got into mongoose

•High-level overview of how mongoose works

•Recent new features

•What I’m working on now

•Lessons learned from my first year

*

What is Mongoose?

•MongoDB ODM for NodeJS

•Schema validation for schema-less database

•Neat syntax for building MongoDB operations

•Very active module:• 400k downloads/month

• 7000 GitHub stars

• 5-20 GitHub issue notifications/day

*

How I got into Mongoose

•Aug 2012: work with janky C++ event loop in HFT

•Nov 2012: start working on fashion startup

•Dec 2012: discover mongoose, cut SLOC in half

•Jan 2013: Leave HFT shop

•Apr 2013: Write original MEAN stack blog post

•Aug 2013: Leave startup land and go to MongoDB

*

How I got into Mongoose

•Dec 2013: Aaron Heckmann leaves MongoDB

•Apr 2014: Handoff over Twitter

*

How does Mongoose Work?

•Core features:• Schema validation

• Syntactic sugar for persisting documents

• Hooks

*

Part 1: Mongoose Schema Validation

•MongoDB is schema-less...

•But you don’t want end users to store arbitrary data

*

Mongoose Schema Validation

*

Part 2: Syntactic Sugar for Updates

•MongoDB updates look like this:

•

•Do I have to manually build the update part?

*

Object.defineProperty()

•Current JavaScript spec lets you hook into property access

*

Syntactic Sugar for Updates

•General idea: define mongoose schema

● When creating doc, loop over all schema paths● For each path, use Object.defineProperty()● Object.defineProperty() setter converts:

● Into:

● save() applies all updates

*

Part 3: Hooks / middleware

•Schema-level business logic

•Suppose you want to:• Add a timestamp every time you save()

• Add your own caching layer to queries

• Log every query based on config file setting

*

Hooks / middleware

•Schema-level business logic

*

Hooks => Plugins

•Higher-level concept of “plugins”

•External hooks

*

Recent Additions

•Mongoose 4.0 released March 25

•Sweet new features:• Query middleware

• Schema validation in the browser

• Validators on update()

*

What I’m Working On Now

•Promises: OOP for async operations

•Single biggest source of mongoose feature requests

*

Promises Explained

•General idea: users don’t like nested callbacks

•See Callback Hell is a Myth

*

Promises Explained

•Promises provide flatter structure

*

This Sounds Great But...

•The promises space is wildly fragmented

•Ask 10 users, get 10 different favorite promises libraries - Bluebird, Q, RSVP, When, native ES6

•Mongoose promises feature requests are usually• Use promises library X natively

• Support promises library Y’s feature Z

*

General Solution

•Mongoose 4 uses mpromise natively

•mpromise designed for ease rather than features or performance, minimal adoption outside of mongoose

•Considering deprecating native promise support

•For now, planned solution is to allow users to specify promise library of choice

*

Challenges

•Hooks and promises are confusing

•Promises need to be resolved

•Should they be resolved before or after post hooks?

•If after, hooks library needs to handle resolving!

*

Final Bit: Lessons Learned

•Linus’ Law:

•“Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix will be obvious to someone”

•Open source code is great because of the combined input of the entire user base

*

Final Bit: Lessons Learned

•When using mongoose, you get contributions from a lot of smart people

● The value of an OS module is (usually) proportional to its user base

*

Final Bit: Lessons Learned

•Putting your code in the open makes you go the extra mile to make it awesome

•And sometimes you have to eat some humble pie

*

First Rule in Running an Open Source Module

•Always be responsive

•Nothing is worse than opening a github issue and not having anybody touch it for a month

•Worse: somebody responds and says this library is no longer maintained

*

Second Rule: Lead with a Clear Vision

•Not necessarily in conflict with the “get people involved”

•Easy for a module to become bloated

•One-off contributors aren’t as invested in the module

•Leads to unclear interfaces

•For instance, yeoman Ionic generator

*

ABD: Always Be Dogfooding

•Why I work on BookaLokal, mongoose plugins, etc.

•“Nobody should start to undertake a large project. You start with a small trivial project, and you should never expect it to get large... If it doesn't solve some fairly immediate need, it's almost certainly over-designed... You need to get something half-way useful first, and then others will say "hey, that almost works for me", and they'll get involved in the project.” - Linus Torvalds

*

Thanks for Listening!

•Slides on:• Twitter: @code_barbarian

• Slideshare: slideshare.net/vkarpov15

•Mongoose on Github