javascript modules done right

JavaScript ModulesDone right

Client & server-side code organization

meetjs · November 2011 · Warsaw, Poland

Mariusz Nowakmariusz@medikoo.com

@medikoo

github.com/medikoo

JavaScript Programmer at Roche Ltd.

Browser

In the beginning there was Browser.How we managed large code bases (?)

Browser

• Files concatenation

Browser

Written usually this style:

MAIN.module = (function() { var privateVar = '..';

var privateMethod = function (args) { // ... };

return { publicProperty: '..', publicMethod: function () { // ... } };

Browser

• Files concatenation

Browser

• Files concatenation• Module pattern

Server

Beginning of 2009

• Growing number of interpreters: Rhino, Spidermonkey, V8, JSCore

• No cross-interpreter standards

• There are programmers who want to write server-side JavaScript but want to have their code runnable on any engine

Server

Beginning of 2009

• There’s a need for standard library API that would allow JavaScript ecosystem grow (as it happened for Ruby, Python or Java)

Server

Beginning of 2009

• There’s a need for standard library API that would allow JavaScript ecosystem grow (as it happened for Ruby, Python or Java)

• ServerJS initiative is announced, renamed later into CommonJS -> http://www.blueskyonmars.com/2009/01/29/what-server-side-javascript-needs/

Server

CommonJS presents Modules specification -> http://www.commonjs.org/specs/modules/1.0/

Server

CommonJS presents Modules specification -> http://www.commonjs.org/specs/modules/1.0/

add.js

exports.add = function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

var add = require('add').add;exports.increment = function(val) { return add(val, 1);};

program.js

var inc = require('increment').increment;var a = 1;inc(a); // 2

Server

In about same time Ryan Dahl creates Node.js, which implements CommonJS Modules

add.js

exports.add = function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

var add = require('add').add;exports.increment = function(val) { return add(val, 1);};

program.js

var inc = require('increment').increment;var a = 1;inc(a); // 2

Server

In about same time Ryan Dahl creates Node.js, which implements CommonJS Modules... with some improvements

add.js

module.exports = function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

var add = require('./add');module.exports = function(val) { return add(val, 1);};

program.js

var inc = require('./increment');var a = 1;inc(a); // 2

Server

How module works:

// increment.js var add = require('./add'); module.exports = function (val) { return add(val, 1); };

Server

How module works:

var exports, module;function (exports, require, module) {

// increment.js var add = require('./add'); module.exports = function (val) { return add(val, 1); };

}.call(exports = {}, exports, function (path) { // import external module}, module = { exports: exports });

MODULE = module.exports;

Server

Let’s get back to module pattern.

Server

Let’s get back to module pattern.

module.js

MAIN.module = (function() { var privateVar = '..'; var privateMethod = function (args) { // ... }; return { publicProperty: '..', publicMethod: function () { // ... } };}());

Server

Module pattern as CommonJS module:

module.js

var privateVar = '..';var privateMethod = function (args) { // ...};

exports.publicProperty: '..',exports.publicMethod: function () { // ...};

program.jsvar foobar = require(‘./module’);

Server

Module pattern as CommonJS module:

module.js

var privateVar = '..';var privateMethod = function (args) { // ...};

exports.publicProperty: '..',exports.publicMethod: function () { // ...};

program.jsvar foobar = require(‘./module’); ↑ we decide locally under which name we’ll access imported module

Server

CommonJS Modules is indeed great specification.What are the benefits ?

Server

First of all, cleanliness and encapsulation on highest available (for JavaScript) level

Server

• There’s no need to write obsolete function wrappers to work in private scope

Server

• We don’t have to deal with long namespaces, each required module is assigned to local variable

Server

• We don’t have to deal with long namespaces, each required module is assigned to local variable

• We can build large complex application without a need to touch global namespace

Server & Browser

How to run it in the Browser ?

Server & Browser

When I started working with CommonJS and Node.js (beginning 2011) I thought there’s solid solution already available

Server & Browser

When I started working with CommonJS and Node.js (beginning 2011) I thought there’s solid solution already available

I’ve found about 8-10 solutions that aimed at something similar, however none of them were really working right

Server & Browser

Main issue was that each of them, was trying to be something more than Node.js modules parser, and “more” seemed to have more attention than dependency parser I was after.

Server & Browser

Main issue was that each of them, was trying to be something more than Node.js modules parser, and “more” seemed to have more attention than dependency parser I was after.

Parsers were incomplete and buggy

Server & Browser

Anyway, time goes by, there are few solutions that deserve focus

Server & Browser

Browserify -> https://github.com/substack/node-browserify

Server & Browser

Very interesting tool, if we’re after reusing native Node.js modules on client-side. Browserify struggles to make it possible, I guess it was main intention behind the project.

Server & Browser

You need to add extra 320 lines of code for client-side to make it work, that’s not impressive, this code could be much smaller.

Server & Browser

You need to add extra 320 lines of code for client-side to make it work, that’s not impressive, this code could be much smaller.

It doesn’t work well with some paths, modules from external packages were not found when I played with that (not sure if it’s bug or limitation)

Server & Browser

AMD -> Asynchronous Module Definition

Server & Browser

AMD -> Asynchronous Module Definition

It gained some attention lately:

http://addyosmani.com/writing-modular-js/

http://blog.millermedeiros.com/2011/09/amd-is-better-for-the-web-than-commonjs-modules/

http://unscriptable.com/index.php/2011/09/30/amd-versus-cjs-whats-the-best-format/

Server & Browser

It was made for browsers - Node.js won’t run AMD module properly

Server & Browser

AMD has not much to do with CommonJS Modules, you need to write modules differently

Server & Browser

• All dependencies needs to be declared at begin of module

Server & Browser

In CommonJS we can reach for external modules in any place at any time, and it’s same for native modules that JavaScript will have in a future.

Server & Browser

In CommonJS we can reach for external modules in any place at any time, and it’s same for native modules that JavaScript will have in a future.

• Again, we need to use function wrappers. Each module needs to be declared with function call

Server & Browser

Why is that ?

Server & Browser

Why is that ?

AMD tries to be solution for two problems:

• Code modularization and organization• Dynamic dependency handling, so we load only that code that is actually used

Server & Browser

Why is that ?

AMD tries to be solution for two problems:

• Code modularization and organization• Dynamic dependency handling, so we load only that code that is actually used

Sounds promising, but is it worth it ?

Server & Browser

How AMD modules look like ?

Server & Browser

How AMD modules look like ?add.jsdefine(function () { return function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; };});

increment.jsdefine(['add'], function (add) { return function(val) { return add(val, 1); };});

program.jsdefine(['increment'], function (inc) { var a = 1; inc(a); // 2});

Server & Browser

Most popular implementation -> http://requirejs.org/requires us to load library made of 2000 lines of code.

Server & Browser

... it’s needed just to run code of our application.

Server & Browser

... it’s needed just to run code of our application.

... and still we cannot use our modules on server-side (Node.js) without extra compilation step.

Server & Browser

Ok, but what about dynamic dependency resolution, isn’t it main purpose of AMD ?

Server & Browser

Ok, but what about dynamic dependency resolution, isn’t it main purpose of AMD ?

In my opinion dynamic dependency resolution should be considered on application functionality level not on modules level

Server & Browser

Summary:

Server & Browser

Summary:

We miss light and clean CommonJS (precisely Node.js) Modules implementation, that would allow us to use them on client side

Server & Browser

Summary:

Implementation that wouldn’t try to be solution for other problems in first place.

Server & Browser

Summary:

Implementation that wouldn’t try to be solution for other problems in first place.

Code for handling modules on client side should be minimal, unnoticeable in size when compared to application code

Server & Browser

Finally new project was born:

Server & Browser

Finally new project was born:

modules-webmake ->https://github.com/medikoo/modules-webmake

npm install -g webmake

Modules Webmake

modules-webmake scans modules, builds dependency tree and produces output that it executable in the browser

Modules Webmake

Current implementation is very basic, we might say, we just started

Modules Webmake

It doesn’t limit us though from building complex applications made from hundreds of modules that originated from dozens of packages

Modules Webmake

It doesn’t limit us though from building complex applications made from hundreds of modules that originated from dozens of packages

With modules-webmake you can build complex applications, right now

Modules WebmakeHow generated file looks like ?

Modules WebmakeHow generated file looks like ?(function (modules) { // 53 lines of import/export logic})({ "root": { "add.js": function (exports, module, require) { module.exports = function () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; }; }, "increment.js": function (exports, module, require) { var add = require('./add'); module.exports = function (val) { return add(val, 1); }; }, "program.js": function (exports, module, require) { var inc = require('./increment'); var a = 1; inc(a); // 2 } }})("root/program");

Modules WebmakeHow generated file looks like ?(function (modules) { // 53 lines of import/export logic})({ "root": { "add.js": function (exports, module, require) { module.exports = function () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; }; }, "increment.js": function (exports, module, require) { var add = require('./add'); module.exports = function (val) { return add(val, 1); }; }, "program.js": function (exports, module, require) { var inc = require('./increment'); var a = 1; inc(a); // 2 } }})("root/program"); <- Execution of module that initiates application

Modules Webmake• modules-webmake can be used both from shell and programmatically - we can bind it to http server in Node.js and generate files on demand

• Dependencies are parsed statically, it introduces some limitations, for specific cases we may force inclusion of chosen modules with ‘include’ option.

• modules-webmake reads local dependencies and those from external packages, one limitation is that it doesn’t recognize two different versions of same package (it will be addressed)

• Up to date documentation can be found at github project page:https://github.com/medikoo/modules-webmake

Modules WebmakeLast week I finished work on application that was built with help ofmodules-webmake.

• It’s HTML5 application, targetted only for modern browsers (newest FF, Chrome and Safari (both OSX & iOS))

• To support Offline mode we needed to put whole application logic to client side, including templates and simple database engine.

• Node.js on server-side, takes care about clients synchronization and saving data to physical database (mongodb)

• Minimalistic client-server communication based on sockets (Socket.IO)

Modules WebmakeFinal JavaScript file for client consists of 273 modules from 19 packages.

176 of mentioned modules works also on server side.It means that about 60% of client-side code is also running on server-side.

Concatenated file is 11 thousand lines longand weights about 370kB before minification and zipping.

It looks quite promising, if we take into account, that it consists of templates for whole application and all other modules that application is made of, including simple database engine.

After application load, all client-server communication is minimal as all pages as generated by client.

Modules WebmakeUpcoming improvements:

• Not only parser, but full modules engine. What for? It will allow to run and test modules with it’s dependencies in differently setup environments, like pure V8 + Modules, or emulation of browser environment.

• Work in a background, it will allow much more optimized and faster file generation (watching file system for changes, immediate serving of generated files when integrated with node.js http server)

• Optional minification, compilation

• Discovering modules that were required but not used

• Splitting into few files (for faster client-side load)

• Modules should not be duplicated in different files, introduction of intelligent more than one file generation

• Small code fixes for buggy engines e.g. quoting reserved keywords used as property names

Future: Harmony

We will have modules natively in JavaScript

http://wiki.ecmascript.org/doku.php?id=harmony:modules_examples

Future: Harmony

Base concept of Harmony Modules is same as in CommonJS Modules.

Future: Harmony

Base concept of Harmony Modules is same as in CommonJS Modules.

Differences lies in dedicated syntax, and powerful optional features like dynamic loading, possibility to load external modules via url etc.

Harmony Modules

Harmony ModulesWhat we currently write for Node.js:add.js

module.exports = function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

var add = require('./add');module.exports = function(val) { return add(val, 1);};

program.js

var inc = require('./increment');var a = 1;inc(a); // 2

Harmony ModulesWill be written that way with Harmony:add.js

export function add () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

import add from './add';export function increment (val) { return add(val, 1);};

program.js

import { increment: inc } from './increment';var a = 1;inc(a); // 2

Harmony ModulesModules Webmake for EcmaScript 3/5:(function (modules) { // 53 lines of import/export logic})({ "root": { "add.js": function (exports, module, require) { module.exports = function () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; }; }, "increment.js": function (exports, module, require) { var add = require('./add'); module.exports = function (val) { return add(val, 1); }; }, "program.js": function (exports, module, require) { var inc = require('./increment'); var a = 1; inc(a); // 2 } }})("root/program");

Harmony ModulesModules Webmake for Harmony:

module add { export function add () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; }; }; module increment { import add from add; export function increment (val) { return add(val, 1); }; }; module program { import { increment: inc } from increment; var a = 1; inc(a); // 2 };

import * from program;

Server & Browser

If you use CommonJS/Node.js Modules today, it won’t be problematic for you to switch to Harmony in few years time

Server & Browser

Today:

CommonJS/Node.js Modules & modules-webmake

Server & Browser

Today:

Tomorrow:

Harmony Modules

Server & Browser

Today:

Tomorrow:

Harmony Modules (& modules-webmake ?)

Questions ?

Thank You !

Mariusz Nowakmariusz@medikoo.com

@medikoo

github.com/medikoo

javascript modules done right

function args

var exports

serverhow module

required module

external module

function val

var sum

function path

Technology

things you can do in javascript es6 that can't be done in...

javascript the good and the bad. caveat we are not studying...

javascript modules and patterns private fields, module,...

write powerful javascript modules to make your apps dry...

modules and assembling of javascript (in russian)

javascript javascript introduction. q. what is javascript?...

rob chohan: titanium+plus modules and javascript bridges:...

javascript done right

javascript for ocf applications - open connectivity … ·...

what is javascript? embedding javascript with html...

javascript development done right

javascript, done right

javascript awesomeness: javascript for beginners

essential javascript -- a javascript tutorial

release 1.0.1 tomasz grajewski - read the docs ·...

2001 deitel & associates, inc. all rights reserved. 1...

2000 deitel & associates, inc. all rights reserved. chapter...

javascript! that's sick! modules, amd, require.js

done by: hanadi muhsen1 tutorial 1. learn the history of...

sams teach yourself angularjs, javascript, and jquery all...