javascript modules done right

95
JavaScript Modules Done right Client & server-side code organization meetjs · November 2011 · Warsaw, Poland

Upload: mariusz-nowak

Post on 11-May-2015

6.669 views

Category:

Technology


2 download

DESCRIPTION

JavaScript Modules Done RightClient & server-side code organization

TRANSCRIPT

Page 1: JavaScript Modules Done Right

JavaScript ModulesDone right

Client & server-side code organization

meetjs · November 2011 · Warsaw, Poland

Page 3: JavaScript Modules Done Right

Browser

In the beginning there was Browser.How we managed large code bases (?)

Page 4: JavaScript Modules Done Right

Browser

In the beginning there was Browser.How we managed large code bases (?)

• Files concatenation

Page 5: JavaScript Modules Done Right

Browser

Written usually this style:

MAIN.module = (function() { var privateVar = '..';

var privateMethod = function (args) { // ... };

return { publicProperty: '..', publicMethod: function () { // ... } };

}());

Page 6: JavaScript Modules Done Right

Browser

In the beginning there was Browser.How we managed large code bases (?)

• Files concatenation

Page 7: JavaScript Modules Done Right

Browser

In the beginning there was Browser.How we managed large code bases (?)

• Files concatenation• Module pattern

Page 8: JavaScript Modules Done Right

Server

Beginning of 2009

• Growing number of interpreters: Rhino, Spidermonkey, V8, JSCore

• No cross-interpreter standards

• There are programmers who want to write server-side JavaScript but want to have their code runnable on any engine

Page 9: JavaScript Modules Done Right

Server

Beginning of 2009

• Growing number of interpreters: Rhino, Spidermonkey, V8, JSCore

• No cross-interpreter standards

• There are programmers who want to write server-side JavaScript but want to have their code runnable on any engine

• There’s a need for standard library API that would allow JavaScript ecosystem grow (as it happened for Ruby, Python or Java)

Page 10: JavaScript Modules Done Right

Server

Beginning of 2009

• Growing number of interpreters: Rhino, Spidermonkey, V8, JSCore

• No cross-interpreter standards

• There are programmers who want to write server-side JavaScript but want to have their code runnable on any engine

• There’s a need for standard library API that would allow JavaScript ecosystem grow (as it happened for Ruby, Python or Java)

• ServerJS initiative is announced, renamed later into CommonJS -> http://www.blueskyonmars.com/2009/01/29/what-server-side-javascript-needs/

Page 11: JavaScript Modules Done Right

Server

CommonJS presents Modules specification -> http://www.commonjs.org/specs/modules/1.0/

Page 12: JavaScript Modules Done Right

Server

CommonJS presents Modules specification -> http://www.commonjs.org/specs/modules/1.0/

add.js

exports.add = function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

var add = require('add').add;exports.increment = function(val) { return add(val, 1);};

program.js

var inc = require('increment').increment;var a = 1;inc(a); // 2

Page 13: JavaScript Modules Done Right

Server

In about same time Ryan Dahl creates Node.js, which implements CommonJS Modules

add.js

exports.add = function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

var add = require('add').add;exports.increment = function(val) { return add(val, 1);};

program.js

var inc = require('increment').increment;var a = 1;inc(a); // 2

Page 14: JavaScript Modules Done Right

Server

In about same time Ryan Dahl creates Node.js, which implements CommonJS Modules... with some improvements

add.js

module.exports = function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

var add = require('./add');module.exports = function(val) { return add(val, 1);};

program.js

var inc = require('./increment');var a = 1;inc(a); // 2

Page 15: JavaScript Modules Done Right

Server

How module works:

// increment.js var add = require('./add'); module.exports = function (val) { return add(val, 1); };

Page 16: JavaScript Modules Done Right

Server

How module works:

var exports, module;function (exports, require, module) {

// increment.js var add = require('./add'); module.exports = function (val) { return add(val, 1); };

}.call(exports = {}, exports, function (path) { // import external module}, module = { exports: exports });

MODULE = module.exports;

Page 17: JavaScript Modules Done Right

Server

Let’s get back to module pattern.

Page 18: JavaScript Modules Done Right

Server

Let’s get back to module pattern.

module.js

MAIN.module = (function() { var privateVar = '..'; var privateMethod = function (args) { // ... }; return { publicProperty: '..', publicMethod: function () { // ... } };}());

Page 19: JavaScript Modules Done Right

Server

Module pattern as CommonJS module:

module.js

var privateVar = '..';var privateMethod = function (args) { // ...};

exports.publicProperty: '..',exports.publicMethod: function () { // ...};

program.jsvar foobar = require(‘./module’);

Page 20: JavaScript Modules Done Right

Server

Module pattern as CommonJS module:

module.js

var privateVar = '..';var privateMethod = function (args) { // ...};

exports.publicProperty: '..',exports.publicMethod: function () { // ...};

program.jsvar foobar = require(‘./module’); ↑ we decide locally under which name we’ll access imported module

Page 21: JavaScript Modules Done Right

Server

CommonJS Modules is indeed great specification.What are the benefits ?

Page 22: JavaScript Modules Done Right

Server

CommonJS Modules is indeed great specification.What are the benefits ?

First of all, cleanliness and encapsulation on highest available (for JavaScript) level

Page 23: JavaScript Modules Done Right

Server

CommonJS Modules is indeed great specification.What are the benefits ?

First of all, cleanliness and encapsulation on highest available (for JavaScript) level

• There’s no need to write obsolete function wrappers to work in private scope

Page 24: JavaScript Modules Done Right

Server

CommonJS Modules is indeed great specification.What are the benefits ?

First of all, cleanliness and encapsulation on highest available (for JavaScript) level

• There’s no need to write obsolete function wrappers to work in private scope

• We don’t have to deal with long namespaces, each required module is assigned to local variable

Page 25: JavaScript Modules Done Right

Server

CommonJS Modules is indeed great specification.What are the benefits ?

First of all, cleanliness and encapsulation on highest available (for JavaScript) level

• There’s no need to write obsolete function wrappers to work in private scope

• We don’t have to deal with long namespaces, each required module is assigned to local variable

• We can build large complex application without a need to touch global namespace

Page 26: JavaScript Modules Done Right

Server & Browser

How to run it in the Browser ?

Page 27: JavaScript Modules Done Right

Server & Browser

How to run it in the Browser ?

When I started working with CommonJS and Node.js (beginning 2011) I thought there’s solid solution already available

Page 28: JavaScript Modules Done Right

Server & Browser

How to run it in the Browser ?

When I started working with CommonJS and Node.js (beginning 2011) I thought there’s solid solution already available

I’ve found about 8-10 solutions that aimed at something similar, however none of them were really working right

Page 29: JavaScript Modules Done Right

Server & Browser

Main issue was that each of them, was trying to be something more than Node.js modules parser, and “more” seemed to have more attention than dependency parser I was after.

Page 30: JavaScript Modules Done Right

Server & Browser

Main issue was that each of them, was trying to be something more than Node.js modules parser, and “more” seemed to have more attention than dependency parser I was after.

Parsers were incomplete and buggy

Page 31: JavaScript Modules Done Right

Server & Browser

Anyway, time goes by, there are few solutions that deserve focus

Page 32: JavaScript Modules Done Right

Server & Browser

Anyway, time goes by, there are few solutions that deserve focus

Browserify -> https://github.com/substack/node-browserify

Page 33: JavaScript Modules Done Right

Server & Browser

Anyway, time goes by, there are few solutions that deserve focus

Browserify -> https://github.com/substack/node-browserify

Very interesting tool, if we’re after reusing native Node.js modules on client-side. Browserify struggles to make it possible, I guess it was main intention behind the project.

Page 34: JavaScript Modules Done Right

Server & Browser

Anyway, time goes by, there are few solutions that deserve focus

Browserify -> https://github.com/substack/node-browserify

Very interesting tool, if we’re after reusing native Node.js modules on client-side. Browserify struggles to make it possible, I guess it was main intention behind the project.

You need to add extra 320 lines of code for client-side to make it work, that’s not impressive, this code could be much smaller.

Page 35: JavaScript Modules Done Right

Server & Browser

Anyway, time goes by, there are few solutions that deserve focus

Browserify -> https://github.com/substack/node-browserify

Very interesting tool, if we’re after reusing native Node.js modules on client-side. Browserify struggles to make it possible, I guess it was main intention behind the project.

You need to add extra 320 lines of code for client-side to make it work, that’s not impressive, this code could be much smaller.

It doesn’t work well with some paths, modules from external packages were not found when I played with that (not sure if it’s bug or limitation)

Page 36: JavaScript Modules Done Right

Server & Browser

AMD -> Asynchronous Module Definition

Page 38: JavaScript Modules Done Right

Server & Browser

AMD ?

Page 39: JavaScript Modules Done Right

Server & Browser

AMD ?

It was made for browsers - Node.js won’t run AMD module properly

Page 40: JavaScript Modules Done Right

Server & Browser

AMD ?

It was made for browsers - Node.js won’t run AMD module properly

Page 41: JavaScript Modules Done Right

Server & Browser

AMD ?

It was made for browsers - Node.js won’t run AMD module properly

AMD has not much to do with CommonJS Modules, you need to write modules differently

Page 42: JavaScript Modules Done Right

Server & Browser

AMD ?

It was made for browsers - Node.js won’t run AMD module properly

AMD has not much to do with CommonJS Modules, you need to write modules differently

• All dependencies needs to be declared at begin of module

Page 43: JavaScript Modules Done Right

Server & Browser

AMD ?

It was made for browsers - Node.js won’t run AMD module properly

AMD has not much to do with CommonJS Modules, you need to write modules differently

• All dependencies needs to be declared at begin of module

In CommonJS we can reach for external modules in any place at any time, and it’s same for native modules that JavaScript will have in a future.

Page 44: JavaScript Modules Done Right

Server & Browser

AMD ?

It was made for browsers - Node.js won’t run AMD module properly

AMD has not much to do with CommonJS Modules, you need to write modules differently

• All dependencies needs to be declared at begin of module

In CommonJS we can reach for external modules in any place at any time, and it’s same for native modules that JavaScript will have in a future.

• Again, we need to use function wrappers. Each module needs to be declared with function call

Page 45: JavaScript Modules Done Right

Server & Browser

Why is that ?

Page 46: JavaScript Modules Done Right

Server & Browser

Why is that ?

AMD tries to be solution for two problems:

• Code modularization and organization• Dynamic dependency handling, so we load only that code that is actually used

Page 47: JavaScript Modules Done Right

Server & Browser

Why is that ?

AMD tries to be solution for two problems:

• Code modularization and organization• Dynamic dependency handling, so we load only that code that is actually used

Sounds promising, but is it worth it ?

Page 48: JavaScript Modules Done Right

Server & Browser

How AMD modules look like ?

Page 49: JavaScript Modules Done Right

Server & Browser

How AMD modules look like ?add.jsdefine(function () { return function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; };});

increment.jsdefine(['add'], function (add) { return function(val) { return add(val, 1); };});

program.jsdefine(['increment'], function (inc) { var a = 1; inc(a); // 2});

Page 50: JavaScript Modules Done Right

Server & Browser

Most popular implementation -> http://requirejs.org/requires us to load library made of 2000 lines of code.

Page 51: JavaScript Modules Done Right

Server & Browser

Most popular implementation -> http://requirejs.org/requires us to load library made of 2000 lines of code.

... it’s needed just to run code of our application.

Page 52: JavaScript Modules Done Right

Server & Browser

Most popular implementation -> http://requirejs.org/requires us to load library made of 2000 lines of code.

... it’s needed just to run code of our application.

... and still we cannot use our modules on server-side (Node.js) without extra compilation step.

Page 53: JavaScript Modules Done Right

Server & Browser

Ok, but what about dynamic dependency resolution, isn’t it main purpose of AMD ?

Page 54: JavaScript Modules Done Right

Server & Browser

Ok, but what about dynamic dependency resolution, isn’t it main purpose of AMD ?

In my opinion dynamic dependency resolution should be considered on application functionality level not on modules level

Page 55: JavaScript Modules Done Right

Server & Browser

Summary:

Page 56: JavaScript Modules Done Right

Server & Browser

Summary:

We miss light and clean CommonJS (precisely Node.js) Modules implementation, that would allow us to use them on client side

Page 57: JavaScript Modules Done Right

Server & Browser

Summary:

We miss light and clean CommonJS (precisely Node.js) Modules implementation, that would allow us to use them on client side

Implementation that wouldn’t try to be solution for other problems in first place.

Page 58: JavaScript Modules Done Right

Server & Browser

Summary:

We miss light and clean CommonJS (precisely Node.js) Modules implementation, that would allow us to use them on client side

Implementation that wouldn’t try to be solution for other problems in first place.

Code for handling modules on client side should be minimal, unnoticeable in size when compared to application code

Page 59: JavaScript Modules Done Right

Server & Browser

Finally new project was born:

Page 60: JavaScript Modules Done Right

Server & Browser

Finally new project was born:

modules-webmake ->https://github.com/medikoo/modules-webmake

npm install -g webmake

Page 61: JavaScript Modules Done Right

Modules Webmake

modules-webmake scans modules, builds dependency tree and produces output that it executable in the browser

Page 62: JavaScript Modules Done Right

Modules Webmake

modules-webmake scans modules, builds dependency tree and produces output that it executable in the browser

Current implementation is very basic, we might say, we just started

Page 63: JavaScript Modules Done Right

Modules Webmake

modules-webmake scans modules, builds dependency tree and produces output that it executable in the browser

Current implementation is very basic, we might say, we just started

It doesn’t limit us though from building complex applications made from hundreds of modules that originated from dozens of packages

Page 64: JavaScript Modules Done Right

Modules Webmake

modules-webmake scans modules, builds dependency tree and produces output that it executable in the browser

Current implementation is very basic, we might say, we just started

It doesn’t limit us though from building complex applications made from hundreds of modules that originated from dozens of packages

With modules-webmake you can build complex applications, right now

Page 65: JavaScript Modules Done Right

Modules WebmakeHow generated file looks like ?

Page 66: JavaScript Modules Done Right

Modules WebmakeHow generated file looks like ?(function (modules) { // 53 lines of import/export logic})({ "root": { "add.js": function (exports, module, require) { module.exports = function () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; }; }, "increment.js": function (exports, module, require) { var add = require('./add'); module.exports = function (val) { return add(val, 1); }; }, "program.js": function (exports, module, require) { var inc = require('./increment'); var a = 1; inc(a); // 2 } }})("root/program");

Page 67: JavaScript Modules Done Right

Modules WebmakeHow generated file looks like ?(function (modules) { // 53 lines of import/export logic})({ "root": { "add.js": function (exports, module, require) { module.exports = function () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; }; }, "increment.js": function (exports, module, require) { var add = require('./add'); module.exports = function (val) { return add(val, 1); }; }, "program.js": function (exports, module, require) { var inc = require('./increment'); var a = 1; inc(a); // 2 } }})("root/program"); <- Execution of module that initiates application

Page 68: JavaScript Modules Done Right

Modules Webmake• modules-webmake can be used both from shell and programmatically - we can bind it to http server in Node.js and generate files on demand

Page 69: JavaScript Modules Done Right

Modules Webmake• modules-webmake can be used both from shell and programmatically - we can bind it to http server in Node.js and generate files on demand

• Dependencies are parsed statically, it introduces some limitations, for specific cases we may force inclusion of chosen modules with ‘include’ option.

Page 70: JavaScript Modules Done Right

Modules Webmake• modules-webmake can be used both from shell and programmatically - we can bind it to http server in Node.js and generate files on demand

• Dependencies are parsed statically, it introduces some limitations, for specific cases we may force inclusion of chosen modules with ‘include’ option.

• modules-webmake reads local dependencies and those from external packages, one limitation is that it doesn’t recognize two different versions of same package (it will be addressed)

Page 71: JavaScript Modules Done Right

Modules Webmake• modules-webmake can be used both from shell and programmatically - we can bind it to http server in Node.js and generate files on demand

• Dependencies are parsed statically, it introduces some limitations, for specific cases we may force inclusion of chosen modules with ‘include’ option.

• modules-webmake reads local dependencies and those from external packages, one limitation is that it doesn’t recognize two different versions of same package (it will be addressed)

• Up to date documentation can be found at github project page:https://github.com/medikoo/modules-webmake

Page 72: JavaScript Modules Done Right

Modules WebmakeLast week I finished work on application that was built with help ofmodules-webmake.

• It’s HTML5 application, targetted only for modern browsers (newest FF, Chrome and Safari (both OSX & iOS))

• To support Offline mode we needed to put whole application logic to client side, including templates and simple database engine.

• Node.js on server-side, takes care about clients synchronization and saving data to physical database (mongodb)

• Minimalistic client-server communication based on sockets (Socket.IO)

Page 73: JavaScript Modules Done Right

Modules WebmakeFinal JavaScript file for client consists of 273 modules from 19 packages.

176 of mentioned modules works also on server side.It means that about 60% of client-side code is also running on server-side.

Concatenated file is 11 thousand lines longand weights about 370kB before minification and zipping.

It looks quite promising, if we take into account, that it consists of templates for whole application and all other modules that application is made of, including simple database engine.

After application load, all client-server communication is minimal as all pages as generated by client.

Page 74: JavaScript Modules Done Right

Modules WebmakeUpcoming improvements:

Page 75: JavaScript Modules Done Right

Modules WebmakeUpcoming improvements:

• Not only parser, but full modules engine. What for? It will allow to run and test modules with it’s dependencies in differently setup environments, like pure V8 + Modules, or emulation of browser environment.

Page 76: JavaScript Modules Done Right

Modules WebmakeUpcoming improvements:

• Not only parser, but full modules engine. What for? It will allow to run and test modules with it’s dependencies in differently setup environments, like pure V8 + Modules, or emulation of browser environment.

• Work in a background, it will allow much more optimized and faster file generation (watching file system for changes, immediate serving of generated files when integrated with node.js http server)

Page 77: JavaScript Modules Done Right

Modules WebmakeUpcoming improvements:

• Not only parser, but full modules engine. What for? It will allow to run and test modules with it’s dependencies in differently setup environments, like pure V8 + Modules, or emulation of browser environment.

• Work in a background, it will allow much more optimized and faster file generation (watching file system for changes, immediate serving of generated files when integrated with node.js http server)

• Optional minification, compilation

Page 78: JavaScript Modules Done Right

Modules WebmakeUpcoming improvements:

• Not only parser, but full modules engine. What for? It will allow to run and test modules with it’s dependencies in differently setup environments, like pure V8 + Modules, or emulation of browser environment.

• Work in a background, it will allow much more optimized and faster file generation (watching file system for changes, immediate serving of generated files when integrated with node.js http server)

• Optional minification, compilation

• Discovering modules that were required but not used

Page 79: JavaScript Modules Done Right

Modules WebmakeUpcoming improvements:

• Not only parser, but full modules engine. What for? It will allow to run and test modules with it’s dependencies in differently setup environments, like pure V8 + Modules, or emulation of browser environment.

• Work in a background, it will allow much more optimized and faster file generation (watching file system for changes, immediate serving of generated files when integrated with node.js http server)

• Optional minification, compilation

• Discovering modules that were required but not used

• Splitting into few files (for faster client-side load)

Page 80: JavaScript Modules Done Right

Modules WebmakeUpcoming improvements:

• Not only parser, but full modules engine. What for? It will allow to run and test modules with it’s dependencies in differently setup environments, like pure V8 + Modules, or emulation of browser environment.

• Work in a background, it will allow much more optimized and faster file generation (watching file system for changes, immediate serving of generated files when integrated with node.js http server)

• Optional minification, compilation

• Discovering modules that were required but not used

• Splitting into few files (for faster client-side load)

• Modules should not be duplicated in different files, introduction of intelligent more than one file generation

Page 81: JavaScript Modules Done Right

Modules WebmakeUpcoming improvements:

• Not only parser, but full modules engine. What for? It will allow to run and test modules with it’s dependencies in differently setup environments, like pure V8 + Modules, or emulation of browser environment.

• Work in a background, it will allow much more optimized and faster file generation (watching file system for changes, immediate serving of generated files when integrated with node.js http server)

• Optional minification, compilation

• Discovering modules that were required but not used

• Splitting into few files (for faster client-side load)

• Modules should not be duplicated in different files, introduction of intelligent more than one file generation

• Small code fixes for buggy engines e.g. quoting reserved keywords used as property names

Page 82: JavaScript Modules Done Right

Future: Harmony

We will have modules natively in JavaScript

http://wiki.ecmascript.org/doku.php?id=harmony:modules_examples

Page 83: JavaScript Modules Done Right

Future: Harmony

We will have modules natively in JavaScript

http://wiki.ecmascript.org/doku.php?id=harmony:modules_examples

Base concept of Harmony Modules is same as in CommonJS Modules.

Page 84: JavaScript Modules Done Right

Future: Harmony

We will have modules natively in JavaScript

http://wiki.ecmascript.org/doku.php?id=harmony:modules_examples

Base concept of Harmony Modules is same as in CommonJS Modules.

Differences lies in dedicated syntax, and powerful optional features like dynamic loading, possibility to load external modules via url etc.

Page 85: JavaScript Modules Done Right

Harmony Modules

Page 86: JavaScript Modules Done Right

Harmony ModulesWhat we currently write for Node.js:add.js

module.exports = function() { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

var add = require('./add');module.exports = function(val) { return add(val, 1);};

program.js

var inc = require('./increment');var a = 1;inc(a); // 2

Page 87: JavaScript Modules Done Right

Harmony ModulesWill be written that way with Harmony:add.js

export function add () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum;};

increment.js

import add from './add';export function increment (val) { return add(val, 1);};

program.js

import { increment: inc } from './increment';var a = 1;inc(a); // 2

Page 88: JavaScript Modules Done Right

Harmony ModulesModules Webmake for EcmaScript 3/5:(function (modules) { // 53 lines of import/export logic})({ "root": { "add.js": function (exports, module, require) { module.exports = function () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; }; }, "increment.js": function (exports, module, require) { var add = require('./add'); module.exports = function (val) { return add(val, 1); }; }, "program.js": function (exports, module, require) { var inc = require('./increment'); var a = 1; inc(a); // 2 } }})("root/program");

Page 89: JavaScript Modules Done Right

Harmony ModulesModules Webmake for Harmony:

module add { export function add () { var sum = 0, i = 0, args = arguments, l = args.length; while (i < l) sum += args[i++]; return sum; }; }; module increment { import add from add; export function increment (val) { return add(val, 1); }; }; module program { import { increment: inc } from increment; var a = 1; inc(a); // 2 };

import * from program;

Page 90: JavaScript Modules Done Right

Server & Browser

If you use CommonJS/Node.js Modules today, it won’t be problematic for you to switch to Harmony in few years time

Page 91: JavaScript Modules Done Right

Server & Browser

If you use CommonJS/Node.js Modules today, it won’t be problematic for you to switch to Harmony in few years time

Today:

CommonJS/Node.js Modules & modules-webmake

Page 92: JavaScript Modules Done Right

Server & Browser

If you use CommonJS/Node.js Modules today, it won’t be problematic for you to switch to Harmony in few years time

Today:

CommonJS/Node.js Modules & modules-webmake

Tomorrow:

Harmony Modules

Page 93: JavaScript Modules Done Right

Server & Browser

If you use CommonJS/Node.js Modules today, it won’t be problematic for you to switch to Harmony in few years time

Today:

CommonJS/Node.js Modules & modules-webmake

Tomorrow:

Harmony Modules (& modules-webmake ?)

Page 94: JavaScript Modules Done Right

Questions ?