node.js in a heterogeneous system
DESCRIPTION
TRANSCRIPT
Node.js !in a heterogeneous system
Alexey Migutsky : Senior Software Developer @ Lohika
What do we do?�
What do we do?�PaaS for BigData analytics in cloud environments.
Ubix.io
What do we use?�
What do we use?�
Scala
Python
Bash
Javascript
What do we use?�
BDAS (Spark + Shark)
AWS
Black Voodoo Magic™
Velocipede™
Brains
What’s next?�
What’s next?�
Node & Maven
CI
Velocipede™
Jenkins & Node testing
?�
Why Node.js?
Why Node.js? Fast and easy Easy websockets support Plays nice with AWS We use it as a web-tier orchestrator
Node & Maven
Node & Maven We use maven (pom.xml) We use npm (packege.json) We have node+maven projects!
Project structure
Project structure + Subprojects are maven modules + Subprojects use common code base + Easy « build process » + Easy deployments - Uber root package.json - Mangled dependencies
Custom modules
Custom modules Live in root /node_modules right now Planning to move to separate git repos Tested with subprojects
Tests
Tests Mocha + Sinon + Chai Sinon-chai + mocha-sinon Chai-as-promised Rewired Jenkins TAP plugin
Test example describe('##getConfiguration', function () {! it('should return promise and resolve with data ', function (done) {! var promise = sysStore.getConfiguration('somekey');! promise.should.eventually.become({key: 'somekey'}) !
" .then(function () {! mockDb.getItem.should.be.calledWithMatch( !
! mockTables.system, !" " {key: sinon.match.string} !" "); !
}) ! .should.notify(done);! });! });
Test Env How-to
Test Env How-to Goals: - run from root using mvn test - run from sub using npm test - human-readable result logs - result >> in console and file - Jenkins support
http://www.mindmeister.com/346337860
package.json
"devDependencies": { "mocha": "*", "sinon": "1.6.*", "chai": "*", "sinon-chai": "*", "mocha-sinon": "*", "chai-as-promised" : "*", "rewire" : "*" }, "scripts" : { "test" : "./runtest.sh" }
Test Env How-to
runtest.sh
#!/bin/bash!mocha $(find test -name '*.js') -R tap | !tee test/testresults.tap; (exit ${PIPESTATUS[0]})
Test Env How-to
The whole command runs in a single line. Wrapped here for display purpose J
pom.xml (maven-antrun-plugin)
<execution> ! <id>test-nodejs</id> ! <phase>test</phase> ! <configuration> ! <tasks name="Run mocha tests"> ! <exec dir="${basedir}”! executable="npm” ! failonerror="true"> ! <arg value="test"/>! </exec> ! </tasks> ! </configuration> ! <goals> ! <goal>run</goal> ! </goals> !</execution>
Test Env How-to
Build process
Build process
- No npm install on build server (too slooow)
- Fixed deps versions (update by-hand)
- All deployable deps are checked in!
- Build phase only copies deps in subprojects
http://www.futurealoof.com/posts/nodemodules-in-git.html
Deployment
Deployment
- Builds are stored at S3
- Copied to instances by orchestration tools
- Managed by upstart
upstart (server.conf) #this should be put in /etc/init/server.conf on Ubuntu !description ”Node.js server" !author "Alexey Migutsky" !!# used to be: start on startup !# until we found some mounts weren't ready yet while booting: !start on started mountall !stop on shutdown !!# Automatically Respawn: !respawn !respawn limit 20 5 !!# Max open files are @ 1024 by default. Bit few. !limit nofile 32768 32768 !!script !
"# Not sure why $HOME is needed, but we found that it is: "!"export HOME="/root” !"exec /usr/local/bin/node /usr/local/server/app.js >> /var/log/server-err.log 2>&1 !
end script !!post-start script !end script
Deployment
http://kvz.io/blog/2009/12/15/run-nodejs-as-a-service-on-ubuntu-karmic/
Deployment hint
Have a look at http://pm2.io/
Continuous Integration
« hack some node.js » npm test
push code
push code
push code
build & test
push code
build & test
store
push code
build & test
store
push code
build & test
store
push code
build & test
store
deploy
That’s it!
That’s it!
Thanks
@mr_mig_by bit.ly/mr_mig