planning for the horizontal: scaling node.js applications
TRANSCRIPT
TheReddest [email protected]
PLANNING FOR THE HORIZONTAL
SCALING NODE.JS APPLICATIONS
Brandon Cannaday
Thursday, April 4, 13
ME
HOSTING, DATA, STATS FOR NODE.JS
modulus.io
Thursday, April 4, 13
WHEN TO SCALE?
1. RESPONSE TIMES
2. CPU USAGE
3. CONCURRENT CONNECTIONS
Thursday, April 4, 13
NODEFLY.COM
Thursday, April 4, 13
STARTING POINT
mydomain.com
SERVER
> NODE
Thursday, April 4, 13
NODE TWEAKS
http.globalAgent.maxSockets = Number.MAX_VALUE;
CONCURRENT OUTGOING CONNECTION LIMIT
Thursday, April 4, 13
HURTLE: THE SERVER
LINUX CONFIGURATION
1. FILE-MAX
2. SOMAXCONN
3. ULIMIT
Thursday, April 4, 13
FILE-MAX
SYSTEM FILE DESCRIPTOR LIMIT
1. Run sysctl -w fs.file-max=65535
2. Run sysctl -p
Thursday, April 4, 13
SOMAXCONN
SOCKET LISTEN QUEUE LENGTH
1. Run sysctl -w net.core.somaxconn=65535
2. Run sysctl -p
Thursday, April 4, 13
ULIMIT
PER PROCESS FILE DESCRIPTOR LIMIT
1. Edit /etc/security/limits.conf
2. Add the following:
* soft nofile 65535* hard nofile 65535root soft nofile 65535root hard nofile 65535
Thursday, April 4, 13
RUNNING SMOOTH
mydomain.com
SERVER
> NODE
Thursday, April 4, 13
HURTLE: THE CPU
BUY A BIGGER BOX
SERVER
> NODE
SERVER
> NODE
1 CORE
4 CORES
Thursday, April 4, 13
MULTICORE NODE
1 2 3 4
100%
CORE
USAGE
Thursday, April 4, 13
CLUSTER MODULE
SERVER
> NODE > NODE
> NODE > NODE
mydomain.com
Thursday, April 4, 13
CLUSTER EXAMPLE
var cluster = require('cluster');var http = require('http');var numCPUs = require('os').cpus().length;
if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); }}else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80);}
The Cluster Module
Thursday, April 4, 13
CLUSTER EXAMPLE
var cluster = require('cluster');var http = require('http');var numCPUs = require('os').cpus().length;
if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); }}else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80);}
Fork Children
Thursday, April 4, 13
CLUSTER EXAMPLE
var cluster = require('cluster');var http = require('http');var numCPUs = require('os').cpus().length;
if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); }}else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80);}
Handle Requests
Thursday, April 4, 13
CLUSTER LISTEN
WORKER
listen(...)
MASTER
Handle
Thursday, April 4, 13
ROLLING UPDATES
1. UPDATE SCRIPT
2. WORKER -> STOP LISTENING
3. KILL WORKER
4. CALL FORK() AGAIN
Thursday, April 4, 13
CLUSTER MODULE
SERVER
> NODE > NODE
> NODE > NODE
mydomain.com
Thursday, April 4, 13
HURTLE: SHARED STATE
SERVER
> NODE > NODE
> NODE > NODE
NO SHARED STATE
Thursday, April 4, 13
INSTALL REDIS
SERVER
> NODE > NODE
> NODE > NODE
REDIS
Thursday, April 4, 13
EXAMPLE 1: SESSION
var express = require('express'), app = express();
app.use(express.cookieParser());app.use(express.session({ secret: 'My Cookie Signing Secret'}));
app.get('/', function(req, res) { req.session.somekey = 'some value';});
MEMORY STORE
Thursday, April 4, 13
EXAMPLE 1: SESSION
var express = require('express'), RedisStore = require('connect-redis')(express), app = express();
app.use(express.cookieParser());app.use(express.session({ store: new RedisStore({ host: 'localhost', port: 6379 }), secret: 'My Cookie Signing Secret'}));
app.get('/', function(req, res) { req.session.somekey = 'some value';});
REDIS STORE
Thursday, April 4, 13
EXAMPLE 2: SOCKET.IO
var RedisStore = require('socket.io/lib/stores/redis') , redis = require('socket.io/node_modules/redis') , pub = redis.createClient() , sub = redis.createClient() , client = redis.createClient();
io.set('store', new RedisStore({ redisPub : pub, redisSub : sub, redisClient : client}));
https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO
Thursday, April 4, 13
RUNNING SMOOTH
SERVER
> NODE > NODE
> NODE > NODE
REDIS
mydomain.com
Thursday, April 4, 13
LAST HURTLE: HORIZONTAL
APP SERVER A
> NODE > NODE
> NODE > NODE
REDIS
APP SERVER B
> NODE > NODE
> NODE > NODE
REDIS
Thursday, April 4, 13
SEPARATE REDIS
APP SERVER A
> NODE > NODE
> NODE > NODE
APP SERVER B
> NODE > NODE
> NODE > NODE
REDIS
SERVER
Thursday, April 4, 13
LOAD BALANCING
APP SERVER A
> NODE > NODE
> NODE > NODE
APP SERVER B
> NODE > NODE
> NODE > NODE
REDIS
SERVER
LOAD BALANCER
SERVER
mydomain.com
Thursday, April 4, 13
LOAD BALANCING
1. MANAGED
2. INSTALL ONE
3. WRITE YOUR OWN
Thursday, April 4, 13
NGINX
http { upstream mydomain_com { server host1.mydomain.com:80; server host2.mydomain.com:80; } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } }}
LOAD BALANCER
SERVER
Thursday, April 4, 13
WRITE ONE
https://github.com/substack/bouncy
Thursday, April 4, 13
BOUNCY
bouncy modulevar bouncy = require('bouncy');
var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com'];
var count = 0;
var server = bouncy(function(req, res, bounce) {
count++; var host = hosts[count % hosts.length];
bounce(host, 80);
});
server.listen(80);
Thursday, April 4, 13
BOUNCY
Server collectionvar bouncy = require('bouncy');
var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com'];
var count = 0;
var server = bouncy(function(req, res, bounce) {
count++; var host = hosts[count % hosts.length];
bounce(host, 80);
});
server.listen(80);
Thursday, April 4, 13
BOUNCY
Create server
var bouncy = require('bouncy');
var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com'];
var count = 0;
var server = bouncy(function(req, res, bounce) {
count++; var host = hosts[count % hosts.length];
bounce(host, 80);
});
server.listen(80);
Thursday, April 4, 13
BOUNCY
Bounce request
var bouncy = require('bouncy');
var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com'];
var count = 0;
var server = bouncy(function(req, res, bounce) {
count++; var host = hosts[count % hosts.length];
bounce(host, 80);
});
server.listen(80);
Thursday, April 4, 13
AFFINITY
SESSION AFFINITYSTICKY SESSIONS
SEND THE SAME PERSON BACK TO THE SAME SERVER
Thursday, April 4, 13
NGINX AFFINITY
http { upstream mydomain_com { sticky; server host1.mydomain.com:80; server host2.mydomain.com:80; } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } }}
Thursday, April 4, 13
CUSTOM AFFINITY
req.headers['x-forwarded-for']
req.connection.remoteAddress
Thursday, April 4, 13
RUNNING SMOOTH
APP SERVER A
> NODE > NODE
> NODE > NODE
APP SERVER B
> NODE > NODE
> NODE > NODE
REDIS
SERVER
LOAD BALANCER
SERVER
mydomain.com
Thursday, April 4, 13
ROLLING UPDATES
1. REMOVE APP SERVER FROM LOAD BALANCER
2. UPGRADE APP SERVER
3. ADD BACK
4. REPEAT
Thursday, April 4, 13
SSL
TERMINATE EARLY
Thursday, April 4, 13
SSL
APP SERVER A
> NODE > NODE
> NODE > NODE
APP SERVER B
> NODE > NODE
> NODE > NODE
REDIS
SERVER
LB
SERVER
SSL SSL TERMINATOR
Thursday, April 4, 13
SSL
LB
SERVER
SSL
mydomain.com
80 443
Thursday, April 4, 13
STUD
https://github.com/bumptech/stud
frontend = [*]:443
backend = [127.0.0.1]:80
ssl = on
pem-file = "myCert.pem"
EXAMPLE CONFIG FILE
Thursday, April 4, 13
RUNNING SMOOTH W/SSL
APP SERVER A
> NODE > NODE
> NODE > NODE
APP SERVER B
> NODE > NODE
> NODE > NODE
REDIS
SERVER
LB
SERVER
SSLmydomain.com
Thursday, April 4, 13
HUGE
REDIS
SERVER
LB
SERVER
SSL LB
SERVER
SSL
Thursday, April 4, 13
DNS
ROUND-ROBIN DNSMULTIPLE RECORDS,
ONE DOMAIN
Thursday, April 4, 13
ROUND-ROBIN DNS
CLIENT 1 1. xxx.xxx.xxx.x2. xxx.xxx.xxx.y
CLIENT 2 1. xxx.xxx.xxx.y2. xxx.xxx.xxx.x
Thursday, April 4, 13
RUNNING SMOOTH
REDIS
SERVER
LB
SERVER
SSL LB
SERVER
SSL
Thursday, April 4, 13
BIG ENOUGH
SERVER
> NODE
Thursday, April 4, 13
BIG ENOUGH
SERVER
> NODE
Thursday, April 4, 13