truly madly deeply parallel ruby applications

Truly Madly Deeply Parallel Ruby (Web)* Applications

@harikrishnan83

* Thanks @headius

How many of you are building web applications?

How many of you have more than 1 user using it at

a time?

How many of you use scaling out as the way hit scale?

How many of you are scaling up to hit scale?

What is a parallel environment?

Parallelism

ConcurrencyThread A - load

Thread B - load

Thread A - increment

Thread B - increment

Thread A - save

Thread B - save

i = 0

i = 0

i = 1

i = 1

Ruby web application deployment

Process Parallelism

Reverse Proxy Unicorn Processes Database Layer

On my current project

We only use EC2 small instances

Because it is very hard to utilize a high spec machine

Process Context Switch is Expensive

Today...

● We have 4 small EC2 instances

● 2 Puma processes run on each

● Handles about 100,000 requests per hour

● And this is our Private alpha

We need to...

● Handle about 1 million requests per hour

● Which means about 40-45 EC2 small instances

This is not trivial

● Costs a lot of money

● Lot of time required to maintain these boxes

● Being elastic will become very important

● Cost also in terms of more Devops time

In general

It is easier to baby sit few boxes

Than a lot!

Ideally, we would like to both scale up and scale out

i.e. we want to achieve the same throughput with, say,

just 5 large instances

Enter thread based parallelism

Why were we not doing this till now?

Threads are hard*They share memory and mutate thingsThey share memory and mutate things

* - Supposedly

And there is the ubiquitous issue

‘Thread Safety’

Before we go there, first lets look at some code

The real question is...

Are you“Safe for Parallelization”

Understanding this will take you a long way in “getting parallel”

Things to remember while moving to threaded parallelism

#1 - Always identify the shared resources

Shared Resource

● Objects

● DB rows

● Caches

● Log files!

#2 - Bank on thread safe libraries

Libraries

● Data structures

● JSON, XML parsing, HTTP clients etc

● Generally, auditing all the gems you use for thread safety is a good idea

If you only use thread safe libraries are you ‘safe for parallelism’?

Rails is thread safe right?

Why is everyone concerned about thread safety in the first place?

#3 - If two libraries are thread safe, code that uses both of them need not be

Rails thread safety model

● Instantiate everything for every request

● No shared state (global objects)

● Different from, say, Java (single servlet object per container, IOC with singletons etc.)

#4 - Try and stick to Rails’ way of handling requests

Are you ‘Safe for parallelism’ if you follow these steps?

Well, it depends...

Validating, say, through a green bar is very hard.

Always give yourself some time to stabilize.

The move is definitely not overnight!

Speaking of the move, move where?

Since Rubinius is mostly MRI like, its simpler

I personally love JRuby more because of my JVM

background

Lots of good things have been spoken about JRuby

Some gotchas based on my experience

JRuby impacts Developers

● The JRuby startup time (mostly because of the JVM startup time) can sometimes kill red-green cycle time

● Sometimes, you should be OK with stooping down to Java code to figure out why something is not working

JRuby impacts OPs

● You no longer have a ruby app in prod, its a Java app

● GC tuning, Process monitoring, Profiling etc. are very different on a JVM

Thread ParallelismReverse Proxy Puma Instance Database LayerThreads

Thank you!@harikrishnan83