SCARLETscalable cloud enabled JIRA

Sanne Grinovero, Sourcesense

What is Scarlet

Scarlet is a custom edition of Atlassian JIRA for demanding customers, to scale both vertically and horizontally.

It solves the limitation of standalone JIRA to be deployed on a single node, applies some performance tuning, enables dynamic cluster resizing with automatic node discovery and failover.

What Scarlet isn't

It's not a JIRA plugin

Doesn't need a cloud

Can scale down to one or zero nodes without data loss

Doesn't cost more than JIRA

Why did we make it?

Some customers need failover for high availability

Having a single node makes maintenance harder

Sometimes vertical scaling is not an optionor is too expensive

We are cool partners

Why did we make it?

British Telecom, UK:25,000 users50 million issues with complex workflows

Another telecom, in Italy:

140 projects100 issues added a day1,000 userscomplex workflows:

issues live in the system for many months

From the technical point of view, it is..It is a source code patch to JIRA:

Enables caching with InfinispanExtreme tuning at Lucene levelRemoves the design barriers to multi-node clustering

It is a complex build systemautomated JIRA repackagingshort time reaction to new JIRA releases

cross-technology virtual appliances builds

Designed to ease deploy and maintenance

Caching with

Infinispan is an extremely scalable, highly available data grid platform - 100% open source, and written in Java.

www.infinispan.org JIRA uses several dozens of special-purpose internal caches; almost all need a java.util.concurrent.ConcurrentMap, which is implemented by Infinispan

Scarlet detects and enables Infinispan on each of JIRA's cachesEach cache can be configured independentlyInfinispan uses state-of-the-art algorithms for maximum efficiency (such as Low Inter-reference Recency Set - LIRS eviction)Scarlet uses Infinispan' support for cache distribution, invalidation and replication for multi-node deployments.

tuning & new strategies

JIRA's usage of Lucene needs to be flexible for all kind of customers: low, medium and high number of concurrent users. Scarlet takes advantage of some of the latest Lucene innovations to reach for peak performancescaling vertically:

shared buffers across index reopeningpooled IndexSearchersin-memory transactional writesenables more low-level tuning patterns

and scaling horizontally:

Index stored in replicated/distributed memory (Infinispan)

Lucene index stored in Infinispan

Infinispan: shared state, cache, index

Concerns with clustering

Internal cachesreplication, distribution or invalidation with Infinispan

Internal state (configuration settings,...)

replication

IndexesInfinispan distributed or replicated

File uploads (issue attachments)

use a shared filesystemin future Scarlet might store files using Infinispan's GridFS

http://www.infoq.com/articles/infinispan-gridfs

Concerns with clustering

Session replicationnot implemented yet: in case the node you're connected to is killed you'll have to login again.

Load balancer

any load balancer should workmod_cluster is recommended when using Apache Httpd

Dynamic configuration of httpd workersServer-side load balance factor calculationhttp://www.jboss.org/mod_cluster

Scarlet with mod_cluster

On a cloud...

Node automatic discovery might be problematicInfinispan (JGroups) has several ways to do it

start with a list of known IPs (TCPPING)use S3 (S3PING)setup a GossipRouter

We test Scarlet using RDS as database and index store (Amazon's MySQL as-a-service), S3PING for node autodiscovery:

1. provide EC2 credentials2. the build system produces and registers a

preconfigured custom AMI 3. start many of them!

On a cloud...

Your nodes are "stateless", all equal:no important data on them

no need to backup the volatile node data you can kill some if you want to scale backyou can start new clones to scale up

All your important data is stored in a database

or S3 a single object to backup and manage

We have build targets to provide appliances for many virtualization environments (VMWare, Xen, KVM, ...)

Using BoxGrinder: http://www.jboss.org/boxgrinder.html

Ease of managementNo "special" nodes needed

no master node, no passive master

the Scarlet package installs exactly the same way as standalone JIRA:

uncompress it configure a datasourceconfigure the jira_home parameter

until we implement GridFS toooptionally, do some tuning

plenty of more options

All nodes activated in the same network will form a clusteryou might need to configure firewall and network

Upgrading to Scarlet from JIRA

1. replace the package, keep configuration files2. start it!

previous indexes are detected and imported in Infinispana new table is added to your DB.

From Scarlet to JIRA1. replace the package2. delete the extra table 3. perform re-indexing

Scarlet just adds a single table to your existing schema.Nothing else is changed in the database, and there's no need to change any configuration files either!

Plugin compatibility

With the ones tested so far no changes where needed, still many plugins are untested and might need fixing.

Greenhopper works fineFisheye works fine

To plugin developers:

avoid to make your own cacheuse JIRA's servicesuse JIRA's cache factories

we are available for helpif you can show us the code we can help even better

How to get it?

The package is free of charge for beta testers pay for setup, tuning and testing time

atlassian@sourcesense.com

www.sourcesense.com/en/events/scarlet.html

scarlet.sourceforge.net

Combined best of breed OSS technologies