lessons learned at wakoopa

Lessons Learnedwhile building Wakoopa

Menno van der SmanDeveloper at Wakoopa

since May 2007

Wakoopa is about software

MacPC

200million

Does it scale?

Scaling the front is easy

Scaling the back is harder

Limits spawn creativity

One server

at Railsmachinesliced into staging, production and db

MySQL 4.1Apache + Mongrel

Background queue

Lots of tools already

BackgroundRB, DelayedJob, Daemons etc

Rolling our own

database basedusing script/runner

processing multiple items in one loopone cache update per loop

Smart queries

INSERT INTO hourly_usage (...) VALUES (...) ON DUPLICATE KEY UPDATE active_seconds = active_seconds + ..., idle_seconds = idle_seconds + ...

UPDATE developers SET active_seconds = active_seconds + ...

Allows for concurrency

Size = rowsize x rows

Make rowsize smaller

sometimes `id` shouldn’t be the

Primary Key

Enter composite_primary_keys

Overrides default PK

Original idea by Dr NicEnhanced for Rail 2.1 by Darrin Holsthttp://compositekeys.rubyforge.org/

class HourlyUsage < ActiveRecord::Base set_primary_keys :user_id, :software_id, :used_at ...end

Archive old datathat’s not frequently used

HourlyUsageto

HourlyUsageStorage

SELECT ... FROM hourly_usage WHERE used_at < '...' INTO OUTFILE '/path/to/file'

LOAD DATA INFILE '/path/to/file' REPLACE INTO TABLE hourly_usage_storage

avoids long locks

Upcoming improvements

MySQL 5.1Partitioning

CREATE TABLE hourly_usage_storage ( ... )PARTITION BY RANGE (software_id) ( PARTITION p0 VALUES LESS THAN (...), PARTITION p1 VALUES LESS THAN (...), PARTITION p2 VALUES LESS THAN (...), PARTITION p3 VALUES LESS THAN (...), .... PARTITION pn VALUES LESS THAN MAXVALUE);

Available in MySQL 5.1

Makes ‘WHERE software_id = ...’ very fast

And many more plans

MySQL 5.1Memcached

SphinxRails 2.1

Amazon EC2

Designing a new infrastructure

Announcing a new blog

Inside Wakoopahttp://inside.wakoopa.com

Questions?

http://wakoopa.com/jobs

Looking for C++ and frontend awesomeness

or suggestions?