habits of highly scalable web applications

Habits of Highly Scalable Web Applications

Eli WhiteZend

http://eliw.com/

Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009

Scaling?

Enabling your application to grow as traffic grows

Only do what you need but …Don't code yourself into a corner


Various aspects

Web ServerDatabaseCaching


In the Beginning …

One Web Server:


Load Balanced


Numerous Options

DNS RotationApache ProxyCloud Services

NetScaler

BIG-IP


Preparation

Ensure you don't preclude this, for example:If using local caching (APC / Zend Server)

Avoid assuming exclusive/single cacheDon't rely on the filesystem

Temporary files, Sessions, etc

If you do have code assuming one server: Encapsulate


Database Scaling

Everyone starts with just one server:

Multiple steps to take as you move forward


Step One: Master/Slave


Master/Slave Preparation

Even with one server:Make code write to master and read from slave


Avoid Slave Lag

Don't write code that would fail with slave lag:

$master->query('update users set comments += 1'); $slave->query('select comments from users');


Step Two: Multiple Slaves


One Slave per Web Server?

Not as flexible


Better Solution: Random


Code to Select Random Slaveclass DB { private static $cfg = array( 'write' => array('mysql:dbname=MyDB;host=10.1.2.3'), 'read' => array('mysql:dbname=MyDB;host=10.1.2.7', 'mysql:dbname=MyDB;host=10.1.2.8', 'mysql:dbname=MyDB;host=10.1.2.9'); );

public static function getConnection($pool) { $max = count(self::$cfg[$pool]) - 1; $dsn = self::$cfg[$pool][mt_rand(0, $max)]; return new PDO($dsn, USER, PASS); }}

$db = DB::getConnection('read');


Step Three: Slave Pools

Virtually divide your slaves into pools

Use this to isolation high database load

Potentially enhance query caches


Possible Pool Layout


Step Four: Partitioning

Simplest Definition:Break your tables or databases into smaller ones


Multiple Masters


Cons of Partitioning

Loss of direct SQL supportIncreased Web Server / PHP loadMore complicated programming


Main Types of Partitioning

HorizontalVertical

Application Level


Vertical Partitioning

“Moving various columns of your table into different tables”

Various methodologies:● Move rarely used columns into auxiliary table● Move often empty columns into auxiliary table● Move columns that are not used in where clauses

Usually done within the same database/server


Vertical Partitioning


Horizontal Partitioning

“Moving various rows of your table into different tables”

Various methodologies:● Range Based● Date Based● Interlaced● User Based

Can be done on one server, or break into multiple masters


Horizontal Partitioning


Application Level Partitioning

“Moving various tables of your DB onto different servers”

Various methodologies:● Move single tables to specific servers● Move groups of related tables together to allow joining


Application Level Partitioning


Multiple Datacenters

You thought worrying about slave lag was bad

Data from multiple sources all needs integrated

Good luck!


Brief Touch on Caching

Often considered performance

Can absolutely be a scalability factor, especially when combined with smaller discrete DB queries

Allows you to get around DB scalability by ignoring the DB


Type of Caching

Single server memory caches

APC or Zend Server Data CacheLimited due to lack of sync'd cache

Distributed

Memcached or Zend PlatformRequired for true scalability enhancement


Caching Best Practices

Write through cache

Choose small, discrete, reusable data units

Don't store data you can't recreate

Store data in as close to final processed form

Questions?

For this presentation & more:

http://eliw.com/

Zend's DevZone:

http://dz.zend.com/

Twitter: @eliw

Rate Me: http://joind.in/591

habits of highly scalable web applications

Documents