evolving archetecture
DESCRIPTION
Looking at historic, current and evolving approaches, I will take you through from how we used to 'live' edit on one server with HTML in the code; to implementing Template Toolkit and 'front end / back end' servers; to the addition of version control; all the way through to distributed caching, file systems and processing (aka Six Apart worship) with 15+ servers.TRANSCRIPT
![Page 1: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/1.jpg)
Evolving architecturemake development easy and your site faster
Leo Lapworth @ YAPC::EU 2007http://leo.cuckoo.org/
![Page 2: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/2.jpg)
Covering
![Page 3: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/3.jpg)
Covering
How running websites has evolved
![Page 4: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/4.jpg)
Covering
How running websites has evolved
Servers improved
![Page 5: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/5.jpg)
Covering
How running websites has evolved
Servers improved
Architecture improved
![Page 6: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/6.jpg)
Covering
How running websites has evolved
Servers improved
Architecture improved
Development tools improved
![Page 7: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/7.jpg)
Covering
How running websites has evolved
Servers improved
Architecture improved
Development tools improved
Code (and code libraries) got better
![Page 8: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/8.jpg)
but not...
![Page 9: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/9.jpg)
but not...
Covering anything in depth
![Page 10: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/10.jpg)
Concepts stolen from
![Page 11: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/11.jpg)
Concepts stolen from
Mod_perl guide
![Page 12: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/12.jpg)
Concepts stolen from
Mod_perl guide
Léon Brocard
![Page 13: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/13.jpg)
Concepts stolen from
Mod_perl guide
Léon Brocard
Six Apart
![Page 14: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/14.jpg)
Concepts stolen from
Mod_perl guide
Léon Brocard
Six Apart
Yahoo!
![Page 15: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/15.jpg)
Concepts stolen from
Mod_perl guide
Léon Brocard
Six Apart
Yahoo!
+ many other people...
![Page 16: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/16.jpg)
Concepts stolen from
Mod_perl guide
Léon Brocard
Six Apart
Yahoo!
+ many other people...
... this is social Perl
![Page 17: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/17.jpg)
Templates
![Page 18: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/18.jpg)
TemplatesAn example of how things have improved...
![Page 19: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/19.jpg)
Recognise this?
![Page 20: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/20.jpg)
Recognise this?use CGI;$q = CGI->new();$name = $q->param('name');print $q->header('text/html');$link = 'http://foo.com/';print "<html><head><title>HTML and code together</title></head><body>";print "<a href=\"$link/$name\">";print "Click here $name</a><br />\n";print "<table>" . join("\n", map { "<tr><td>$_</td></tr>"} qw(this is hard to maintain));print "</table>";
![Page 21: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/21.jpg)
How about...
![Page 22: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/22.jpg)
How about...
Now, was it cellspacing, or cellpadding I need for ie4.02?
![Page 23: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/23.jpg)
and...
![Page 24: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/24.jpg)
and...
I'm a developer, not a designer. I shouldn't have to know this £@$%!
![Page 25: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/25.jpg)
Use templates...
![Page 26: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/26.jpg)
Use templates...
use Template;
![Page 27: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/27.jpg)
Code, not HTML
![Page 28: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/28.jpg)
Code, not HTMLuse CGI;use Template;
$q = CGI->new();print $q->header('text/html');my %vals = ( name = $q->param('name'),);
$template = Template->new({ INCLUDE => '/path/' });$template->process('hello.html',\%vals);
![Page 29: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/29.jpg)
I just code, I have clean code and I don't
have to know HTML or browser compatibility
issues
![Page 30: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/30.jpg)
Simple changes make a huge difference
![Page 31: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/31.jpg)
Servers
![Page 32: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/32.jpg)
Single server
![Page 33: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/33.jpg)
Single server
Internet
Web server
(static & dynamic)
Database server
All editing
![Page 34: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/34.jpg)
Lots of...
![Page 35: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/35.jpg)
and...
I thought I only changed one thing, but
it's still broken
![Page 36: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/36.jpg)
1 live server 1 development server
![Page 37: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/37.jpg)
1 live server 1 development server
Internet
Live server
Web server
(static & dynamic)
Database server
All editing
Development server
Web server
(static & dynamic)
Database server
![Page 38: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/38.jpg)
Does not solve:
What did I change last?
Who changed what / when?
![Page 39: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/39.jpg)
You need...
![Page 40: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/40.jpg)
You need...
![Page 41: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/41.jpg)
Version control & docs
![Page 42: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/42.jpg)
Version control & docs
Use SVK with Subversion
![Page 43: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/43.jpg)
Version control & docs
Use SVK with Subversion
![Page 44: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/44.jpg)
Version control & docs
Clkao - SVK
Use SVK with Subversion
![Page 45: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/45.jpg)
Version control & docs
Clkao - SVK
Use Trac (http://trac.edgewall.org)
Use SVK with Subversion
![Page 46: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/46.jpg)
Version control & docs
Clkao - SVK
Use Trac (http://trac.edgewall.org) - wiki
Use SVK with Subversion
![Page 47: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/47.jpg)
Version control & docs
Clkao - SVK
Use Trac (http://trac.edgewall.org) - wiki- task tracker
Use SVK with Subversion
![Page 48: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/48.jpg)
Version control & docs
Clkao - SVK
Use Trac (http://trac.edgewall.org) - wiki- task tracker- web interface to Subversion
Use SVK with Subversion
![Page 49: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/49.jpg)
![Page 50: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/50.jpg)
Tests - all you need on one slide
![Page 51: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/51.jpg)
Tests - all you need on one slide
Write tests, write tests often
![Page 52: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/52.jpg)
Tests - all you need on one slide
Write tests, write tests often
Run tests, run tests often
![Page 53: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/53.jpg)
Tests - all you need on one slide
Write tests, write tests often
Run tests, run tests often
Tests are good
![Page 54: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/54.jpg)
Tests - all you need on one slide
Write tests, write tests often
Run tests, run tests often
Tests are good
Found a bug? - write the test that replicates it. You know it's fixed when your test passes
![Page 55: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/55.jpg)
Tests - all you need on one slide
Write tests, write tests often
Run tests, run tests often
Tests are good
Found a bug? - write the test that replicates it. You know it's fixed when your test passes
Use Test::More, and then test some more!
![Page 56: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/56.jpg)
Making your site faster
![Page 57: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/57.jpg)
mod_perl
![Page 58: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/58.jpg)
mod_perl
Code caching (loaded/compiled once)
![Page 59: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/59.jpg)
mod_perl
Code caching (loaded/compiled once)
Server only spends time running the code
![Page 60: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/60.jpg)
mod_perl
Code caching (loaded/compiled once)
Server only spends time running the code
Apache::Registry for old/existing code
![Page 61: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/61.jpg)
mod_perl
Code caching (loaded/compiled once)
Server only spends time running the code
Apache::Registry for old/existing code
Apache::SizeLimit (safety net)
![Page 62: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/62.jpg)
mod_perl
Code caching (loaded/compiled once)
Server only spends time running the code
Apache::Registry for old/existing code
Apache::SizeLimit (safety net)
Set it high
![Page 63: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/63.jpg)
mod_perl
Code caching (loaded/compiled once)
Server only spends time running the code
Apache::Registry for old/existing code
Apache::SizeLimit (safety net)
Set it high
Check it
![Page 64: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/64.jpg)
Front/back end split
![Page 65: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/65.jpg)
Front/back end split
Internet
Front end
Web server
(static & proxy to
application server)
Development
server
Back end
(application server)
Web server
(dynamic)
Database server
Slow
Fast
![Page 66: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/66.jpg)
or put another way...
![Page 67: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/67.jpg)
or put another way...
Internetslow and
demanding kids
(users)
![Page 68: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/68.jpg)
or put another way...
Internetslow and
demanding kids
(users)
Front endDinner-ladies - good at
serving precooked (static) food. Can order from à la carte chefs
and then serve
Canteen
![Page 69: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/69.jpg)
or put another way...
Internetslow and
demanding kids
(users)
Back endChefs - good at
making à la carte (dynamic) food
Kitchen
Front endDinner-ladies - good at
serving precooked (static) food. Can order from à la carte chefs
and then serve
Canteen
![Page 70: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/70.jpg)
Internet
Front end
Web server
(static & proxy to
application server)
Development
server
Back end
(application server)
Web server
(dynamic)
Database server
Slow
Fast
![Page 71: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/71.jpg)
Internet
Front end
Web server
(static & proxy to
application server)
Development
server
Back end
(application server)
Web server
(dynamic)
Database server
Slow
Fast
![Page 72: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/72.jpg)
Speed up the frequent...
![Page 73: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/73.jpg)
Speed up the frequent...
Are you re-creating the same objects/data structures again and again?
![Page 74: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/74.jpg)
Speed up the frequent...
Are you re-creating the same objects/data structures again and again?
Can this information persist for a period of time?
![Page 75: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/75.jpg)
Add caching
![Page 76: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/76.jpg)
Add caching
Persisting data for a period of time
![Page 77: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/77.jpg)
Add caching
Persisting data for a period of time
Simple to implement:
![Page 78: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/78.jpg)
Add caching
Persisting data for a period of time
Simple to implement:
Does cache exist?
![Page 79: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/79.jpg)
Add caching
Persisting data for a period of time
Simple to implement:
Does cache exist?
YES -> retrieve it and return it
![Page 80: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/80.jpg)
Add caching
Persisting data for a period of time
Simple to implement:
Does cache exist?
YES -> retrieve it and return it
NO -> create data/object, store in cache and return it
![Page 81: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/81.jpg)
Things you might cache
![Page 82: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/82.jpg)
Things you might cache
Search result sets (a list of ids)
![Page 83: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/83.jpg)
Things you might cache
Search result sets (a list of ids)
Individual items
![Page 84: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/84.jpg)
Things you might cache
Search result sets (a list of ids)
Individual items
Look-ups of information from a database
![Page 85: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/85.jpg)
Things you might cache
Search result sets (a list of ids)
Individual items
Look-ups of information from a database
Fetching of information from external sources
![Page 86: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/86.jpg)
Centralise
![Page 87: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/87.jpg)
Centralise
Put caching methods in one package
![Page 88: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/88.jpg)
Centralise
Put caching methods in one package
Put any generic methods in a package
![Page 89: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/89.jpg)
We now have a cache
Internet
Front end
Web server
(static & proxy to
application server)
Development
server
Back end
(application server)
Web server
(dynamic)
Database server
Slow
Fast
Cache
![Page 90: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/90.jpg)
We now have a cache
Internet
Front end
Web server
(static & proxy to
application server)
Development
server
Back end
(application server)
Web server
(dynamic)
Database server
Slow
Fast
CacheCache
![Page 91: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/91.jpg)
Adding in the cache...
![Page 92: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/92.jpg)
Adding in the cache...
InternetKids
![Page 93: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/93.jpg)
Adding in the cache...
InternetKids
Front endDinner-ladies
![Page 94: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/94.jpg)
Adding in the cache...
InternetKids
Back endChefs
Front endDinner-ladies
![Page 95: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/95.jpg)
Adding in the cache...
CacheMagic
freezer
InternetKids
Back endChefs
Front endDinner-ladies
![Page 96: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/96.jpg)
Coping with more traffic
![Page 97: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/97.jpg)
Coping with more traffic
Internet
Front end
Web server
(static & proxy
to application
server)
Stage server
Back end
(application server)
Web server
(dynamic)
Development
server (in the
office)
Database
server
Back end
(application server)
Web server
(dynamic)
![Page 98: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/98.jpg)
Coping with more traffic
Internet
Front end
Web server
(static & proxy
to application
server)
Stage server
Back end
(application server)
Web server
(dynamic)
Development
server (in the
office)
Database
server
Back end
(application server)
Web server
(dynamic)
![Page 99: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/99.jpg)
That's two separate disks
![Page 100: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/100.jpg)
That's two separate disks
Internet
Front end
Web server
(static & proxy
to application
server)
Stage server
Back end
(application server)
Web server
(dynamic)
Development
server (in the
office)
Database
server
Back end
(application server)
Web server
(dynamic)Cache
Cache
![Page 101: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/101.jpg)
Second kitchen, two freezers...
![Page 102: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/102.jpg)
Second kitchen, two freezers...
InternetKids
![Page 103: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/103.jpg)
Second kitchen, two freezers...
InternetKids
Front endDinner-ladies
![Page 104: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/104.jpg)
Second kitchen, two freezers...
InternetKids
Back endChefs
Kitchen 1
Kitchen 2
Front endDinner-ladies
![Page 105: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/105.jpg)
Second kitchen, two freezers...
InternetKids
Back endChefs
Kitchen 1
Kitchen 2
CacheFreezers
Front endDinner-ladies
![Page 106: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/106.jpg)
We want a cache across servers...
![Page 107: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/107.jpg)
We want a cache across servers...
EnterMemcache
![Page 108: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/108.jpg)
We want a cache across servers...
EnterMemcache
See Léon Brocard's talkat 16:20 for the details
![Page 109: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/109.jpg)
Internet
Front end
Web server
(static & proxy
to application
server)
Stage server
Back end
(application server)
Web server
(dynamic)
Development
server (in the
office)
Database
server
Back end
(application server)
Web server
(dynamic)
Cache
![Page 110: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/110.jpg)
Shared freezer
![Page 111: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/111.jpg)
Shared freezer
InternetKids
![Page 112: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/112.jpg)
Shared freezer
InternetKids
Front endDinner-ladies
![Page 113: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/113.jpg)
Shared freezer
InternetKids Back end
Chefs
Kitchen 1
Kitchen 2
Front endDinner-ladies
![Page 114: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/114.jpg)
Shared freezer
InternetKids Back end
Chefs
Kitchen 1
Kitchen 2
CacheFreezer
Front endDinner-ladies
![Page 115: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/115.jpg)
Maintaining servers
![Page 116: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/116.jpg)
Maintaining servers
Keep all servers identical where possible
![Page 117: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/117.jpg)
Maintaining servers
Keep all servers identical where possible
Package your code (e.g. .debs)
![Page 118: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/118.jpg)
Maintaining servers
Keep all servers identical where possible
Package your code (e.g. .debs)
Centralise your crontabs and configuration files (and put in version control)
![Page 119: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/119.jpg)
Maintaining servers
Keep all servers identical where possible
Package your code (e.g. .debs)
Centralise your crontabs and configuration files (and put in version control)
Deploy with one script: rsync + ssh + apt-get update & apt-get upgrade (or your OS equivalent) to each server
![Page 120: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/120.jpg)
Simplify further
Internet
Front end
Perlbal
load balancer /
proxy
Stage server
Back end
Web server
(static+dynamic)
Development
server (in the
office)
Database
server
Back end
Web server
(static+dynamic)
Cache
![Page 121: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/121.jpg)
Simplify further
Internet
Front end
Perlbal
load balancer /
proxy
Stage server
Back end
Web server
(static+dynamic)
Development
server (in the
office)
Database
server
Back end
Web server
(static+dynamic)
Cache
![Page 122: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/122.jpg)
Simplify further
Internet
Front end
Perlbal
load balancer /
proxy
Stage server
Back end
Web server
(static+dynamic)
Development
server (in the
office)
Database
server
Back end
Web server
(static+dynamic)
Cache
![Page 123: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/123.jpg)
Waiters: cheaper, faster and less demanding
![Page 124: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/124.jpg)
Waiters: cheaper, faster and less demanding
InternetKids
![Page 125: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/125.jpg)
Waiters: cheaper, faster and less demanding
InternetKids
Front endWaiters - good at serving quickly and dealing with annoying customers or
Kitchens
![Page 126: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/126.jpg)
Waiters: cheaper, faster and less demanding
InternetKids Back end
Chefs - simple and complex meals(static and dynamic)
Kitchen 1
Kitchen 2
Front endWaiters - good at serving quickly and dealing with annoying customers or
Kitchens
![Page 127: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/127.jpg)
Waiters: cheaper, faster and less demanding
InternetKids Back end
Chefs - simple and complex meals(static and dynamic)
Kitchen 1
Kitchen 2
CacheFreezer
Front endWaiters - good at serving quickly and dealing with annoying customers or
Kitchens
![Page 128: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/128.jpg)
More tips for faster user experience
![Page 129: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/129.jpg)
mod_gzip
![Page 130: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/130.jpg)
mod_gzip
HTML/css/xml
![Page 131: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/131.jpg)
mod_gzip
HTML/css/xml
![Page 132: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/132.jpg)
mod_gzip
HTML/css/xml gzip
![Page 133: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/133.jpg)
mod_gzip
HTML/css/xml gzip
![Page 134: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/134.jpg)
mod_gzip
HTML/css/xml gzip Users
![Page 135: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/135.jpg)
mod_gzip
HTML/css/xml gzip
If a browser support compressions... compress(javascript can be tricky - at least minify)
Users
![Page 136: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/136.jpg)
Cache headers (expiry)
![Page 137: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/137.jpg)
Cache headers (expiry)
![Page 138: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/138.jpg)
Cache headers (expiry)
Let web browsers know how long to cache
![Page 139: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/139.jpg)
Cache somethings forever
![Page 140: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/140.jpg)
Cache somethings forever
/includes/js/<VERSION>/common.js
![Page 141: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/141.jpg)
Cache somethings forever
Ensures user has version which matches HTML (client could have cache of old HTML)
/includes/js/<VERSION>/common.js
![Page 142: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/142.jpg)
Cache somethings forever
Ensures user has version which matches HTML (client could have cache of old HTML)
Use include file to update all pages
/includes/js/<VERSION>/common.js
![Page 143: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/143.jpg)
Handling images: MogileFS
![Page 144: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/144.jpg)
Handling images: MogileFS
Store images by group (replication levels)
![Page 145: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/145.jpg)
Handling images: MogileFS
Store images by group (replication levels)
Spread IO across disks & servers
![Page 146: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/146.jpg)
Handling images: MogileFS
Store images by group (replication levels)
Spread IO across disks & servers
Retrieve from fastest server
![Page 147: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/147.jpg)
Handling images: MogileFS
Store images by group (replication levels)
Spread IO across disks & servers
Retrieve from fastest server
Integrate with Perlbal, hidden from the user
![Page 148: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/148.jpg)
Handling images: MogileFS
Store images by group (replication levels)
Spread IO across disks & servers
Retrieve from fastest server
Integrate with Perlbal, hidden from the user
Automatic resilience
![Page 149: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/149.jpg)
Handling images: MogileFS
Store images by group (replication levels)
Spread IO across disks & servers
Retrieve from fastest server
Integrate with Perlbal, hidden from the user
Automatic resilience
![Page 150: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/150.jpg)
Basic rules to consider
![Page 151: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/151.jpg)
Basic rules to consider
Centralise (code/templates/configuration/deployment etc)
![Page 152: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/152.jpg)
Basic rules to consider
Centralise (code/templates/configuration/deployment etc)
Test, run lots of them
![Page 153: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/153.jpg)
Basic rules to consider
Centralise (code/templates/configuration/deployment etc)
Test, run lots of them
Cache if you need it; memcache is great
![Page 154: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/154.jpg)
Basic rules to consider
Centralise (code/templates/configuration/deployment etc)
Test, run lots of them
Cache if you need it; memcache is great
Keep is simple (perlbal especially)
![Page 155: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/155.jpg)
Basic rules to consider
Centralise (code/templates/configuration/deployment etc)
Test, run lots of them
Cache if you need it; memcache is great
Keep is simple (perlbal especially)
Sometimes it's just easier to buy another server (this is not always true)
![Page 156: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/156.jpg)
We covered...
![Page 157: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/157.jpg)
We covered...
perlbal
![Page 158: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/158.jpg)
We covered...
perlbal
memcached
![Page 159: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/159.jpg)
We covered...
perlbal
memcached
svk (subversion)
![Page 160: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/160.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
![Page 161: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/161.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
rsync
![Page 162: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/162.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
rsync
gzip
![Page 163: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/163.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
rsync
gzip
mogileFS
![Page 164: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/164.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
rsync
gzip
mogileFS
server architecture
![Page 165: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/165.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
rsync
gzip
mogileFS
server architecture
Test::More
![Page 166: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/166.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
rsync
gzip
mogileFS
server architecture
Test::More
mod_perl
![Page 167: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/167.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
rsync
gzip
mogileFS
server architecture
Test::More
mod_perl
Template::Toolkit
![Page 168: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/168.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
rsync
gzip
mogileFS
server architecture
Test::More
mod_perl
Template::Toolkit
HTTP Cache headers
![Page 169: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/169.jpg)
We covered...
perlbal
memcached
svk (subversion)
trac
rsync
gzip
mogileFS
server architecture
Test::More
mod_perl
Template::Toolkit
HTTP Cache headers
and...
![Page 170: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/170.jpg)
...how a kitchen should run
![Page 171: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/171.jpg)
...how a kitchen should run
![Page 172: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/172.jpg)
...how a kitchen should run
Any questions?
![Page 173: Evolving Archetecture](https://reader034.vdocuments.us/reader034/viewer/2022042613/554940bfb4c905054d8b505a/html5/thumbnails/173.jpg)
Evolving architecturehttp://leo.cuckoo.org/projects/ea/
Leo Lapworthhttp://leo.cuckoo.org/