Download - Web Performance Culture and Tools at Etsy
![Page 1: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/1.jpg)
Web Performance Culture and Tools at Etsy
Mike Brittain@mikebrittain
Director of Eng., InfrastructureEtsy
![Page 2: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/2.jpg)
All of this started in the last year.
![Page 3: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/3.jpg)
![Page 4: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/4.jpg)
Make PerformanceMatter
![Page 5: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/5.jpg)
$0.20 per item3.5% transaction fee
![Page 6: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/6.jpg)
Focus on the marketplace
![Page 7: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/7.jpg)
ecommerce
![Page 8: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/8.jpg)
ecommercead supported
![Page 9: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/9.jpg)
ecommercead supportedsubscription
![Page 10: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/10.jpg)
ecommercead supportedsubscription“free”
![Page 11: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/11.jpg)
$$$
![Page 12: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/12.jpg)
“Prove it.”
![Page 13: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/13.jpg)
“Our own years of testing have conclusively shown that
when speed of a feature or product improves, usage,
quite simply, goes up.”
http://googleblog.blogspot.com/2009/12/this-week-in-search-121809.html
![Page 14: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/14.jpg)
100+ web, api, jobs40+ databasesMany more doing lots of other stuff...
![Page 15: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/15.jpg)
$$$
![Page 16: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/16.jpg)
Performance is fun
![Page 17: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/17.jpg)
I want you to tell mehow you feel...
![Page 18: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/18.jpg)
mill
isec
onds
95th percentile
Average
![Page 19: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/19.jpg)
mill
isec
onds This is when we
made it go faster!95th percentile
Average
![Page 20: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/20.jpg)
I am better than you.
![Page 21: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/21.jpg)
Make PerformanceMatter
![Page 22: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/22.jpg)
PerformanceCulture
![Page 23: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/23.jpg)
Our perf team is two people...
![Page 24: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/24.jpg)
Measure
![Page 25: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/25.jpg)
MeasureBuild tools
![Page 26: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/26.jpg)
MeasureBuild toolsTrain & Coordinate
![Page 27: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/27.jpg)
MeasureBuild toolsTrain & CoordinateMotivate
![Page 28: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/28.jpg)
MeasureBuild toolsTrain & CoordinateMotivateDedicated
![Page 29: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/29.jpg)
Our perf team is two people...
But every engineer thinksabout performance
![Page 30: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/30.jpg)
Everything is slow. Please fix.
![Page 31: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/31.jpg)
![Page 32: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/32.jpg)
![Page 33: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/33.jpg)
page generationtime
![Page 34: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/34.jpg)
2011 Goal800 ms on five pages
![Page 35: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/35.jpg)
![Page 36: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/36.jpg)
![Page 37: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/37.jpg)
![Page 38: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/38.jpg)
![Page 39: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/39.jpg)
http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
![Page 40: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/40.jpg)
http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
![Page 41: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/41.jpg)
Performance degradationis a P2 bug
![Page 42: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/42.jpg)
Performance degradationis a P2 bug
WARNING: This part is hard
![Page 43: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/43.jpg)
2012 Goal800 ms for all pages
![Page 44: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/44.jpg)
Denormalize dataOptimize queriesRewrite from scratchCaching (memcache, app code)
Engineering
![Page 45: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/45.jpg)
Network switchesOperating systemsNewer hardwareAdditional hardware capacity
Operations
![Page 46: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/46.jpg)
Identify concerns earlyKill lagging features
Product
![Page 47: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/47.jpg)
![Page 48: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/48.jpg)
PerformanceCulture
![Page 49: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/49.jpg)
Tools
![Page 50: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/50.jpg)
How do you make the graphs?
![Page 51: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/51.jpg)
Perf-related data belongs in your server logs
![Page 52: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/52.jpg)
LogFormat %l %t \"%r\" %>s %b \"%{Referer}i\"
\"%{User-Agent}i\" %{custom_field}n
apache_note(“custom_field”, $whatever);
![Page 53: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/53.jpg)
LogFormat "%{True-Client-IP}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %{etsy_user_id}n %{php_bytes}n %{php_microsec}n %D" combined
![Page 54: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/54.jpg)
$timer_start = microtime(true); ...$timer_diff = microtime(true) - $timer_start;
register_shutdown_function()
apache_note('php_microsec', $timer_diff);
![Page 55: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/55.jpg)
web0060 66.249.71.110 - - [24/Aug/2011:04:16:52 +0000] "GET /listing/12189259/tropical-etched-pair-of-lampwork-glass HTTP/1.1" 200 11034 "-" "Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html)" - 13399576 505780 554876
![Page 56: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/56.jpg)
Using something else?time, http method, request uri, response code, referer, user-agent, response time, response memory, custom segmentation fields...
![Page 57: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/57.jpg)
Quick averagesgrep "GET /listing/" access.log | \
awk '{sum=sum+$(NF-1)} END {print sum/NR}'
![Page 58: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/58.jpg)
Quick graphsgrep "GET /listing/" access.log | \
perl -pe "s/.*\[.*\d{4}:(\d{2}):(\d{2}):\d{2}.*\]/\1:\2/" | \
awk '{print $1, $(NF-1)}' > /tmp/pagetimes.dat
gives you...
![Page 59: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/59.jpg)
Quick graphs# /tmp/pagetimes.dat
18:37 251.018:38 252.118:39 253.518:40 251.018:45 250.0 and then...
![Page 60: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/60.jpg)
Quick graphs# GNUPLOT
set terminal png
set output 'listings.png'
set yrange [0:2000]
set xdata time
set timefmt "%d/%B/%Y:%H:%M:%S"
set format x "%H:%M"
plot '/tmp/pagetimes.dat' using 1:2 with points
![Page 61: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/61.jpg)
Quick graphs
![Page 62: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/62.jpg)
LogsterRun by cronMaintains a cursor on log filesSimple parsing & aggregationOutput to Ganglia or Graphite
github.com/etsy
![Page 63: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/63.jpg)
Logster & Graphite
Signed-in vs. Signed-out
![Page 64: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/64.jpg)
Response timesby bucket
time (ms)
resp
onse
s (pa
ges)
![Page 65: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/65.jpg)
“Look at your Data”John Rauser, Velocity 2011
http://www.youtube.com/watch?v=coNDCIMH8bk
![Page 66: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/66.jpg)
![Page 67: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/67.jpg)
Idea lifted fromhttp://www.codinghorror.com/blog/2011/06/performance-is-a-feature.html
![Page 68: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/68.jpg)
Don’t forget the API
![Page 69: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/69.jpg)
Measure performancedata in Production
![Page 70: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/70.jpg)
Profiling
![Page 71: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/71.jpg)
Lightweight, inline profiling
Start and end times wrapped around service calls databases, memcache, apc, 3rd party services, etc.
Enable with query string
“I/O Profiler”
![Page 72: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/72.jpg)
“I/O Profiler”
![Page 73: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/73.jpg)
“I/O Profiler”
![Page 74: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/74.jpg)
XHProfgithub.com/facebook/xhprof
Deployed on pre-prod machineDirect access to profilingEnable with query string
![Page 75: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/75.jpg)
![Page 76: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/76.jpg)
Profile againstProduction
![Page 77: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/77.jpg)
Make performance matter
![Page 78: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/78.jpg)
Start small
![Page 79: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/79.jpg)
Make graphs and reports
![Page 80: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/80.jpg)
Create simple tools
![Page 81: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/81.jpg)
Celebrate performance wins
![Page 83: Web Performance Culture and Tools at Etsy](https://reader034.vdocuments.us/reader034/viewer/2022052315/554bb17db4c905ae618b5923/html5/thumbnails/83.jpg)
Jobs at EtsyWe are currently hiring for a number of positions in engineering and operations.
http://www.etsy.com/careers