performance testing crash course (dustin whittle)
Post on 28-Jul-2015
725 Views
Preview:
TRANSCRIPT
Agenda• Why performance matters?
• Case studies on business impact of performance
• Tools of the trade
• MultiMechanize
• Bees with Machine Guns
• Google PageSpeed
• AppDynamics
• Best practices
• Server Side
• Client Side
• Integrating into your process
• Develop -> Test -> Deploy -> Repeat
• What was the performance impact the last release?
• What about the impact of that configuration change or package upgrade?
Dustin Whittle
• dustinwhittle.com
• @dustinwhittle
• San Francisco, California, USA
• Technologist, Traveler, Pilot, Skier, Diver, Sailor, Golfer
Microsoft found that Bing searches that were 2 seconds slower
resulted in a 4.3% drop in revenue per user
How fast is fast enough?§ Performance is key to a great user experience
-‐ Under 100ms is perceived as reac1ng instantaneously
-‐ A 100ms to 300ms delay is percep1ble
-‐ 1 second is about the limit for the user's flow of thought to stay uninterrupted
-‐ Users expect a site to load in 2 seconds
-‐ A>er 3 seconds, 40% will abandon your site.
-‐ 10 seconds is about the limit for keeping the user's aAen1on
§ Modern applicaGons spend more Gme in the browser than on the server-‐side
git clone https://github.com/cloudfoundry-samples/hello-spring-cloud
Benchmarking springone-demo.cfapps.io (be patient) Finished 187 requests
Server Software: Apache-Coyote/1.1 Server Hostname: springone-demo.cfapps.io Server Port: 80
Document Path: / Document Length: 5217 bytes
Concurrency Level: 1 Time taken for tests: 10.039 seconds Complete requests: 187 Failed requests: 0 Keep-Alive requests: 187 Total transferred: 1021768 bytes HTML transferred: 975579 bytes Requests per second: 18.63 [#/sec] (mean) Time per request: 53.687 [ms] (mean)
Benchmarking springone-demo.cfapps.io (be patient) Finished 659 requests
Server Software: Apache-Coyote/1.1 Server Hostname: springone-demo.cfapps.io Server Port: 80
Document Path: / Document Length: 5217 bytes
Concurrency Level: 10 Time taken for tests: 10.015 seconds Complete requests: 659 Failed requests: 0 Keep-Alive requests: 659 Total transferred: 3600776 bytes HTML transferred: 3438003 bytes Requests per second: 65.80 [#/sec] (mean) Time per request: 151.970 [ms] (mean)
** SIEGE 3.0.6** Preparing 10 concurrent users for battle.The server is now under siege...Lifting the server siege... done.
Transactions: 623 hitsAvailability: 100.00 %Elapsed time: 9.57 secsData transferred: 3.10 MBResponse time: 0.15 secsTransaction rate: 65.10 trans/secThroughput: 0.32 MB/secConcurrency: 9.91Successful transactions: 623Failed transactions: 0Longest transaction: 0.30Shortest transaction: 0.10
SPROXY v1.02 listening on port 9001 ...appending HTTP requests to: ./urls.txt
...default connection timeout: 120 seconds
wget -r -l 0 -t 1 --spider -w 1 -e "http_proxy = http://127.0.0.1:9001" "http://acmedemoapp.com/"
sort -u -o urls.txt urls.txt
http://acmedemoapp.com/ http://acmedemoapp.com/about http://acmedemoapp.com/cart
http://acmedemoapp.com/currency/change/EUR http://acmedemoapp.com/currency/change/GBP http://acmedemoapp.com/currency/change/USD
http://acmedemoapp.com/login http://acmedemoapp.com/register/
http://acmedemoapp.com/t/brand/bookmania http://acmedemoapp.com/t/category/books http://acmedemoapp.com/t/category/mugs
http://acmedemoapp.com/t/category/stickers http://acmedemoapp.com/terms-of-service
** SIEGE 3.0.6** Preparing 10 concurrent users for battle.The server is now under siege...Lifting the server siege... done.
Transactions: 623 hitsAvailability: 100.00 %Elapsed time: 9.57 secsData transferred: 3.10 MBResponse time: 0.15 secsTransaction rate: 65.10 trans/secThroughput: 0.32 MB/secConcurrency: 9.91Successful transactions: 623Failed transactions: 0Longest transaction: 0.30Shortest transaction: 0.10
import requests
class Transaction(object): def run(self): r = requests.get(‘http://acmedemoapp.com/) r.raw.read()
import mechanize import time
class Transaction(object): def run(self): br = mechanize.Browser() br.set_handle_robots(False)
start_timer = time.time() resp = br.open(‘http://acmedemoapp.com/) resp.read() latency = time.time() - start_timer self.custom_timers['homepage'] = latency
start_timer = time.time() resp = br.open(‘http://acmedemoapp.com/cart') resp.read() latency = time.time() - start_timer self.custom_timers['cart'] = latency assert (resp.code == 200)
A utility for arming (creating) many bees (micro EC2 instances)
to attack (load test) targets (web applications)
# ~/.boto
[Credentials]
aws_access_key_id=xxx aws_secret_access_key=xxx
[Boto]
ec2_region_name = us-west-2 ec2_region_endpoint = ec2.us-west-2.amazonaws.com
bees up -s 2 -g default -z us-west-2b -i ami-bc05898c -k
appdynamics-dustinwhittle-aws-us-west-2 -l ec2-user
Connecting to the hive. Attempting to call up 2 bees.
Waiting for bees to load their machine guns... . . .
Read 2 bees from the roster. Bee i-3828400c: running @ 54.212.22.176 Bee i-3928400d: running @ 50.112.6.191
Read 2 bees from the roster. Connecting to the hive. Assembling bees. Each of 2 bees will fire 500 rounds, 25 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. … Offensive complete. Complete requests: 1000 Requests per second: 306.540000 [#/sec] (mean) Time per request: 163.112000 [ms] (mean) 50% response time: 151.000000 [ms] (mean) 90% response time: 192.000000 [ms] (mean) Mission Assessment: Target crushed bee offensive. The swarm is awaiting new orders.
Read 2 bees from the roster. Connecting to the hive. Assembling bees. Each of 2 bees will fire 50000 rounds, 500 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. … Offensive complete. Complete requests: 100000 Requests per second: 502.420000 [#/sec] (mean) Time per request: 360.114000 [ms] (mean) 50% response time: 451.000000 [ms] (mean) 90% response time: 402.000000 [ms] (mean) Mission Assessment: Target crushed bee offensive. The swarm is awaiting new orders.
Read 2 bees from the roster. Connecting to the hive. Assembling bees. Each of 2 bees will fire 50000 rounds, 500 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. Bee 0 is joining the swarm. Bee 1 is joining the swarm. Bee 0 is firing his machine gun. Bang bang! Bee 0 lost sight of the target (connection timed out). Bee 1 lost sight of the target (connection timed out). Offensive complete. Target timed out without fully responding to 2 bees. No bees completed the mission. Apparently your bees are peace-loving hippies. The swarm is awaiting new orders.
curl "https://www.googleapis.com/pagespeedonline/v1/runPagespeed?url=http://dustinwhittle.com/&key=xxx"
Test for failures
• NetFlix Simian Army + Chaos Monkey
• What happens if you lose a caching layer?
• What happens if dependencies slow down?
Best Practices• Treat performance as a feature
• Capacity plan and load test the server-side
• Optimize and performance test the client-side
• Understand your starting point
• Instrument everything
• Monitor performance in development and production
• Measure the difference of every change
• Automate performance testing in your build and deployment process
• Understand how failures impact performance
Find these slides on SpeakerDeck
https://speakerdeck.com/dustinwhittle
top related