gomez / universal mind webinar - are you ready? a new approach for ensuring your web applications...
DESCRIPTION
Ensuring great experiences in today’s web 2.0 world requires a new approach to application load testing. Load Testing 2.0 solutions help to find and fix problems you’ve never been able to before, across the entire web application delivery chain. Thanks to their ease of use and on-demand delivery, 2.0 load testing solutions are more accessible than ever. Join Universal Mind’s Matt Hintze and Gomez’s Colin Mason for this complimentary webinar and learn best practices for: * Ensuring web applications scale under load * Identifying web application performance problems prior to launch * Reducing web application and infrastructure change risk * Identifying performance problems caused by 3rd party content and services * Testing on-demand without huge software and hardware investments * Achieving rapid paybackTRANSCRIPT
Are You Ready?A New Approach for Ensuring Your Web Applications Perform Under Load
Colin Mason - Product Manager Gomez
Matt Hintze - Director of Web Experience Services, Universal Mind
Colin Mason - Product Manager, Gomez
Web Experience Practice
The Universal Mind Web Experience Practice
Web Experience Practice
Core team of Allaire, Macromedia, and Adobe experts
Over 10 years of load testing experience.
We Work with customers to resolve load related incidents.
We Work with customers to prevent load related incidents.
We Focus on maximizing the end user experience.
Load Testing should answer these questions
How many servers do we need?
Can the network infrastructure support the expected traffic?
Will the site stay up during long periods of stress?
Are response times acceptable?
Does our fail-over solution work?
Are we ready to go live?
Why Load Test?You can’t improve what you can’t measure
Wh i l d t ti t t th b i ?
You can t improve what you can t measure
Why is load testing paramount to the business ?
Ensures that web applications deliver superior customer experience to drive greater on-line revenue.
Revenue risked when critical web pages are underperforming
Reduce web application operating costs.
Maintain peak web application performance and availability to protect brand reputation.
Ch M tChange Management.
Why Load Test?You can’t improve what you can’t measure
Why is load testing paramount to the business (cont) ?
You can t improve what you can t measure
Service level compliance of vendors, partners and internal customers.
The Oprah Effect - Avoid being blind sided by load related catastrophic failuresfailures.
Stress test to size server and network infrastructure requirements.
Identify and eliminate bottlenecksIdentify and eliminate bottlenecks.
Tune application code, server settings, and validate CDN performance.
Validate capacity and performance expectations especially for trafficValidate capacity and performance expectations, especially for traffic spikes from campaigns.
Load testing methodology
3 focus areas of load testing
Application TuningIdentify code bottlenecks and scalability problemsIdentify code bottlenecks and scalability problems
Server TuningTune server threading CPU utilization and memoryTune server threading, CPU utilization, and memory
Endurance testingEnsure site performance and availability over long Ensure site performance and availability over long
periods of usage and stress.
Load testing methodology (cont)
The Tuning Process
(cont)
Tuning Cycle
Change one thing at a time.
Load Test and measure performance.Load Test and measure performance.
Document results.
Repeat until the optimal performance is identified (sweet spot)spot).
Modify code or server configs
St T tStress Test
Document Performance Results
Do it again
Real World Examples & Experiences.
The cost of not load testing
Preparation for marketing campaignsp g p g
Gomez Profile
▪ First SaaS Web Application E i M t S it
▪ HeadquartersExperience Management Suite
▪ Web App Experience management across the application lifecycle
▪ Lexington, MA
▪ Global Offices pp y
▪ 80,000+ internet experience measurement points in 162 countries
▪ UK
▪ Germany
▪ Over 2,500 Customers Worldwide
▪ 12 out of 20 most visited US Websites
▪ Switzerland
▪ China▪ 12 out of 20 most visited US Websites
utilize Gomez
▪ 250+ Employees
9
▪ 250+ Employees
Web Application Delivery In A Web 2.0 World
▪ Web applications ppincreasingly assembled at browser level▪ 3rd party
performance from ads, videos, shopping cart vary widely cart vary widely under load
▪ Content Delivery kNetwork
configuration and effectiveness impact end user experience
10
end-user experience
Critical To Test Across Entire Web Application Delivery Chain
11
Load Testing 2.0 –The Next Generation
▪ Load Testing 2.0 - evolution of performance t ti f W b 2 0 ldtesting for a Web 2.0 world
▪ Self-service testing gives you control to test as needed
▪ Test both internal and external web application components to ensure transactions perform under load
T t l d b d li ti d i▪ Test cloud-based applications and services
▪ SaaS based Load Testing 2.0 solutions enable on-demand, scalable testing without associated hardware demand, scalable testing without associated hardware investment and maintenance costs
▪ Test the full web application delivery chain all the way d ’ d k
12
to your end users’ desktops
Firewall Bottleneck
CompanyCompany• Online Sporting Goods Retailer
Situation• As part of a major system upgrade a number
of key infrastructure pieces were upgraded• Following a Best Practices Methodology theFollowing a Best Practices Methodology the
testing began with a simple test designed to validate the new firewall
• First Step was to benchmark the current• First Step was to benchmark the current firewall
• Second Step repeat the test on the new firewall
13
firewall
Firewall Bottleneck
Test 1: Old Firewall• Users ramp up, and the firewall supported more than 2500 transactions per second.
• This was more than the approx• This was more than the approx 2000 transactions per second seen in peak live traffic.
Test 2: New FirewallU b t th• Users ramp up, but the transaction rate bottlenecks at 1000 per second.
• Response times for users
14
climb.
Firewall Bottleneck
▪ Cause/Resolution
▪ The new firewall was only licensed for 1000 transactions per second.
R l th li ith th t i li th h d ▪ Replace the license with the enterprise license they had purchased.
▪ Business Impact
▪ This was a simple mistake, but not uncommon, and if it was not caught their next peak day customers would was not caught their next peak day customers would have been negatively impacted by
▪ Higher response times
15
▪ Connection errors
Third-Party Bottleneck
Company• Online Retailer
Situation• The application consists of a typical
e Commerce type applicatione-Commerce type application• Several 3rd Parties are now involved
in serving up key content
16
Third-Party Issues
The load increases throughout the test.
The transaction rate increases and then falls off as response ti li btimes climb.
Errors are seen, all on a 3rd party
bj tobject.
17
Third-Party Bottleneck
▪ Cause/Resolution
▪ The 3rd party hardware was insufficient for overall demands on the application
B d SLA th 3 d t h d t i th i ▪ Based on SLAs the 3rd party had to improve their performance in order to get paid
▪ All aspects of the user experience delivery chain must be tested
▪ Business ImpactBusiness Impact
▪ Customers negatively impacted by
▪ Higher response timesHigher response times
▪ Time-out errors18
Real User Bottleneck
Company• Regional Online News Source
Sit tiSituation• Began testing for the 2008 election season• Looking to validate performance of overallLooking to validate performance of overall
application, but in particular for users in 2 key geographies
• Testing started with the homepage• Testing started with the homepage
19
The False Positive
There was only 1 page error and 11
Increase and hold load and not exceed response times of 4 seconds and Success Rate of 99%
Page response
p gerrors total out of 60000+ transactions.
Rate of 99%
Page response times stayed under 4 seconds, outside of one brief blipp
By traditional test standards the test passed20
By traditional test standards the test passed.
The False PositiveReal User Experience
Last Mile data showing substantial number ofLast Mile data showing substantial number of measurements greater than 4 seconds
21
The False PositivePrimary Geographies
Key geographies for this customer are New York and Pennsylvania.
The response time met the 4
second average goal.
Availability was Less than 99%
By these standards the test failed.Less than 99%.
22
The False PositiveUser Experience Drill Down
• Last Mile Users exceeding the 4-d l 29%second goal – 29%
• Overall availability was 96.3%
• Availability for Last Mile Users exceeding 4 seconds – 87%
• Last Mile Users reporting 10+ second response times – 8%
B th t d d th t t f il d23
By these standards the test failed.
False Positive
▪ Cause/Resolution
▪ The key action item was for the customer to focus on improving the response times of the page template from which the homepage was designedwhich the homepage was designed
▪ Focusing on cutting 1.5-2 seconds off the response time for the page would move another 16% of the Last Mile users into the 4 second response time rangeusers into the 4 second response time range
▪ Business Impact
▪ Customers negatively impacted by
▪ Higher response times
24
▪ Availability issues
Ensure Web Experiences Scale Under Load
▪ Deploy applications and ▪ Deploy applications and infrastructure with confidence
▪ Combine high volume Combine high volume application load testing with broad geographic experience testing
▪ Ensure on-demand testing flexibility
▪ Detect failures in user experience not caught by traditional load testingtraditional load testing
Find Out How Gomez Can Help You
Some of Our CustomersEnsuring great experiences in a Web 2.0 world requires a new approach to application load testingapplication load testing.
Load Testing 2.0• Self-serviceT t t id i ith d ill d t b• Tests outside-in, with drill down to web application components
• SaaS with no investment or maintenance costs and rapid paybackcosts and rapid payback
• Full desktop browser testing across globally distributed geographies
L MLearn More:Load Testing 2.0 Whitepaperhttp://www.gomez.com/registration/white_paper.php?id=41
26Speak To A Gomez Solutions Expert
Toll Free: 877.372.6732