![Page 1: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/1.jpg)
‘Tis the SeasonA Holiday 2014 Ecommerce Checklist
Preparing for best case retail success
![Page 2: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/2.jpg)
Holiday 2013 was
BIG
![Page 3: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/3.jpg)
![Page 4: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/4.jpg)
2014 will be
BIGGER
![Page 5: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/5.jpg)
![Page 6: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/6.jpg)
mobile matters
![Page 7: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/7.jpg)
![Page 8: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/8.jpg)
your site needs to
PERFORM
![Page 9: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/9.jpg)
40 % abandon a website that takes more than 3 seconds to load
A 1 second delay in page response can result in a 7% reduction in conversions
79% of shoppers who are dissatisfied with website performance are less likely to buy from the same site again.
![Page 10: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/10.jpg)
If an ecommerce site is making
$10,000 a day, a 1 second delay could cost you
$250,000 in lost sales every year
![Page 11: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/11.jpg)
Start NowPrepare for the holiday rush
![Page 12: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/12.jpg)
‘Tis the SeasonA Holiday 2014 Ecommerce Checklist
Preparing for best case retail success
![Page 13: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/13.jpg)
Elizabeth Scott, TenzingDirector of Technical Services @BethxScott
Brad Johnson, SOASTAVP Product Marketing@bradjohnsonsv
Your Hosts
![Page 14: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/14.jpg)
Infrastructure and Application
Monitoring
Load Testing
Marketing
Agenda
![Page 15: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/15.jpg)
Poll Question
![Page 16: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/16.jpg)
![Page 17: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/17.jpg)
Infrastructure and Application
![Page 18: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/18.jpg)
Assess Your Infrastructure
Current infrastructure performance and capacity baseline:
Server Hardware
Storage
Network Database Application
![Page 19: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/19.jpg)
Assess Your Infrastructure
Know how many users can be on your site: Do a load test
Understand your vendor’s capacity planning process
Develop an emergency capacity plan
![Page 20: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/20.jpg)
Ensure your environment is up to date on latest patch levels
Develop an emergency patch and security plan
Secure Your Environment
![Page 21: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/21.jpg)
Ensure ports on your servers are not open unnecessarily
Limit access
Secure Your Environment
![Page 22: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/22.jpg)
Develop and test your Denial of Service response strategy
Run a vulnerability scan
Secure Your Environment
![Page 23: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/23.jpg)
Tune queries, search results and session limits
Leverage a CDN
Tune Your Application
![Page 24: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/24.jpg)
Find free capacity:
Adjust your caching
Adjust ‘type ahead’ search
Remove/adjust view all options
Adjust other features which are resource hogs
Tune Your Application
![Page 25: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/25.jpg)
Monitoring
![Page 26: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/26.jpg)
Application PerformanceEnd User ExperienceTraffic Levels
Measure, Analyze, Act
![Page 27: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/27.jpg)
•Upstream ConnectionsPayment GatewayShipping and Fulfillment
Measure, Analyze, Act
![Page 28: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/28.jpg)
Plan for the Inevitable
![Page 29: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/29.jpg)
Assess your risk …
WHAT WILL GO DOWN AND WHY?
Plan for the Inevitable
![Page 30: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/30.jpg)
•Create a planWebsite overload plan
Incident response plan
Security/DoS response plan
Plan for the Inevitable
![Page 31: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/31.jpg)
COMMUNICATE
Plan for the Inevitable
![Page 32: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/32.jpg)
Modern Load Testing
![Page 33: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/33.jpg)
True Story
![Page 34: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/34.jpg)
• Division of $6B Company
True Story
![Page 35: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/35.jpg)
• Division of $6B Company• Cyber Monday & last Free Shipping Day
True Story
![Page 36: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/36.jpg)
• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”
True Story
![Page 37: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/37.jpg)
• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”• Crashed on Cyber Monday
True Story
![Page 38: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/38.jpg)
• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”• Crashed on Cyber Monday• No answers on Thursday
True Story
![Page 39: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/39.jpg)
• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”• Crashed on Cyber Monday• No answers on Thursday• Production testing on Friday
reproduced the issue
True Story
![Page 40: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/40.jpg)
Is there anything missing from the
plan?
![Page 41: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/41.jpg)
Prerequisite: Know Your Real Users
Modern Load Testing
Stop guessing about what to test
![Page 42: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/42.jpg)
Most accurate models of real user behavior
Meaningful scale & appropriate concurrency
Control to start, stop & modify tests without delays
Test on your schedule
Modern Load Testing
Test for spikes – using the cloud
![Page 43: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/43.jpg)
Failover Scenarios Success Disasters End to End
Test what you don’t think needs to be tested
Modern Load Testing
![Page 44: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/44.jpg)
Measure, Test and Act in Real Time
Modern Load Testing
![Page 45: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/45.jpg)
Don’t Wait until it’s too late
Compress processes for more test iterations
Traditional Approach
TIME
Test development Infra. setup Fix Repeat?
RetestTest Analyze
Modern Load Testing
![Page 46: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/46.jpg)
Don’t Wait until it’s too late
Compress processes for more test iterations
/
Traditional Approach
TIME
Test development Infra. setup Fix Repeat?
RetestTest Analyze
CloudTest
Modern Load Testing
![Page 47: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/47.jpg)
Don’t Wait until it’s too late
Compress processes for more test iterations
/
Traditional Approach
TIME
Test development Infra. setup Fix Repeat?
RetestTest Analyze
/ / / / Extra time!CloudTest
Modern Load Testing
![Page 48: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/48.jpg)
Use real user data toiterate dev & test plans
Run iterative load tests via Jenkins
Daily (or more frequent) updates keep teams quality focused
Embedded failure details & bottleneck data isolate issues
Manage to a performance
baseline
Mobile? Run functional automation on real devices for functional & performance validation
Spin up suitable, consistent test environments in the
cloud or the lab
I JUST MADE A CODE CHANGE
Jenkins runs any unit tests as usual
Test Continuously – including performance!5. Don’t Wait
![Page 49: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/49.jpg)
s
Memory leaks
Test Scale
Sta
ge /
Tea
m
Dev & Test
Release& Deploy
Network& Ops
CDN file placement
Load Balancer configuration
Network bandwidth
Network configuration
DNS routing
Inadequate server resources
Default configuration settings
Unbalanced web serversAuto-scaling failures
Latency between systems
Slow third-party plug-ins
Garbage collection
Database thread counts
Inefficient database queriesSlow pages
Conflict with other apps
Test Lab Staging Production (100% +++)
Search technology limits
Method-level tuning
Max sockets exceeded
Firewall max capacity
Global latency variance
Security bottlenecks
Continuous tuning for full-scale…and well beyond
Start early, end in the real environment
6. Test in production and at Production scale
![Page 50: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/50.jpg)
![Page 51: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/51.jpg)
• Tests for flash sales with unknown spikes
• mPulse analyzes real user performance
• Continuous tests of mobile and web traffic
“SOASTA was very easy to work with and could accommodate our requirements in a four-week
timeframe.” Gavin Cliffe, OZSale CTO.
OZSale
![Page 52: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/52.jpg)
NO ISSUES
“SOASTA was very easy to work with and could accommodate our requirements in a four-week
timeframe.” Gavin Cliffe, OZSale CTO.
OZSale
160% more visitors
$5M AUD in one sale
50% mobile
![Page 53: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/53.jpg)
Critical Factors for Performance Testing Retail Sites
Test for spikes Test what you think
doesn’t need to be tested
Test expecting failure Analyze & act in real
time
Don’t forget about the user experience
Monitor everything Don’t wait until just
before the holidays Test in production
![Page 54: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/54.jpg)
Marketing
![Page 55: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/55.jpg)
Communicate your campaign plan internally and with stakeholders
Establish a campaign change process
Approval
Communication
Campaign Management
![Page 56: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/56.jpg)
Establish a change freeze on application and infrastructure
Limit publishing and catalogue changes
Campaign Management
![Page 57: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/57.jpg)
Campaign Best Practices
![Page 58: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/58.jpg)
Segment your list into smaller groups
Trickle your emailManage your media content
Email Best Practices
![Page 59: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/59.jpg)
Don’t get too fancyDevelop test cases
for all of your discount codes
Promotions and Discounts
![Page 60: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/60.jpg)
Avoid using discount structures that require a database search
Don’t use new discount structures
Promotions and Discounts
![Page 61: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/61.jpg)
TEST, TEST, TEST!
Promotions and Discounts
![Page 62: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/62.jpg)
Are you interested in a FREE Retail Readiness Performance Test
![Page 63: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/63.jpg)
Contact SOASTA:[email protected]
866.344.8766Follow us:
Twitter.com/cloudtestFacebook.com/cloudtest
Retail Readiness Page: soasta.com/retail-readiness/ Case studies: soasta.com/customers/ecommerce/
Webinars future and past: soasta.com/knowledge-center/webinars/
CloudTest Lite is FREE (forever)!soasta.com/FREE
Resources from SOASTA
Thank you
![Page 64: Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness](https://reader038.vdocuments.us/reader038/viewer/2022110203/55d08546bb61ebba7d8b4803/html5/thumbnails/64.jpg)
www.SOASTA.com
more than managed hosting. commerce anywhere.www.tenzing.com