production performance testing in the cloud
DESCRIPTION
Testing in production for online applications has evolved into a critical component of successful performance testing strategies. Dan Bartow explains the fundamentals of cloud computing, its application to full-scale performance validation, and the practices and techniques needed to design and execute a successful testing-in-production strategy. Drawing on his experiences, Dan describes the methodology he has used for testing numerous online applications in a production environment with minimal disruption. He explains how to create a performance testing strategy to give your team critical data about how your online application performs and scales. Learn how to create a robust lab-to-production ecosystem that delivers the answers about what will happen when peak traffic hits your site. Take back practical approaches to mitigate the three most common problems—security, test data, and potential live customer impact—that arise when embarking on testing in production.TRANSCRIPT
TK PM Tutorial
10/1/2013 1:00:00 PM
"Production Performance
Testing in the Cloud"
Presented by:
Dan Bartow
SOASTA
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Dan Bartow
SOASTA, Inc.
At SOASTA, Dan Bartow is Vice President of Product Management for the industry leading and
award-winning product CloudTest. Prior to joining SOASTA, Dan was Senior Manager of
Engineering at Intuit where his team was responsible for the speed and stability of TurboTax
Online. During the past decade he has been responsible for the performance of websites for
dozens of leading consumer brands.
8/27/2013
1
Create a One-Page Capacity
Model for High-Traffic Web
Applications
Dan Bartow
SOASTA
About the Speaker
SOASTAVP Product Management
CloudTest Evangelist
IntuitSr. Manger, Engineering
TurboTax Online and E-com
ATGSr. Deployment Engineer
American Airlines, Best Buy, Target, Turbotax Online, QuickenOnline, MySpace,
Dennys, Dominos, Mattel, Hallmark, FAA, US Army, AT&T Wireless, Alcatel,
Newsweek, Oprah, NeimanMarcus, SBC, Plantronics, Kodak, JCrew, Cingular,
Newell Rubbermaid
Work
8/27/2013
2
Poor performance.
The Problems
1. No knowledge of what the critical metrics are at each tier of an applications architecture
1. No record of what the critical metric values were at the peak last year
1. No idea of what those metrics should be at the next peak traffic day
1. No indication of what the capacity is today and how that relates to the upcoming peak
8/27/2013
3
The Solution
The one page capacity model
(or at least a huge step in the right direction)
What one looks like
8/27/2013
4
How to build it
Step 1.
Create a simple architecture diagram, containing mostly infrastructure, that lists key technologies in play for each tier
You don’t need this
8/27/2013
5
Getting closer7
What to include in the diagram
1. Key infrastructure components
2. Critical services and their infrastructure where possible
3. Third party components (if applicable)
Hint: If you can’t test it, you probably don’t need to include it. #controversial
8/27/2013
6
A reasonable diagram
How to build it
Step 2.
Identify the metrics that should be tracked for each tier
8/27/2013
7
Three layers of monitoring
Layer 1 - Customer
Layer 2 - Server
Layer 3 - Container
Concurrent sessions
Pages (requests) / sec
CPU %
Memory
Network IO
Disk IO
JVM Heap usage
Threads / second
Login servlet invocations / sec
Database connections
To make this all happen (and be
really useful)
You need comprehensive monitoring
You need to be able able to test to scale
8/27/2013
8
Metrics examples for common
technologies
Metric examples for common
technologiesCPU
Memory
TX/sec (especially SSL TX)
Bandwidth/sec
CPU
Memory
Threads / sec
Container
CPU
Memory
Threads or processes / sec
Container
CPU
Memory
JVM heap usage
Garbage collection interval
Threads / sec
Application
Key servlet invocation count (login, order
placement)
Database connection count
Key servlet execution time
Application
Key GET/POST actions
Key method calls
External connections
Container
CPU
Memory
Connections
Queries / sec
(SELECT/INSERT/UPDATE/
DELETE)
Table or row locks
8/27/2013
9
How to build it
Step 3.
Fill in the values for your peak last year (the peak second)
How to build it
Step 4.
Project and do simple math for the upcoming year. Base
this math on top level percentages and honor relationships
between metrics where needed.
8/27/2013
10
Contact Information
Dan BartowVP Product Management
SOASTA Inc.
Email: [email protected]
Twitter: www.twitter.com/PerfDan
LinkedIn: www.linkedin.com/in/danbartow/
Blog: www.soasta.com