build your custom performance testing framework

T17 Performance Testing

10/16/2014 1:30:00 PM

Build Your Custom Performance Testing Framework

Presented by:

Prashant Suri

Rackspace

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Prashant Suri

Rackspace Prashant Suri is a senior member of the performance engineering team at Rackspace, the open cloud company. In his role, he helps performance test and benchmark Rackspace's cloud products and applications, writing his own test frameworks to help. In his early years, Prashant developed a number of products on Java/J2EE stack but was fascinated by the vastness of performance engineering. He enjoys improving products’ scaling and robustness. Through the years, he has helped products, developed and used by Fortune 500, scale better. A presenter at international testing conferences, Prashant can be reached at [email protected].

Build your custom Performance Testing Framework Prashant Suri

RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Agenda

2

• Evolution of the web and growing complexity of products. • Essentials of performance engineering. • Popular performance test tools. • Holistic performance test framework. • Components of a performance test framework. • Deep dive on an example: performance testing at RAX • Advantages of building your own framework.


Evolution of the Web

3


Essentials of Performance Engineering

• Performance objectives – Latency, Throughput, Utilization, Workload.

• Performance Modeling – Scenarios, Objectives, Requirements, Budgets, Metrics.

• Architecture and Design Guidelines – Principles, Practices and Patterns

• Performance and Scalability Frame – Coupling and cohesion, Communication, Concurrency, Resource Management, Caching, State Management, Data Structures/Algorithms.

• Measurement, Tuning and Analysis.

4


Popular Performance Test Tools

5


There’s No Silver Bullet

6

• Performance testing is more than simulating load. • Accurate and intricate work load models need to be implemented.

• Traffic to be simulated from world geographies. • Testing in service oriented architectures comes with its own challenges.

• Reporting and pattern analysis is key for successful performance engineering.

• Performance monitors and profiling equally important.


Holistic Performance Test Framework

7


Components of the framework

8

Controller Agents Database & Graphing

Engine

Sim

ulat

ing

Load

s

•  Brain behind the complete orchestration. Parses and executes the data script.

•  Picks up threads from queues, and hit application under test.

•  Repository for metrics of response times, throughputs, errors etc.

•  Can scale dynamically at run time, to add load generation capacity.

•  Maintains the test details, workload model details and objectives of any performance test.

Capable of distributing load from different data centers.

•  Distributed across geographies to simulate identical world traffic.

•  Optimal database tuning for handling 1000+ transactions per second test.

Mon

itorin

g an

d D

ebug

ging

Could be integrated to application monitors and profilers.

•  Feeds all metrics data into the Database.

•  Correlates and consolidates all test data.

Updated information on runtime activity from database is read and published.

•  Logging for every granular transaction with application under test.

•  Historical comparisons and pattern analysis.

Could be leveraged for analysis and data based decision making.

•  Distributed agents through way of logging are consolidated for complete test.

•  Profiling and application under test data, for analyzing deviation and anomalies.


Adobe CQ Microsoft® SharePoint Microsoft® Exchange Rackspace Email

Case Study – Capabilities galore! Single F/W to simulate on all Products


Sample Script – Design workload models!

10

RACKSPACE® HOSTING | WWW.RACKSPACE.COM 11

Execution – Distribute across Agents through a queue

RACKSPACE® HOSTING | WWW.RACKSPACE.COM 12

Run time statistics of Performance Test


Consolidated Reports

13


Consolidated Reports

14


Advantages of Building your own frameworks

15

• Cost savings on commercial off-the-shelf tools. • Flexibility to simulate complex workload at any scale (literally!)

• Simplicity in designing complex workload models. • No constraints and capability of leveraging as per product needs.

• Ease of integration with CI/CD Pipeline. • Load simulator and application resource monitors in a single framework setup.

• Efficient and inclusive engagement for Performance Engineering in overall product development lifecycle.

RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM

RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM

Questions…

build your custom performance testing framework

Technology