blazemeter- effective performance reporting

EFFECTIVE PERFORMANCE REPORTING USING

APACHE JMETER

JULY 31, 2012

THE LOAD TESTING CLOUD

A DEV-TEST CLOUD SERVICE 100%

COMPATIBLE WITH THE OPEN-SOURCE

APACHE JMETER

AGENDA

Performance Attributes

Creating Load Test Reports

Understanding Performance KPIs

JMeter Reporting Elements

BlazeMeter Reporting Plugin

Generating Advanced JMeter Reports

PERFORMANCE ATTRIBUTES

• Speed / Responsiveness

• How fast does the page load?

• How quickly can the system process a transaction?

• Scalability

• Can the application handle the expected end user load?

• Does the application throughput degrade as the user load increases?

PERFORMANCE ATTRIBUTES…

• Efficiency and Capacity Planning

• Are you using the right resources

• Can your infrastructure carry the

load?

• Reliability/Availability/

Recoverability

• What is the mean time between

failure (MTBF)?

• Does the application recover after

a crash? Does it lose user data

after crash?

UNDERSTANDING PERFORMANCE KPIS

Internet

Application Metrics • Response Time • Throughput • Error Rate

Browser Rendering Metrics* • Total Rendering Time • Heavy Images/CSS/JS • DNS Lookup

System Metrics • CPU • Memory • Disk / IO • Network

Platform Metrics • DB • App-server • Application

User Load

Re

sp

on

se

Tim

e

User Load

Re

qu

es

ts / s

ec

End User

Server

UNDERSTANDING PERFORMANCE KPIS…

Throughput

Error

•Measured from the end-user perspective

•Time taken to completely respond to request

•TTLB TTFB

Response Time Web

Server

Inter

net App

Server

DB

Server

DB

Server

Response Time

Total Response Time =

Network latency + Application latency + Browser Rendering Time

•Transactions are specific to applications

•In its simplest form, it is requests / sec

Throughput =

[TRANSACTIONS] / Second

•Defined in terms of the success of the request

•Error at HTTP level (404, 501)

•Application level error

CREATING LOAD TEST REPORTS

Capture Application Metrics • Response Time • Throughput • Errors

Capture Server Metrics • CPU / Memory / Disk / IO • Network • Application • Platform

Tables • Response Time

(avg/min/max/%/stddev) • Throughput (average) • Errors (success % / types)

Graph / Charts • Scatter / Line • Overlay

Correlate Application Metrics • User Load - Response Time • User Load - Throughput • User Load - Errors

Correlate System Metrics • User Load - Server Metrics • User Load - Network • User Load - Platform

Summarize • Overall Performance • Important Trends • Threshold Violations

Trends / Thresholds • Response Time Trends • Throughput Trends • Threshold Violation • Utilization (Server Metrics) Trends

1. Capture

2. Correlate

3. Plot / Tabulate

4. Trends / Thresholds

5. Customize / Summarize

6 . Compare

SAMPLE REPORT ELEMENTS (SNAPSHOTS)

Photo Credits:

• http://msdn.microsoft.com/en-us/library/bb924371.aspx

• Sanitized past projects

JMETER REPORTING ELEMENTS (LISTENERS)

• JMeter Listeners

• JMeter elements that display performance test metrics / output

• Various types of Listeners (Raw / Aggregated / Graphical)

• Doesn’t have inherent capability to measure system metrics*

• Useful for basic analysis

JMeter Report using xslt stylesheet

• Style-sheet under ‘extras’ folder

• .jtl output must be in xml format

– jmeter.save.saveservice.output.for

mat=xml

• Integrate using ant

GENERATING ADVANCED JMETER REPORTS

Other Reporting Options

• JMeter CSV results + Excel

• Process results programmatically

(perl / python etc.)

• BlazeMeter Reporting Plug-in

Photo Credits:

• http://www.programmerplanet.org/pages/projects/jmeter-ant-

task.php

WHAT HAPPENED?

TO LABEL A AND KPI B AT TIME C

BLAZEMETER REPORTING PLUGIN

BENEFITS

• Store a report per test run,

including

• Script that was used to run the

test

• Logs & JTL file

• Compare results of two test runs

• See an improvement trend

• Compare current with previous in

real time

• Share with co-workers

KPIS AVAILABLE IN A JMETER TEST RESPONSE TIME - THE TIME IT TAKES A REQUEST TO FULLY LOAD

• Indicates the performance level of the entire system under test (web server +

DB).

• Represents the average response time during a specific minute of the test.

BLAZEMETER REPORTING PLUGIN COMPARE TWO REPORTS

BlazeMeter - Changing the Economics of Load Testing via the Cloud’

‘BlazeMeter - Code probing, not Angry Birds will define cloud’s success’

‘BlazeMeter - Startup Offers JMeter Cloud Load Testing at Scale’

THANK YOU!

HTTP://BLAZEMETER.COM/