opens ta load testing pr

8/3/2019 Opens Ta Load Testing Pr

1/27

SpecialTopics

Open Source Tools for WebApplication Performance Testing

Open Source Tools for WebApplication Performance Testing

Dan Downing, VP Testing Services

MENTORAGROUP

Atlanta Boston DC

www.mentora.com


2/27

Copyright 2006 Mentora Group Inc.

2

Objectives

Present the basics of using OpenSTA for loadtesting web apps.

Things you will learn:

Capabilities of OpenSTA Understand how to interpret its performance data

Detect and repair performance bottlenecks


3/27


3

Terms & Concepts

History

Develop by Cyrano, a French software company Released into public domain in 2001

Collector: System resource monitor script Commander: Main UI and test launcher

Modeling: Script development SCL: Script Control Language Task Group: Load scenario script or Collector script Test: Set of scripts comprising load scenario and related results

Timers: Start-Stop transaction timing markers for actions to be timed Variables: Script Parameters
http://opensta.org/http://opensta.org/


4/27


4

Capabilities

Applications it will test Protocols: Standard http/https only (notOracle 11i, SAP, RMI, web services) Development technologies: D/HTML, javascript, XML payloads, jvm servers (notjava

applets, latest technologies) Functionality (LoadRunner = 10)

Script development: 7/10 (Handles sessionid cookies; UI intuitivess ok; SCLlanguage support good)

Test/Debug: 6/10 (Need error-handling and content checks to ensure scriptfunctioning properly)

Test execution: 7/10 (slow refresh on summary stats Results analysis: 3/10 (built-in graphs limited; need to do much in Excel)

Robustness Excellent thread model; spawns a new thread only when it needs it to drive more load Sometimes hangs, crashes (must manually clean up processes and sometimes reboot) Conflicts with other load testing tools listener processes (run only one at a time)

Limitations Launches max of 1664 users per Commander session Watch out for

Trip-ups with mix of http and https pages Too many parameter files can create file open bottleneck, throttling load Code your own error handling of http 400 and 500 errors

That said, it works great on many web apps!
http://opensta.org/


5/27


5

Download & Install

V.1.4.3 12/04 iscurrent release

5.7 MB download

Windows installer

Runs on XP, W2K,NT SP5

Installer (usually)

runs flawlessly


6/27Copyright 2006 Mentora Group Inc.

6

Components & Process

Modeler

Commander

Excel

Step 2: Define & Launch Test

Define load scenario; configure Collectors; launch test;monitor interactively

Step 3: Export, Graph, & Analyze Results

Select & filter results; export; generate graphs; analyze& interpret.

1

2

3

OpenSTA

Step 1: Develop test scripts

Record, parameterize user-entered data, parameterize dynamicsystem data, adjust think-times & timers, test/debug

NameServer*

* Listener process that communicates betweenCommander, load drivers, & target system



7

Architecture

SD

SunE N T E R P R I S E

4 0 0 0

DRIVENULTRASPARC

Load Driving System

System Under Test

OpenSTA

Local load

driver(s)

Remote load driver(s)

Web Relay Daemon (opt.)

Name Server



8

Directory Structure

Base directory; location pointed to by Tools/Repository Path

Main applications DLLs

More DLLs, exes, ini files

More DLLs

Test assets (by project)

Screen shots

Data files

Compiled scripts

Collector scripts

Scripts (source SCL code)

Tests

Load scenarios

Test results folders



9

Architecture: Processes

ArchMgrDmn.exe Controls remote load drivers

CyrDmn.exe Listener daemon (started by Name Server) DaemonCFG.exe -- Name Server configuration utility

OSComnander.exe Main Commander UI

TestManager.exe Overall test execution manager

TestPlugin,exe Test configuration plug-in TExecuter_htp.exe Runtime engine that runs task groups

TExecturer_ntp.exe Runtime engine that runs Collectors

TModeller_Web.exe Script Modeler UI



10

Script Modeler

Scripteditor

pane

Menu barToolbar

Htmlheader,

bodypanes

Replayoutputpane



11

Modeling Scripts: Cookies

Defined automatically during Record

Handled automatically during replay..



12

Modeling Scripts:

Parameterizing User-Entered Data

Define File variable with proper scope

Substitute user-entered static data witha variable; do global edit

.

.

.

.

.

. Process file values

MUTEX

NEXT

SET

Display value to ensure its working

LOG



13

Modeling Scripts:Capturing System Response Values

.

.

.

Load Response Header/Body into a buffer(character array)

Parse desired value and load into a variable Do global replace so it gets fed back on

later urls



14

Commander

Repository

Tree Viewpane

Runtimeoptionspanel

Test start/stop ToolbarMenu bar

Test Configuration tab


15/27


15

Commander: Monitoring a Test

Execution

errorwindow

Userrampgraph

Executionsummary

pane

Test start/stop Toolbar

Monitoring tab


16/27


16

Commander: Exporting Results

httpresponse

list

Audit log

Timers list

Test start/stop Toolbar

Results tab


17/27


17

Exporting Bandwidth

Bandwidthgraph


18/27


18

Graphing & Interpreting:

Scalability

All workflowsscale well

though 60users,degradingexponentially

above that level


19/27


19


Page Times

Sharp degradationof "Display AllStudies" indicates

this page is themajor contributorto RSD processdegradation


20/27


20


Bandwidth UtilizationBandwidth Utilization

0

100

200

300

400

500

600

1 2 3 4 5 6 7 8 910 11121314 1516171819202122232425262728293031 32333435363738 39404142 434445

Test Session Time

Load(concurrentusers)

0

2

4

6

8

10

12

Mbps

Bandwidth rampssteadily androughly plateauscharacteristicallyat maximum load

Average bandwidthutilization is 6.8Megabits per second

Bandwidth linesshape indicates nonetwork throughputbottleneck

Average bandwidthutilization is ~7 Mbps


21/27


21

Graphing & Interpreting:Application Errors

http server errorsbegin above 120concurrent users

Error spikes indicate

unstable applicationwell before targetload

HTTP Error Responses

0

100

200

300

400

500

600

Test Session Time (min)

Load

(co

ncurrentusers)

0

50

100

150

200

250

Errors

Concurrent Users 20 60 120 175 235 295 350 410 453 477 497 495 495 492 488 487 486 485 485 484 479 479 478

HTTP 5xx Errors 0 1 1 80 94 178 33 134 1 88 46 230 1 1 211 71 158 92 146 88 1 99 19

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44

Stablerange

Application unstableabove 120 concurrent

users


22/27


22

Advanced Topics

Development standards

Naming conventions for Scripts, Timers, Tests Results directory structure

Graphing results Pivot tables Creating & customizing 2 axis charts

Advanced script modeling Response parsing functions for capturing system dynamic values Parameter file management with ACQUIRE MUTEX Randomizing think times with GENERATE Error handling with ON ERROR

Handling state ids in redirects with EXTRACT Debug statements

OpenSTA init settings Timeout Max Ports


23/27


23

Resources: On-Line Help

Make it yourfriend!

Can get all thebasics and more


24/27


24

Resources: FAQs

Many beyond basichow-tos in theseFAQs

Many forum

questions from newusers direct youback to the FAQs


25/27


25

Resources: OpenSTA Forum

Very active forum; email-based

Quick responses to questions

Anyone can sign up, read, submit posts

Many OpenSTA experts respond


26/27


26

Resources: Other

OpenSTA Tutorial On-line Course

http://mentora.com/training.htm

Load testing presentations http://www.mentora.com/library.htm

Load testing blogs www.testingreflections.com Antony Marcano www.exploratorytesting.com James Bach


27/27


27

Questions?

[email protected]

www.mentora.com

opens ta load testing pr

Documents