performance testing for web applications – techniques, metrics and profiling

Performance Testing for Web Applications Techniques, Metrics and Profiling

Ștefan PopaSoftware Test Automation Specialist

Performance Testing for Web Applications: Techniques, Metrics and ProfilingSlide 4 of 27

Agenda

Techniques Stage the environment Build test assets Test procedure

Metrics analysis System Database

Profiling Ensemble level Application and database level


Performance testing for web applications1. Techniques

Stage the environment: Deploy testbed

Eliminate deployment issues

Build tests assets: Analyze client data to develop an approach strategy

Build scenarios

Test procedure: Manual & automated tests

Automate data collecting and processing


Stage the environment: Deploy testbed

Administrator / Management

Database

Payment Gateway

Frontend

Backend

Users

Create a testbed similar to client’s production


Stage the environment: Eliminate deployment issues

Client’s working infrastructure

Application server tuning:

- Network issues- Data compression- Connection pools- Memory allocation

Test DB server tuning:

- Best practices or DBA expertise- Resource allocation for the shared pool- Database parameters


Build test’s assets: Investigate logs and database

Analyze client data Investigate logs Investigate database

Build scenarios


Build test’s assets: Analyze client data to develop an approach strategy

Analyze client’s previous product logs or establish new possible scenarios based on existing clients


Build test’s assets: Build the scenarios

Create the actions

Set the goals(expected latencies)


Build test’s assets: Manual performance tests

Manual testing: Finding issues/bugs with minimal effort

Have a reference for automated tests


Build test’s assets: Manual performance tests: results

1 Login 1.42 0.68 0.592 Press "Existing customer" button. 0.33 0.23 0.313 Enter the account's Billing Name. Click the "Find" button 0.29 0.17 0.034 Click the "Continue" button. 1.88 1.53 1.125 Click the button "Update Resources". 0.16 0.21 0.036 Select a value from the "Service Number" drop down list. 0.68 0.78 0.437 Insert an available "IMEI“. Click the "Retrieve" button. 0.45 0.5 0.218 Press add to cart 2.58 2.56 1.859 Click the "Service payments" button. 0.49 0.17 0.17

10 Fill in a value for "Amount to pay” Press "Add to cart" 1.66 1.37 1.2111 Press "Check Out" button. 1.75 1.06 0.7812 Press "Cash" button. 0.33 0.2 0.1713 Press "Submit" 3.50 2.81 2.1714 Press "Finish Sale" 0.23 0.12 0.06

Test RUN ID 1 2 3Test Tool Manual - Fiddler Manual - Fiddler Manual - Fiddler# of users 1 1 1# of cycles 10 10 10


Build test’s assets: Develop an automated solution for testingTransposing the manual scenario into an automated script


Build test’s assets: Calibrate testsValidate script by running with 1 user as in previous manual run.

Data must be consistent with manual tests

Validate actual results

Differences between manual and automated tests

Test RUN ID 3 4

Test Tool Manual - Fiddler

Automatic - jMeter

# of users 1 1# of cycles 3 10

Login 0.57 0.53

Press "Existing customer" button. 0.03 0.06

Enter the first name/last name Click the "Find" button 0.07 0.08

Click the "Continue" button. 0.7 0.76

Click "Update Service" 0.17 0.17

Select Service Number 1.4 1.36

Change "Rate Plan" 0.51 0.51

Set "Effective Date" (Immediate) 0.1 0.1

Add to cart. 2.17 2.03

Click Check Out 0.75 0.91

Press "Check" 0.29 0.31

Submit Check number and press Submit 2.51 2.55

Press Close & Finish sale button. 0.35 0.15


Test runs, analyze resultsDetect low load performance issues

Raise limit to client’s expected load

Run ID 5 6 7 8 9 10 11 Test Tool Automatic

(jmeter)Automatic

(jmeter)Automatic

(jmeter)Automatic

(jmeter)Automatic

(jmeter)Automatic

(jmeter)Automatic (jmeter)

# of users 1 10 20 50 50 50 100

1 Open OnlineStore main page. 0.62 0.16 0.94 0.14 0.08 0.07 0.08

2 Sort by provider: LG 0.07 0.04 0.46 0.13 0.06 0.05 0.05

3 Select a preconfigured phone model with promo 0.42 0.35 1.38 6.8 3.1 0.08 0.09

4 Add to cart the phone. 0.03 0.02 0.25 0.08 0.05 0.01 0.02

5 Enter a valid ZIP code and click the Submit button. 1.41 1.52 2.97 13.96 0.05 0.87 0.94

6 Click the Continue button. 0.82 0.88 2.58 6.21 1.24 0.6 0.63

7 Click on the "Postpaid plan type" hyperlink. 0.06 0.03 0.49 0.04 1.16 0.03 0.03

8 Select a rate plan. 0.02 0.01 0.37 0.02 0.01 0.02 0.03

9 Acknowledge age restrictions 0.05 0.03 0.25 0.05 0.02 0.03 0.04

10 Click the "Add to cart" button. 0.3 0.37 1.14 7.14 2.67 0.06 0.07

11 Skip add to cart an accessory. 0.06 0.06 0.4 0.1 0.02 0.04 0.04

12 Click the Check Out button. 0.05 0.03 0.53 0.05 0.03 0.07 0.08

13 Select New Account 0.42 0.81 1.69 4.06 0.71 0.32 0.37

14 Enter new user information. Press Continue Checkout 0.49 0.98 2.15 3.76 0.59 0.35 0.37

15 Enter Account Owner Info 0.38 0.44 1.32 4.7 0.24 0.24 0.25

16 Select Personal address or Postal Service address 1.32 4.6 9.44 47.06 28.18 7.46 11.04

17 Verify Cart 0.05 0.03 0.32 0.12 0.02 0.03 0.03

18 Select Subscriber info and select “new number" 0.17 0.24 0.8 5.54 0.19 1.47 1.62

19 Select Personal address or Postal Service address 1.99 3.17 5.75 10.34 1.91 0.07 0.08

20 Select Shipping Details & select Shipping method 0.5 1.09 2.24 5.31 0.99 0.66 0.87

21 Review cart and confirm Checkout 1.43 2.63 5.79 6.42 3.89 0.04 0.05

22 Agree License and Contract 3.23 3.54 4.87 4.76 0.41 0.33 0.33

23 Enter & submit credit card details. 2.21 5.64 9.53 27.89 6.67 3.01 2.88

24 Order Complete. Press Continue Shopping 0.11 0.09 0.8 0.14 0.06 0.05 0.06

25 Main page loads. Press logout 0.09 0.1 0.52 0.12 0.05 0.06 0.06


Performance testing for web applications2. Metrics analysis

Automatic data gathering & processing

Database analysis

System analysis


Metrics analysis: Automate collecting and processing test data


Metrics analysis: Analyze database issues


Metrics analysis: Analyze application server issues


Metrics analysis: Analyze network issues

Analyze system’s working parameters to help building system requirements.


Performance testing for web applications3. Profiling

System-wide load distribution

Database

Application


Profiling: System-wide load distribution analysis

Performance profiling of staging environment’s overview

Use of APM tools

Eliminate multiple test runs


Profiling: Database profiling

Workload summary & CPU load


Profiling: Database profilingSQL query analysis


Profiling: Database profilingExecution plan analysis


Profiling: Application profilingBreaking down the problems

Finding atomic method’s elapsed time to reduce application server load


Performance tests: Publish the results after multiple iterations Run ID 30 31 66 68 72 73 # of users 1 10 10 50 50 50

1 Open OnlineStore main page. 0.98 0.3 0.09 0.07 0.07 0.06

2 Sort by provider: LG 0.52 0.28 0.05 0.05 0.04 0.05

3 Select a preconfigured phone model with promo 1.38 1.18 0.09 0.08 0.08 0.08

4 Add to cart the phone. 0.15 0.11 0.01 0.01 0.01 0.01

5 Enter a valid ZIP code (57722) and click the Submit button. 1.19 1.37 1.28 0.87 0.85 0.91

6 Click the Continue button. 0.88 1.04 0.82 0.6 0.58 0.61

7 Click on the "Postpaid plan type" hyperlink. 0.66 0.27 0.03 0.03 0.02 0.04

8 Select a rate plan. 0.07 0.07 0.02 0.02 0.02 0.02

9 Acknowledge age restrictions 0.66 0.28 0.03 0.03 0.03 0.02

10 Click the "Add to cart" button. 1.27 0.97 0.06 0.06 0.06 0.07

11 Skip add to cart an accessory. 0.19 0.24 0.04 0.04 0.04 0.04

12 Click the Check Out button. 0.2 0.14 0.07 0.07 0.07 0.07

13 Select New Account 0.49 0.56 0.28 0.32 0.29 0.3

14 Enter new user information. Press Continue Checkout 0.84 0.86 0.34 0.35 0.34 0.35

15 Enter Account Owner Info 0.47 0.9 0.2 0.24 0.19 0.35

16 Select Personal address or Postal Service address 9.16 9.33 6.65 7.46 5.41 3.38

17 Verify Cart 0.24 0.18 0.03 0.03 0.03 0.04

18 Select Subscriber info and select "Give me an Viaero number" 0.49 0.5 1.34 1.47 1.37 0.49

19 Select Personal address or Postal Service address 2.79 2.24 0.07 0.07 0.07 0.07

20 Select Shipping Details & select Shipping method 0.95 1.05 0.62 0.66 0.61 0.74

21 Review cart and confirm Checkout 1.77 1.92 0.04 0.04 0.04 0.04

22 Agree License and Contract 0.72 0.71 0.29 0.33 0.3 0.42

23 Enter & submit credit card details. 4.73 5.26 3.01 3.01 2.73 2.15

24 Order Complete. Press Continue Shopping 0.37 0.24 0.06 0.05 0.05 0.05

25 Site Loads. Press logout 0.13 0.26 0.07 0.06 0.06 0.04


Ștefan Popa - Software Test Automation Specialist

performance testing for web applications – techniques, metrics and profiling

Technology

web applications techniques

processing performance

latenciesperformance

agenda techniques

manual automated tests

press cash button

retrieve button

profiling1 manual fiddler