[hcmc stc jan 2015] making it count – agile test metrics
TRANSCRIPT
Row Labels Count of Priority Note:
Blocker 13 until approx. end-May 2014,
Critical 59 Major was Priority default
Major 141
Minor 125
Trivial 17
Grand Total 355
Row Labels Count of Urgency Note 1:
Low [must be if Deferred or Won't Fix] 17 Medium is Urgency default
Medium 196
Soon - High 108 Note 2:
Urgent [must be if Pri=Blocker] 34 24 bugs don't have Urgency,
Grand Total 355 opend before it became
mandatory
4%
16%
40%
35%
5%
Blocker
Critical
Major
Minor
Trivial
Priority
Count of Priority
5%
55%
30%
10%
Low [must be if Deferred orWon't Fix]
Medium
Soon - High
Urgent [must be if Pri=Blocker]
Urgency
Count of Urgency
Note 1:
Some defects,
presumably
created before
it became mandatory,
don’t have a value
Note 2:
Code dev …
is the default for
Fault Source
1%
0% 1%
1%
3%
0%2%
3%
7%
1%
65%
4%
0%4%
0%
5%
2%0%
Resolution, Weighted, by Found During
Cannot Reproduce Story / Systemtest (by QCA)Duplicate CIS internal Acceptancetest (by/for PO)Duplicate Story / System test (byQCA)Duplicate Walkthrough
Fixed CIS internal Acceptance test(by/for PO)Fixed Code review
Fixed Customer Acceptance Test
Fixed Integration test (by QCA)
Fixed Production
Fixed Regression test
Fixed Story / System test (by QCA)
Fixed Walkthrough
Rejected - Test Error Production
Rejected - Test Error Story / Systemtest (by QCA)Won't Fix Regression test
Resolution
Found During
Sum of Weighting
73%
3%
1%
11%
2%
2%
0% 0%
4%
1%
2%
0%
1%
Fault Source, Weighted
Code development
Configuration parameter
Deployment process
No real fault; Bug rejected
Production data
Testing - Data
Testing - Environment
Fault Source
Sum of Weighting
84%
0%
1%
2%
13%
Quality Char., Weighted
Functionality
Integration - interfaces with othersystems
Performance (response orthroughput) / resource efficiency (e.g.
memory leak, CPU or disk usage)
Security (authentication,authorisation, penetration etc.)
Usability (ease of use, appearance,on-line help, user guide etc.)
Quality Characteristic
Sum of Weighting
76%
4%
1% 7%
2% 2%
0%
0%4%
1%
2%0%
1%
Fault Source, Weighted
Code development
Configuration parameter
Deployment process
No real fault; Bug rejected
Production data
Testing - Data
Testing - Environment
(blank)
Fault Source
Sum of Weighting
2%
85%
8%
3% 2% Error Type, Weighted
Ambiguity - spec had >1way to interpret
Inaccuracy - code / spec /parameter value incorrect
No real fault; Bug rejected
Omission - s/thing left outof code / spec / config.
(blank)
Error Type
Sum of Weighting
3% 0%3%
8%
77%
4%1%
4%
Found During, Weighted
CIS internal Acceptance test(by/for PO)Code review
Customer Acceptance Test
Production
Story / System test (by QCA)
Walkthrough
Regression test
Integration test (by QCA)
Found During
Sum of Weighting
1%
8% 0%
43%
4%
4%
3%
3%
1%
5%
5%
2%0%
0%
0%
1%
1%
0%
0%
1% 0%
0%
0%
0%
1%
1%
1%
1%0%
1%
0%0%
1%
1%
0%
0%
0% 1%
Components, Weighted CIPA
Monitoring
Translations
(blank)
Manual Editor
Back Office
Credit Reports
Batch Processing
Statistics
Data Upload
Searching
Administration
Back Office, Matching
Component/sSum of Weighting
Product A Release 1 82%
Product A Release 2 87%
Product A Release 3 84%
Product B Release 1 91%
Product B Release 1 96%
Product C Release 1 86%
Product C Release 2 79%
Product C Release 3 92%
%