e software quality - university college londoncrest.cs.ucl.ac.uk/cow/15/slides/stefanwagner.pdf ·...
TRANSCRIPT
www.uni-stuttgart.de
PredictionSoftware Quality
Stefan Wagner
The 15th CREST Open Workshop25 October 2011
London, UK
"Quality is a complex and multi-faceted concept... it is also the source of great confusion."
–David A. Garvin
I know it when I see it
Quality Model
Functionality
Reliability
Performance
Usability
Portability
Maintainability
ISO 9126
"By the time you figure out you have a quality problem it is probably too late to fix it."
–John S. Reel
Development
Quality assurance
Re
vie
w
Te
st
An
alysi
s
Softwarequality control
Continuous
Deissenboeck, Wagner et al., IEEE Software, 2008
Evaluation
Specification
Qualitymodel
Qualityrequirements
Change requests
Analysis TestModification
MaintenancePr
oduc
t
Comment
Function
Module
Activity
-based
quality m
odel
Deissenboeck, Wagner et al., ICSM'07
Activities
Entities
Impacts
How can we assess and predictquality?
Prediction research concentrates on bug prediction...
1. Scoring approach
The Benchmark for Software Quality
Project partners
Format string injection
Variable manipulation
Embedding scripts
AttackPr
oduc
t
Cookie
Dynamic Web page
Static field
Format string injection
Variable manipulation
Embedding scripts
Attack
CookieDynamic Web
page Static field
Sanitation Sanitation Immutability
Format string injection
Variable manipulation
Embedding scripts
Attack
CookieDynamic Web
page Static field
Sanitation Sanitation Immutability
HTTP cookie formed from untrusted input
Quality Impact Evaluation Specification Language (QIESL)
result = distributeRatio(100, %%Missing destructor%% /%%#Classes%%);
• Java-based syntax• Access to factors and measures• Helper functions• Aggregation, evaluation, calibration
Format string injection
Variable manipulation
Embedding scripts
Attack
CookieDynamic Web
page Static field
Sanitation Sanitation Immutability
HTTP cookie formed from untrusted input
QIESL:result = distributeRatio(
100, %%HTTP cookie formed from untrusted input%% /%%Cookie creations%%);
QIESL:result = 100 - %%Sanitation@Cookie%%;
Format string injection
Variable manipulation
Embedding scripts
Attack
CookieDynamic Web
page Static field
Sanitation Sanitation Immutability
HTTP cookie formed from untrusted input 5 findings
75 points
25 points
Validation on OSS projects
JabRef
TV-Browser
RSSOwl
Log4J
Checkstyle
Rankingfrom model
Rankingfrom experts
Best
Worst JabRef
TV-BrowserRSSOwl
Log4J
Checkstyle
2. Bayesian net
Bayesian net example
# fieldfailures
Low = 0.6High = 0.4
Low HighSmall 0.7 0.1Med 0.2 0.2Large 0.1 0.7
Test effortCode
complexity
Node Probability Table
Format string injection
Variable manipulation
Embedding scripts
AttackPr
oduc
t
Cookie
Dynamic Web page
Static field
Attack
Injection
Format string injection
Resource manipulation
Embedding scripts
Variable Manipulation
Sanitation of dynamic web page
Locality of field
Sanitation of cookie
Vulnerability
density
COS density
DWS density
FDL density
Distribution
Measurement
AgenaRisk: http://www.agenarisk.com
Validation
Goals: gather experiences test predictive validity
Maintainability Security
Wagner, Information and Software Technology, 2010
Tomcat7 – 43 KLOC3 – 6 years
300 KLOC2.5 years
PredictiveValidity
NASA1 NASA2 NASA3 NASA4 Tomcat
70,0
12,1
24,821,7
6,0 6,0
36,1
19,219,415,9
Average change effort in person hours
PredictionObservation
Vulnerabilities per MLOC
Conclusions
• Attempts to assess and predict a broader notion of quality
• Simple scoring approach• Bayesian net approach• Problems
– Missing measures for quality attributes– Missing independent quality assessments for comparisons– Missing data– Aggregation and weighting