predicting reliability of software systems under development
TRANSCRIPT
![Page 1: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/1.jpg)
Predicting reliability of software systems
under development
A multiple case study of large industrial embedded software projects
![Page 2: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/2.jpg)
Support organizations in making decisions with respect to:
–Optimal allocation of test resources
–Asses the release readiness of software under
development
SRGMs: Software Reliability Growth Models
Objectives
Image: http://flylib.com/books/1/428/1/html/2/files/10fig07.gif
![Page 3: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/3.jpg)
• RQ1: Which SRGMs are best to assist decisions for
optimal allocation of testing resources?
• RQ2: Which SRGMs are best for assessing the release
readiness of a software system?
• RQ3: Does using information from earlier projects
improve release readiness assessment?
• RQ4: How to make the choice of SRGM more
effective?
Research Questions
![Page 4: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/4.jpg)
CASE STUDY DESIGN
Company
(unit of analysis)
Application
domain
Software development process for
studied projects
Volvo Cars
CorporationAutomotive
V-shaped software development mostly
using sub-suppliers for implementation
Ericsson Telecom Agile development, mostly in-house
SAAB EDS Defense EquipmentWaterfall development (old projects)
with development concentrated in-house
![Page 5: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/5.jpg)
Software Development Process
![Page 6: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/6.jpg)
SRGMs: Software Reliability Growth Models
No Model Name Shape Structure Mean Value Function Ref.
1 Musa-Okumoto (MO) Concave NHPP 𝑚 𝑡 = 𝑎 ln(1 + 𝑏𝑡) [28]
2 Goel-Okumoto (GO) Concave NHPP 𝑚 𝑡 = 𝑎 (1 − 𝑒−𝑏𝑡) [29]
3 Inflection-S model S-shaped NHPP 𝑚 𝑡 =𝑎 (1 − 𝑒−𝑏𝑡)
(1 + 𝛽𝑒−𝑏𝑡)
[30]
4 Delayed-S model S-shaped NHPP 𝑚 𝑡 = 𝑎 (1 − 1 + 𝑏𝑡 𝑒−𝑏𝑡 ) [31]
5 Rayleigh model S-shaped NHPP 𝑚 𝑡 = 𝑎 (1 − 𝑒−
𝑡𝑏
2
)[32]
6 Logistic model S-shaped Trend 𝑚 𝑡 =𝑎
(1 + 𝑒−𝑏(𝑡−𝑐))[33]
7 Gompertz model S-shaped Trend 𝑚 𝑡 = 𝑎 𝑒−𝑏𝑒−𝑐𝑡 [34]
8 Linear model Linear Trend 𝑚 𝑡 = 𝑔 ∗ 𝑡 + 𝑐 [27]
![Page 7: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/7.jpg)
Metrics used for evaluation
𝑀𝑆𝐸 =1
𝑛
1
𝑛
𝑌𝑖 − 𝑌𝑖2
𝐵𝑃𝑅𝐸 =𝑃𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 − 𝐴𝑐𝑡𝑢𝑎𝑙
𝑃𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 + 2𝜂 ∗ (𝐴𝑐𝑡𝑢𝑎𝑙 − 𝑃𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑);
Mean Square Error (MSE)
Balanced Predicted Relative Error (BPRE)
𝑊ℎ𝑒𝑟𝑒, 𝜂 = 0 𝑖𝑓 𝑃𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 > 𝐴𝑐𝑡𝑢𝑎𝑙1 𝑖𝑓 𝑃𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 < 𝐴𝑐𝑡𝑢𝑎𝑙
Image: http://flylib.com/books/1/428/1/html/2/files/10fig07.gif
![Page 8: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/8.jpg)
a) Which SRGMs are best to assist decisions
for optimal allocation of testing resources?
Data
![Page 9: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/9.jpg)
Metrics used for evaluation
Mean Square Error (MSE) Balanced Predicted Relative Error (BPRE)
Image: http://flylib.com/books/1/428/1/html/2/files/10fig07.gif
RQ1: Which SRGMs are best to assist decisions for optimal allocation of testing resources?
RQ2: Which SRGMs are best for assessing the release readiness of a software system?
RQ3: Does using information from earlier projects improve release readiness assessment?
![Page 10: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/10.jpg)
a) Which SRGMs are best to assist decisions
for optimal allocation of testing resources?
![Page 11: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/11.jpg)
b) Which SRGMs are best for assessing the
release readiness of a software system?
![Page 12: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/12.jpg)
c) Does using information from earlier projects
improve release readiness assessment?
![Page 13: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/13.jpg)
Summary of results
Case unit (domain)
Software
development
process
Observed shape
of defect inflow
profile
Recommended SRGMs
For testing
resource(s)
allocation
For release readiness assessment
Only using current
project data
Using historic
information
1. Automotive V-model S-shape, Concave Logistic Logistic Logistic
2. Telecom Lean + Agile Concave, Convex Gompertz Logistic Musa-Okumoto
3. Defense Equip Waterfall S-shape, Concave Logistic Gompertz Logistic
RQ4: How to make the choice of SRGM more effective?
![Page 14: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/14.jpg)
How to make the choice of SRGM more effective?
![Page 15: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/15.jpg)
How to make the choice of SRGM more effective?
![Page 16: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/16.jpg)
How to make the choice of SRGM more effective?
Projects/
Releases
Defect inflow intensity trend until half-way through the project Predicted
shape of
defect inflow
profile
Overall
trend
Trend after
reaching
maximum
Defect inflow intensity trend characteristics
A1, A3, A4
& C1Increasing Decreasing
Defect inflow intensity first increases, maximizes near
to half-way and then decreasesS-shape
B1, B3 &
B4Decreasing Decreasing
Early defects, defect inflow intensity maximum early
then decreases smoothlyConvex
A2, B2, B5
& C2Increasing Increasing
Late defects, defect inflow intensity trend is positive
throughout half-way of project timelineConcave
![Page 17: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/17.jpg)
How to make the choice of SRGM more effective?
Predicted
shape of
defect inflow
profile
Recommended SRGMs
For testing resource(s)
allocation
For release readiness
assessment using
current project data
S-shape Logistic Logistic
Convex Gompertz Gompertz
Concave Delayed-S Logistic
![Page 18: Predicting reliability of software systems under development](https://reader034.vdocuments.us/reader034/viewer/2022042716/55cef0f4bb61ebed288b4665/html5/thumbnails/18.jpg)
Thank You