simulation of systems development processes – why, what ... · process management system...
TRANSCRIPT
![Page 1: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/1.jpg)
Simulation of Systems Development Processes – Why, What and How?
LUCAS Breakfast Presentation – 20 April 2011 Dietmar Pfahl
![Page 2: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/2.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 1
Content
1. Background 2. Current Research
• SPS* – Why? • SPS – What? • SPS – How?
3. Future Research Goals • VSEL**
*SPS = Software Process Simulation **VSEL = Virtual Software Engineering Laboratory
![Page 3: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/3.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 3
My Background – Industry and Academia
Siemens AG (1987-90, 1993-96) German Aerospace Research Est. (1992)
Fraunhofer IESE (1996-2005)
University of Calgary (2005-2007, adjunct since 2008)
Simula Research Laboratory (2008-09) University of Oslo (2008-10)
![Page 4: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/4.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 4
My Background – Applied Research
Process Management System Development System
Requirements Analysis
System Design
SW-/HW-- Requirements
Analysis
SW Architecture
SW Design
Coding
SW Integration
System Integration
Deployment
Project Management
Quality Management
Product Management
![Page 5: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/5.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 11
My Background – Applied Research
Process Management System Development System
Requirements Analysis
System Design
SW-/HW-- Requirements
Analysis
SW Architecture
SW Design
Coding
SW Integration
System Integration
Deployment
Project Management
Quality Management
Product Management
Software process simulation (Siemens, DaimlerChrysler, Bosch, Chartwell, 1994-95, 1997-98, 2004, 2005, 2008-10)
![Page 6: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/6.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 12
Content
1. Background 2. Current Research
• SPS* – Why? • SPS – What? • SPS – How?
3. Future Research Goals • VSEL**
*SPS = Software Process Simulation **VSEL = Virtual Software Engineering Laboratory
![Page 7: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/7.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 13
SPS Models – Why?
It’s standard in other engineering disciplines:
• Industrial building processes • Product engineering
processes • Chemical engineering
processes • Pharmaceutical engineering
processes • Hydrological engineering
processes • Manufacturing processes • Biological/Ecological
processes • Flight simulators • Aeronautics • ….
![Page 8: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/8.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 14
SPS Models – Why?
• To understand – …the interesting characteristics of an existing or desired (complex,
socio-technical) development system (process, project) – SPS models as analysis tools (e.g., for risk analysis)
• To predict – …the interesting characteristics of the system (process, project) by
analysing its model’s performance – SPS models as planning tools
• To evaluate – …the properties of alternative systems (process, project) by comparing
its models’ performances – SPS models as planning tools
• To communicate – …the design and functioning of the system (process, project) to others – SPS models as documentation and education tools
![Page 9: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/9.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 15
Content
1. Background 2. Current Research
• SPS* – Why? • SPS – What? • SPS – How?
3. Future Research Goals • VSEL**
*SPS = Software Process Simulation **VSEL = Virtual Software Engineering Laboratory
![Page 10: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/10.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 16
SPS Models – What?
• Operational Release Planning (ORP)
– Example of using SPS for planning, risk analysis, and decision-making – Complements existing optimization models for Strategic Release
Planning (à DECIDERelease)
Reasons for ORP re-planning: - Drop in/out of developers - Addition/deletion of features - Adjustment of effort estimates - Adjustment of productivity
estimates - Adjustment of task dependencies All the above … … in any combination … at any point in time during
release (repeatedly)
![Page 11: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/11.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 17
SPS Models – What?
• ORP: Planning and Re-Planning in the context of SRP
![Page 12: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/12.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 19
SPS Models – What?
• Re-Planning Example (enlarged):
Developer D4 becomes
unavailable at the beginning
of week 10
![Page 13: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/13.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 20
SPS Models – What?
• Operational Release Planning – Example of using SPS for planning, risk analysis, and decision-making – Complements existing optimization models for Strategic Release
Planning (DECIDERelease)
Histogram (Spreadsheet1 1v*50c)
Var1 = 50*0.5*normal(x, 24.5413, 1.4342)
20.5 21.0 21.5 22.0 22.5 23.0 23.5 24.0 24.5 25.0 25.5 26.0 26.5 27.0 27.5 28.0 28.5
Var1
0
2
4
6
8
10
12
No
of o
bs
Sampling of input parameter values from probability distributions
Monte-Carlo & Process Simulation
Distributions of
output parameter values Histogram (Spreadsheet1 1v*50c)
Var1 = 50*0.5*normal(x, 24.5413, 1.4342)
20.5 21.0 21.5 22.0 22.5 23.0 23.5 24.0 24.5 25.0 25.5 26.0 26.5 27.0 27.5 28.0 28.5
Var1
0
2
4
6
8
10
12
No
of o
bs
![Page 14: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/14.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 21
SPS Models – What?
• Risk Analysis for Release Planning To analyze impact of task effort and developer productivity over/under-estimation on
– Release Duration – Developer allocation to tasks – Task scheduling:
→ Start and end times of tasks
new-Baseline100case0new-Baseline100case0-old
"Alloc-F-T-D"[Feature,Task,D1]
0 7.5 15 22.5 30Time (Week)
F1,T1F1,T2F1,T3F2,T1F2,T2F2,T3F3,T1F3,T2F3,T3F4,T1F4,T2F4,T3F5,T1F5,T2F5,T3F6,T1F6,T2F6,T3F7,T1F7,T2F7,T3F8,T1F8,T2F8,T3
![Page 15: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/15.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 22
• Baseline Case • Triangular Distributions
SPS Models – What?
![Page 16: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/16.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 23
• Effect on Duration • Case 9: Distribution of Durations Histogram (Spreadsheet1 1v*50c)
Var1 = 50*0.5*normal(x, 24.5413, 1.4342)
20.5 21.0 21.5 22.0 22.5 23.0 23.5 24.0 24.5 25.0 25.5 26.0 26.5 27.0 27.5 28.0 28.5
Var1
0
2
4
6
8
10
12
No
of o
bs
N(24.54, 1.43)
SPS Models – What?
![Page 17: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/17.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 24
• Effect on Duration • Single Sample T-Test
?
alpha = 0.05
SPS Models – What?
![Page 18: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/18.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 25
• Effect on Developer Allocation • Total number of allocations: 8 x 3 = 24
• With 6 developers:
Box & Whisker Plot
Median 25%-75% Min-Max Case 1
Case 2Case 3
Case 4Case 5
Case 6Case 7
Case 8Case 9
Case 10Case 11
Case 12
0
2
4
6
8
10
12
14
16
18
20
22
24
Median: 29% change Median: 58% change
SPS Models – What?
![Page 19: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/19.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 26
SPS Models – What?
• Effect on Task Scheduling
• Formulas:
• Data: Dv_diff ∈ [0, 1] new-Baseline100case0new-Baseline100case0-old
"Alloc-F-T-D"[Feature,Task,D1]
0 7.5 15 22.5 30Time (Week)
F1,T1F1,T2F1,T3F2,T1F2,T2F2,T3F3,T1F3,T2F3,T3F4,T1F4,T2F4,T3F5,T1F5,T2F5,T3F6,T1F6,T2F6,T3F7,T1F7,T2F7,T3F8,T1F8,T2F8,T3
![Page 20: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/20.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 27
Content
1. Background 2. Current Research
• SPS* – Why? • SPS – What? • SPS – How?
3. Future Research Goals • VSEL**
*SPS = Software Process Simulation **VSEL = Virtual Software Engineering Laboratory
![Page 21: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/21.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 28
SPS Models – How?
• A Virtual Software Engineering Laboratory (VSEL) – Exploring the idea of SPS building blocks (macro-patterns) – Currently development of an open source MATLAB/SIMULINK
library under way (à Th. Birkhoelzer, Konstanz) – Example: Composition of SPS representing the V-model
→ E.g., used to investigate the effects of various factors (e.g., quantity and quality of personnel resources, effectiveness of inspection and test activities) on project performance (effort consumption, duration, defect density)
![Page 22: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/22.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 29
SPS Models – How?
• Macro-Pattern 1: Development and Verification
![Page 23: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/23.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 30
SPS Models – How?
• Example Process
V-Model: Development, Verification (Inspection), Validation (Test)
Requirements
Code
System Test
Integration Test
Unit Test
Design
System Level
Subsystem Level
Module Level
V-Model Process Structure
Macro-Pattern 1
![Page 24: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/24.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 32
SPS Models – How?
• Implementation (System Dynamics, MATLAB/SIMULINK)
![Page 25: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/25.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 33
<Policies (Rules)>
33
SPS Models – How?
• GENSIM 2.0 – Generic Simulator
Verreq Devreq ValST
ValIT
ValUT
Verdes Devdes
Vercode Devcode
<Learning>
<Communication>
Requirements Analysis and Specification Design Implementation
ST
IT
UT
Functionality Reliability Usability …
<Skills>
<Techniques>
<Tools>
![Page 26: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/26.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 34
SPS Models – How?
Possible Analyses (list not complete): • What combinations (and intensity levels) of development, verification
and validation techniques should be applied in a given context to achieve defined time, quality or cost goals?
• How do workforce size and skill variations impact project performance (project duration, effort consumption, code quality)?
• Does investment in training pay off for specific development contexts and goals?
• Do investments in improving development, verification, and validation techniques pay off for specific development contexts and goals?
• What are the promising areas of research for improving development, verification, and validation techniques?
![Page 27: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/27.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 35
Content
1. Background 2. Current Research
• SPS* – Why? • SPS – What? • SPS – How?
3. Future Research Goals • VSEL**
*SPS = Software Process Simulation **VSEL = Virtual Software Engineering Laboratory
![Page 28: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/28.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 36
Process Management
Project Management
Quality Management
Product Management
Virtual Software Engineering Laboratory
Dec
isio
n Su
ppor
t
Future Research Goals – Big Picture
System Development System Requirements
Analysis
System Design
SW-/HW-- Requirements
Analysis
SW Architecture
SW Design
Coding
SW Integration
System Integration
Deployment
MBD MBT
![Page 29: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/29.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 37
VSPL – Why/What/How?
Virtual Software Engineering Laboratory (VSEL) • Why
– To provide a test-bed for evaluating issues or new ideas in a variety of contexts
– To support decision-making • What
– To analyse/visualise cost-effectiveness of new development approaches, e.g., model-based development (i.e., model-based testing)
– To analyse effects of local improvements (e.g., better skills for a specific tasks, better effectiveness of a specific inspection or test technique) or local problems (e.g., unclear or incomplete requirements, requirements volatility) on overall project/process/organisational performance
• How – Provision of a library of customizable model components macro-
patterns) – Calibration of customised process simulator based on empirical data and
expert knowledge
![Page 30: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/30.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 38
Tack så mycket! Questions?
![Page 31: Simulation of Systems Development Processes – Why, What ... · Process Management System Development System Requirements ... analysing its modelʼs performance ... F5,T1 F5,T2 F5,T3](https://reader030.vdocuments.us/reader030/viewer/2022032711/5adae4ab7f8b9add658d3d2b/html5/thumbnails/31.jpg)
LUCAS Presentation at Lund University
© Dietmar Pfahl, 2011 Slide 39
Publication Summary (since 1994)
• Books/Proceedings + Book Chapters: 8 + 8 • Journals: 17
– Software Process Improvement & Practice: 5 – Information and Software Technology: 3 – The Journal of Systems and Software: 2 – IJSEKE: 2 – Empirical Software Engineering: 1 – IEEE Transactions on Reliability: 1 – Others: 3
• Conference and Workshop Proceedings: 51 – SEKE (incl. SEDECS): 6 – ProSim (ICSE-Workshop): 6 – PROFES: 5 – ICSP: 4 – ISESE: 3 – METRICS: 3 – ESCOM: 3 – LSO: 3 – Others: 18