simulating for software engineering using systems dynamics jenny longster martin shepperd brunel...

Simulating for Software Engineering using Systems

Dynamics

Jenny LongsterMartin Shepperd

Brunel University, BT

Contents

• Introduction to Simulation• Simulation Approach• Introduction to System Dynamics• Basic Modelling Building Blocks• Simulating Software Projects• Putnam Model of Software Lifecycle• The BT Research Project• Introduction to COTS• The COTS Requirements Model• Results• Discussion and Future Work

Introduction to Simulation

• Definitions– System

• a system exists and operates in time and space

– Model • simplified representation of a system at some point in time or space• intended to promote understanding of the real system

– Simulation • manipulation of a model so that time is compressed• behaviour of the system under specific conditions may be studied• enables interactions to be seen in a matter of minutes

• Dynamic complexity– systems have cause and effect– today’s systems more and more complex

Simulation Approach

• Two main approaches– Discrete modelling– Continuous modelling

• Discrete event modelling – time advances when a discrete event occurs to the next event

time• e.g. traffic – car arrives• an associated action takes place • implies placing a new event in an event queue

– difficult to monitor continually changing variables - clunky• Continuous Simulation based on System Dynamics

– time is controlled by continuous variables expressed as differential equations

– see the interactions between key process factors– considered to be better at showing qualitative relationships– method chosen

Introduction to System Dynamics

• Used to solve real world problems– pressing business/process issues

• Systems Thinking– the ability to see the world as a complex system– understand that “you can’t do just one thing” and that

“everything is connected to everything else”• System Dynamics

– exponential growth of technology and complexity– helps understand complexity– design better operating policies– guide changes in systems

Introduction to System Dynamics

• Seeking to solve a problem often makes it worse– Policies create unanticipated side effects– Attempts to stabilize systems may destabilize them

further– Decisions may provoke reactions by others seeking to

restore the balance upset• Examples of adverse reactions to policies

– Low tar cigarettes– Pesticides and herbicides– IT has not enabled the “paperless office”– Antilock brakes cause people to drive more

aggressively

Basic Modelling Building Blocks:iThink

StocksPurpose: Stocks are accumulations. They collect

whatever flows into and out of them

FlowsPurpose: to fill and drain stocks. The arrow indicates

the direction of the flow

ConvertersPurpose: converts inputs to outputs - can hold values

for constants, define external inputs to the model, perform algebraic calculations

ConnectorsPurpose: connects model elements

Rate

Level

Calculation

Level

Calculation

Example Model: Publication Output

• Consider a university with 120 people active in research• 120 people composed of 60 PhD researchers and 60

professors• University wishes to maintain the total number of people

active in research at 120 – Professors quit at a rate of 10 per year– One new PhD student hired for each professor who quits– PhDs don't quit! – 6 years to develop a professor from a PhD– PhDs produce 3 papers per year– Professors produce 10 papers per year– All 120 active researchers are fully active (!)

iThink Model

Initial Professors = 60

Initial Researchers = 10, 10, 10, 10, 10, 10

Quits = 10

Hires = quits

xfer = 6 years

Publication rate = 3 * researchers + 10 * professors

iThink Model Results

stable publication output, stable number of profs, PhDs

Management Policy change• suppose the

university introduces a new policy– Professors must

clean toilets as part of their job description

– Annoys professors– Quit rate moves

from 10 to 15 a year

Results of Policy Change

• in the 10th year university management notices its publications have dropped from 780 per year to 570 per year

• it wonders what has happened• where does it look for the problem?

– around the 10th month – the university finds that it still has 120 active researchers, yet

there are now 30 professors and 90 PhDs – A most puzzling situation!– The university, with it's built in hiring rule, essentially an auto

pilot no thought action, hired one PhD for each professor that quit

– this one time transition in quit rate set off a 6 year transition within the university

• leading to a new equilibrium state with 30 professors and 90 PhDs

Results of Policy Change

Publication rate dropped from 780 to 570 per year

Simulating Software Projects• Two models

– General Cost Estimation Model– Model specifically for BT project

• General Cost Estimation Model– Micro model

• bottom up • capture data on small software parts e.g. technology, user changes

etc.• COCOMO

– Used to estimate desired values - effort, time– Conjecture – applicable outside the organisation?

– Macro model • Top down• only use most important large variables– project cost, time, size• Norden/Putnam model

– Classic non-linear model– See relationship between schedule, manpower and size – not direct

trade off– Conjecture

Norden Manpower Distribution Model

• Theory of project modelling– Project seen as set of problems to be solved– Development ends when all problems are transformed into

solutions– Requires manpower, time and creativity– Manpower effort , K (man.years) is an indicator of number of

problems to be solved– Manpower available=manpower required

• development optimised in terms of time and cost

Manpower Distribution

Putman Model of the Software Lifecycle

• Norden model extended by Putnam to model software lifecycle– Formal model for software development estimating

• Assumed software development follows Norden manpower distribution– With noise – changing requirements etc

• Data collection– 1970s, 200 projects– Verified manpower distribution– Peak manning near delivery time

• Example usage– Manpower Effort K=95 man.years, delivery time 1.75 years– Can calculate

• number of people at peak (33 persons)• Average rate of team build-up (mo/td)

– 1.5 person/month - reasonable

o

d

Km

t e

Difficulty

• Slope of manpower distribution at start time t=0– Called Difficulty D– Larger value shows project is more difficult to manage

• Manpower demand is high• Schedule is short• Higher number of people to be integrated and done quickly

– Putnam observed Difficulty with 200 projects– Developed a derivative called manpower build-up related to the

nature of the development• =8 Entirely new software, complex• =15 New stand alone system• =27 Rebuilt from existing software

– Larger value = steeper curve

2d

KD

t

0D

0D0D0D

iThink Putnam Model Interface

Project manger inputs

Effort remaining to be employed (man.years)

Delivery time

Nature of Development

iThink Putnam Model

Simulating Software Projects: Putnam Model Results

Simulating Software Projects: Cutting Development Time

Simulating Software projects: Changing the Nature of the

Development

Second Model: The BT Research Project

• Software industry has undergone significant change – from writing all code in house– to system building with existing component products– Integration of Commercial-Off-The-Shelf (COTS) products

• COTS projects have specific features that add new factors – need to be addressed for successful software development

• Conducted a systematic review– real-world COTS projects– evidence-based– extracted success and failure factors

• Package up Results– simulation of effects of success / failure factors– model the requirements stage of a COTS-based project

iThink COTS Requirements Model

COTS Requirements Model Interface

COTS Requirements Model Interface: Project Properties

COTS Model: Acquisition Factors

COTS Model: Requirements Success Factors

COTS Requirements Model: Results

Lower Success Factors and Decreasing Schedule

Conclusions

• Learning curve about COTS projects

• Powerful, flexible tool for analysis

• Business visualisation tool – BT

• Packaging up mathematical ideas– Putnam model

• Packaging up Systematic review results

Discussion and Further Work

– Extend COTS requirements model to whole lifecycle– Calibrate models

• Data?• Ask people to rate importance of success factors?• Is the Putnam model applicable to COTS projects?• Can we make it so?

– Monte Carlo Simulation• Single point solution (e.g. calendar months) represents

average (50% probability) based on certainty of assumptions• Equally likely to under-run as over-run• For any other probability assign random values and simulate• Enables a ‘best-bid’ solution