Software Estimation and Performance Measurement

Harold van HeeringenSoftware Cost Engineer, Sogeti Nederland B.V.Sizing, Estimating & ControlISBSG presidentNESMA boardCOSMIC IAC, representing the Netherlands

Sogeti StockholmApril 19 2013

Software Estimation and Performance Measurement

3

Overview

Software ProjectsSoftware Project EstimationFunctional Size MeasurementEstimating & Performance MeasurementISBSG repositoryTrends in the marketConclusions

4

Software projects

Software project industry: low maturityLow estimation maturityNo or little formal estimation processesNo or little use of historical data

Lots of schedule and cost overrunsStandish Chaos reports: Most projects fail or are at least unsuccessful

Low customer satisfaction ratesIn Europe: only slightly higher than the financial sector

5

Software project estimationMost of the projects are estimated by ‘experts’

Bottom up, task by task effort estimation

Usually very optimistic (>30%)Experts estimate, but other people (juniors) do the jobForgotten activities (e.g. testscript reviews)No feedback loop with past projects: experts don’t learn from past estimates and actuals

No scenario’s: duration, team size, etc.Not objective, transparent, verifiable and repeatable

Not defendable!‘Easy’ to push back by stakeholders

No risk assessment (distribution of the estimate)

6

Software Cost engineering

Not a real profession yetConsultant software metricsEstimation officerBid specialist

Parametric estimatesFunctional size measurement size of the softwareProductivity rates from historical data or industry dataParametric estimation tools

Objective, repeatable, transparent and verifiable

Defendable!!‘Impossible’ to push back by stakeholders

Risk assesment (distribution of the estimate)

7

Functional Size Measurement

Function Point Analysis (NESMA, IFPUG or COSMIC)Measure the functional user requirements – size in function points;ISO standards – objective, verifiable, repeatable;Strong relation between functional size and project effort needed;More function points means more functionality (e.g. more value);the user or client understands that more functionality means a higher price

What to do with the results?Project effort/duration/cost estimationProject benchmarking/performance measurementUse in Request for Proposal management (answer price/FP questions)

What about historical data?Company data (preferably for estimation)Industry data (for benchmarking)

8

Software Estimation

Measure the Size of the functional user requirements

Determine the right productivity figures, based on own history data or industry project data

Use Parametric tools to estimate effort, duration, team size, quality- non linear relations

Scenario analysis- different durations- different team size- varying influence of non-functional requirements- etcetera

9

Software equation (Putnam)

Size/productivity

= Effort 1/3 * duration4/3

Eff

ort

Duration

Plan A: 6 months, 4.500 hours

Plan B: 7 months, 2.400 hours

10

Duration

Plan ADuration: 6 monthsEffort: 4.500 hoursMax. team size: 5,8 fteMTTD: 1,764 days

Plan BDuration: 7 monthsEffort: 2.400 hoursMax. team size: 2,7 fteMTTD: 2,816 days

Which duration have the experts in mind??

Eff

ort

(hou

rs)

Same project, different duration

11

Short demo of QSM SLIM

12

Cost of wrong estimatesNon-linear extra costs

-Planning errors

-team enlargement more expensive, not faster

-Extra management attention / overhead

-Stress: More defects, lower maintainability !!

Linear extra kosten

Extra hours will be used

13

Sogeti SEC

Sizing, Estimating & ControlCertified (COSMIC) Function Point AnalystsMetrics consultants

Responsible for metrics part of a quotation.Size: FPA/COSMICEstimation: SEER-SEM / QSM / Sogeti tool / ISBSGProduct: Methodical Estimation Report (scenario’s)Pricing: EUR/FP Quality: Defects/FP

Centers of Excellence: MS.Net, Java, Oracle, mobile, drupal, sharepoint, BI, etc.

14

Estimating & Performance Measurement

PLAN

Estimate

AdministrateEvaluate

Adjust & Report

Size measurement: FPAHistorical dataEstimation tools

Finetune Estimation modelAnalyse productivity, Report productivity

Start: Estimate request

Start: Project completed

Results: - Parametric Estimation- Expert Estimation

Result: -Management report, -Adjusted model

Result: -Growing project DB, -Performance measurement-Updated expert knowledge

ACT

CHECK DO

Start: Project start

Continuous data collection• effort hours registration• defect registration• change measurement• project characteristics

Result: Project data

Data collection and administration• Collect project data• Measure size• Benchmark the project

Start: Periodically

Expert Estimate

15

Performance Measurement

Measure the performance of completed projects- Size in function points delivered- Defects found in various test phases- Hours spent per main activity- Compare actuals to plan

Derive the main metrics for the project, e.g.- Productivity (hours/FP)- Cost (EUR/FP)- Quality (Defects/FP)- Schedule / Budget slippage

Store and Benchmark the data – internal and external

16

Historical data: ISBSG repositories

International Software Benchmarking Standards GroupIndependent and not-for-profitMembers are non-profit organizations, like IFPUG and NESMA

Grows and exploits two repositories of software data:New development projects and enhancements (> 6000 projects)Maintenance and support (> 1000 applications)

Everybody can submit project data DCQ on the siteAnonymousFree benchmark report in return

Special reports, Practical Project Estimation book, CompendiumPortal to access the project data

17

Trends in performancePI vs Effective FP

100 150 200 250 300 350 400 450 500 550 600

Effective FP

5

10

15

20

25

PI

Java Totaal QSM Business 4GL ISBSG Avg. Line Sty le

Needed for management to control and manageNeeded to express to the market in proposalsNeeded to make sure we keep improving

18

Trend information

19

Trends

Supplier Performance Measurement- Reality check of supplier’s quotations based on industry data- Agreement on performance improvements: hours/FP or EUR/FP

Request for Proposals- Output based pricing- Price per function point asked- Often cheapest one selected – which is usually the wrong choice

Increasing professionalism in project estimation- Basis of Estimate – Recommended practice by AACE for software services- Tools, research and better functional sizing methods (e.g. COSMIC)- More people understand the importance of parametric estimates

20

Conclusions

Estimation & Performance Process is important! Increasing maturity of software realization

More accurate estimates, based on history data defendable estimates;‘Professional’ image communicated to client;Benchmarking learning organization improvement;Performance Measurement manage by actual performance dataData is needed when answering metrics questions in RFP / RFI

Sogeti – international collaboration is importantShared Best Practices Sogeti SEC wishes to help you when possible!

@haroldveendamHarold.van.heeringen@sogeti.nl

Harold van Heeringen Senior Consultant Software Metrics /Software Cost Engineer

Sogeti Sizing, Estimating & Control (SEC)

President ISBSG (International Software Benchmarking Standards Group (www.isbsg.org))Board member NESMA (Netherlands Software Metrics Association (www.nesma.nl))IAC member COSMIC (www.cosmicon.com)