the practice of software estimation

© 2008 by Phymata Solutions Inc. (Everett Toews)© 2008 by Phymata Solutions Inc. (Everett Toews)This presentation may not be modified in any way.This presentation may not be modified in any way.Permission to use this presentation is granted to Permission to use this presentation is granted to

eWorld ES Inc. provided this copyright notice is included.eWorld ES Inc. provided this copyright notice is included.

The Practice ofThe Practice ofSoftware EstimationSoftware Estimation



Presentation RoadmapPresentation Roadmap

OverviewOverview TechniquesTechniques PresentationPresentation Wrap upWrap up QuestionsQuestions



Why Talk About Software Why Talk About Software Estimation?Estimation?

My MotivationMy Motivation Advancing the Art and Science of Advancing the Art and Science of

Software DevelopmentSoftware Development Making a livingMaking a living My ExperienceMy Experience



Current State of Software Current State of Software EstimationEstimation

Most companies looking to avoid Most companies looking to avoid estimates that are incorrect by 100% estimates that are incorrect by 100% or moreor more

Goal – A good estimation approach Goal – A good estimation approach that provides estimates that are that provides estimates that are within 25% of the actual results %75 within 25% of the actual results %75 of the timeof the time



Estimates, Targets and Estimates, Targets and CommitmentsCommitments

Estimate (layman’s definition) – A Estimate (layman’s definition) – A prediction of how long a project will take prediction of how long a project will take or how much it will cost.or how much it will cost.

Target – A statement of a desirable Target – A statement of a desirable business objective.business objective.

Commitment – A promise to deliver Commitment – A promise to deliver defined functionality at a specific level of defined functionality at a specific level of quality on a certain date.quality on a certain date.

Don’t let the target become the estimateDon’t let the target become the estimate



What is a Good Estimate?What is a Good Estimate? Single point estimates are meaninglessSingle point estimates are meaningless

All estimates should include a probabilityAll estimates should include a probability



What is a Good Estimate?What is a Good Estimate?

Making a commitmentMaking a commitment Ability to support a project’s successAbility to support a project’s success Realistic project targetsRealistic project targets Estimate – A good estimate is an estimate Estimate – A good estimate is an estimate

that provides a clear enough view of the that provides a clear enough view of the project reality to allow the project project reality to allow the project leadership to make good decisions about leadership to make good decisions about how to control the project to hit its targets.how to control the project to hit its targets.



How Good an Estimator Are How Good an Estimator Are You?You?

1.1. Surface temperature of the SunSurface temperature of the Sun2.2. Latitude of ShanghaiLatitude of Shanghai3.3. Area of the Asian continentArea of the Asian continent4.4. The year of Alexander the Great’s birthThe year of Alexander the Great’s birth5.5. Total value of US currency in circulation in 2004Total value of US currency in circulation in 20046.6. Total volume of the Great LakesTotal volume of the Great Lakes7.7. Worldwide box office receipts for TitanicWorldwide box office receipts for Titanic8.8. Total length of the coastline of the Pacific OceanTotal length of the coastline of the Pacific Ocean9.9. Number of book titles published in the US since Number of book titles published in the US since

1776177610.10. Heaviest blue whale ever recordedHeaviest blue whale ever recorded

Quiz © 2006 Steve McConnellQuiz © 2006 Steve McConnellAll Rights Reserved. Permission to copy this quiz is granted All Rights Reserved. Permission to copy this quiz is granted

provided that this copyright notice is included.provided that this copyright notice is included.



How Good an Estimator Are How Good an Estimator Are You?You?

1.1. 10,00010,00000F / 6,000F / 6,00000CC2.2. 31 Degrees North31 Degrees North3.3. 17,139,000 miles17,139,000 miles22 / 44,390,000 km / 44,390,000 km22

4.4. 356 BC356 BC5.5. $719.9 billion$719.9 billion6.6. 6 x 106 x 101515 gallons / 2.3 x 10 gallons / 2.3 x 101616 liters liters7.7. $1.835 billion$1.835 billion8.8. 84,300 miles / 135,663 kms84,300 miles / 135,663 kms9.9. 22 million22 million10.10. 380,000 pounds / 190 tons / 170,000 380,000 pounds / 190 tons / 170,000

kgs / 170 metric tonskgs / 170 metric tonsQuiz © 2006 Steve McConnellQuiz © 2006 Steve McConnell

All Rights Reserved. Permission to copy this quiz is granted All Rights Reserved. Permission to copy this quiz is granted provided that this copyright notice is included.provided that this copyright notice is included.



90% Confident90% Confident

90% confident closer to 30% 90% confident closer to 30% confidentconfident

Narrow ranges are more accurate?Narrow ranges are more accurate? Pressure to provide narrow ranges is Pressure to provide narrow ranges is

self inducedself induced Estimating unknown quantitiesEstimating unknown quantities



Overestimate or Overestimate or UnderestimateUnderestimate

Arguments against overestimationArguments against overestimation Arguments again underestimationArguments again underestimation Penalty for overestimationPenalty for overestimation Penalty for underestimationPenalty for underestimation The best projects come from the The best projects come from the

most accurate estimatesmost accurate estimates



Benefits of Accurate Benefits of Accurate EstimatesEstimates

Improved status visibilityImproved status visibility Higher qualityHigher quality Better coordination with other Better coordination with other

departmentsdepartments Better budgetingBetter budgeting Increased credibilityIncreased credibility Early risk infoEarly risk info PredictabilityPredictability



The Cone of UncertaintyThe Cone of Uncertainty



Where Does Estimation Where Does Estimation Error Come From?Error Come From?

Chaotic development processChaotic development process Unstable requirementsUnstable requirements Omitted activitiesOmitted activities Unfounded optimismUnfounded optimism Subjectivity & biasSubjectivity & bias Off-the-cuff estimatesOff-the-cuff estimates Unwarranted precisionUnwarranted precision Unfamiliar business/technology areaUnfamiliar business/technology area Simplification of the estimateSimplification of the estimate ……



Estimate InfluencesEstimate Influences

SizeSize Diseconomies of scaleDiseconomies of scale

Kind of software being developedKind of software being developed Personnel factorsPersonnel factors Programming languageProgramming language



Estimate InfluencesEstimate Influences

1.1. Product complexityProduct complexity

2.2. Requirements analyst capabilityRequirements analyst capability

3.3. Programmer capabilityProgrammer capability

4.4. Time constraintsTime constraints

5.5. Personnel continuityPersonnel continuity



Considerations in Choosing Considerations in Choosing Estimation TechniquesEstimation Techniques

What’s being estimated?What’s being estimated? Project sizeProject size Development methodologyDevelopment methodology Development stageDevelopment stage Accuracy possibleAccuracy possible



Count, Compute, JudgeCount, Compute, Judge

Count if possibleCount if possible Compute when you can’t countCompute when you can’t count Use judgment alone as a last resortUse judgment alone as a last resort

Count ComputeWhat Size, Features Size, Effort, Schedule, FeaturesSize S M L S M LStage Early-Late Early-MiddleStyle Iter. & Seq. Iter. & Seq.Accuracy High High



What to CountWhat to Count

Something that is highly correlatedSomething that is highly correlated Something available sooner than Something available sooner than

laterlater Something that will produce a Something that will produce a

statistically meaningful averagestatistically meaningful average Understand what you are countingUnderstand what you are counting



Calibration and Historical Calibration and Historical DataData

Industry dataIndustry data Organizational (Historical) dataOrganizational (Historical) data Project dataProject data

Calibrate withIndustry Average Data

Calibration withOrganizational Data

Calibration withProject Specific Data

WhatSize, Effort, Schedule, Features

Size, Effort, Schedule, Features


Size S M L S M L S M LStage Early-Middle Early-Middle Middle-LateStyle Iter. & Seq. Iter. & Seq. Iter. & Seq.Accuracy Low-Medium Medium-High High



Historical DataHistorical Data

Accounts for organizational Accounts for organizational influencesinfluences

Avoids subjectivity, unfounded Avoids subjectivity, unfounded optimism and politicsoptimism and politics

Data to collectData to collect Size, Effort, Time and DefectsSize, Effort, Time and Defects

How to calibrateHow to calibrate



Individual Expert JudgmentIndividual Expert Judgment

Expert in what?Expert in what? Structured expert judgmentStructured expert judgment GranularityGranularity ChecklistChecklist

Use of Structured Process

Use of Estimation Checklist

Estimating Task Effort in Ranges

Comparing Task Estimates to Actuals

WhatEffort, Schedule, Features

Effort, Schedule, Features



Size S M L S M L S M L S M LStage Early-Late Early-Late Early-Late Middle-LateStyle Iter. & Seq. Iter. & Seq. Iter. & Seq. Iter. & Seq.Accuracy High High High N/A



Use of RangesUse of Ranges

Single point estimates are badSingle point estimates are bad Best case/Worst caseBest case/Worst case Optimistic worst casesOptimistic worst cases Expected Case = [BestCase + (3 x Expected Case = [BestCase + (3 x

MostLikelyCase) + (2 x WorstCase)] / 6MostLikelyCase) + (2 x WorstCase)] / 6 Task | Best | Most Likely | Worst | ExpectedTask | Best | Most Likely | Worst | Expected RiskRisk



Compare Estimates to Compare Estimates to ActualsActuals

Magnitude of Relative Error = Magnitude of Relative Error = (ActualResult - EstimatedResult) / (ActualResult - EstimatedResult) / ActualResultActualResult

ComparisonComparison FeedbackFeedback



Decomposition and Decomposition and RecompositionRecomposition

DecompositionDecomposition The Law of Large NumbersThe Law of Large Numbers Decomposition via WBSDecomposition via WBS Adding the Best Case and Worst CaseAdding the Best Case and Worst Case

Decomposition by Feature or Task

Decomposition by Work Breakdown Structure (WBS)

Computing Best and Worst Cases from Standard Deviation

What Size, Effort, Features Effort Effort, ScheduleSize S M L M L S M LStage Early-Late Early-Middle Early-LateStyle Iter. & Seq. Iter. & Seq. Iter. & Seq.Accuracy Medium-High Medium Medium



Estimation by AnalogyEstimation by Analogy

Simple processSimple process

Estimation by AnalogyWhat Size, Effort, Schedule, FeaturesSize S M LStage Early-LateStyle Iter. & Seq.Accuracy Medium



Estimation by AnalogyEstimation by Analogy

1.1. Similar previous projectSimilar previous project

2.2. CompareCompare

3.3. Build up the estimateBuild up the estimate

4.4. Create an effort estimateCreate an effort estimate

5.5. Consistent assumptionsConsistent assumptions



Proxy-Based EstimatesProxy-Based Estimates

Identify proxyIdentify proxy Estimate or count the proxyEstimate or count the proxy

Fuzzy LogicStandard Components Story Points T-Shirt Sizing

What Size, Features Size, EffortSize, Effort, Schedule, Features

Effort, Cost, Schedule, Features

Size M L S M L S M L M LStage Early Early-Middle Early-Middle EarlyStyle Seq. Iter. & Seq. Iter. & Seq. Seq.Accuracy Medium Medium Medium-High N/A



Proxy-Based EstimatesProxy-Based Estimates

Fuzzy LogicFuzzy Logic Standard ComponentsStandard Components Story PointsStory Points T-Shirt SizingT-Shirt Sizing



Expert Judgment in GroupsExpert Judgment in Groups

Group ReviewsGroup Reviews Wideband DelphiWideband Delphi

Group Reviews Wideband DelphiWhat Size, Effort, Schedule, Features Size, Effort, Schedule, FeaturesSize M L M LStage Early-Middle EarlyStyle Iter. & Seq. Seq.Accuracy Medium Medium



Software Estimation ToolsSoftware Estimation Tools

Simulate project outcomesSimulate project outcomes Probability analysisProbability analysis Account for diseconomies of scaleAccount for diseconomies of scale Account for creeping requirementsAccount for creeping requirements

Software Estimation ToolsWhat Size, Effort, Schedule, FeaturesSize M LStage Early-MiddleStyle Iter. & Seq.Accuracy High




Estimation of less common software Estimation of less common software issuesissues

Planning options and tool integrationPlanning options and tool integration What-if analysisWhat-if analysis Referee unrealistic project expectationsReferee unrealistic project expectations Objective authorityObjective authority Sanity checking manual estimatesSanity checking manual estimates Estimating large projectsEstimating large projects




Data needed to calibrate toolsData needed to calibrate tools Effort in staff monthsEffort in staff months ScheduleSchedule Size in LOCSize in LOC

Available toolsAvailable tools



Use of Multiple ApproachesUse of Multiple Approaches

No single estimation technique is No single estimation technique is perfectperfect

Convergence is good, spread is badConvergence is good, spread is bad MisconceptionsMisconceptions Different resultsDifferent results



Flow of Software Estimates on Flow of Software Estimates on a Well-Estimated Projecta Well-Estimated Project

Poorly estimated projectsPoorly estimated projects Well estimated projectsWell estimated projects

Size > Effort > Schedule | Cost | FeaturesSize > Effort > Schedule | Cost | Features

Changing to More AccurateMethods Later in the

Estimate Refinement Basedon Project-Specific Data

What Size, Effort, Schedule, Features Size, Effort, Schedule, FeaturesSize M L S M LStage Early-Late Early-LateStyle Seq. Seq.Accuracy High High



Well Estimated ProjectsWell Estimated Projects

ReestimateReestimate RefinementRefinement

Make up the timeMake up the time Add the timeAdd the time Multiply the whole schedule by the Multiply the whole schedule by the

magnitude of the slipmagnitude of the slip Presenting reestimationPresenting reestimation



Standardized Estimation Standardized Estimation ProceduresProcedures

Protects againstProtects against Off the cuff estimatesOff the cuff estimates GuessingGuessing Changing the estimate because a Changing the estimate because a

stakeholder doesn’t like itstakeholder doesn’t like it Encourage consistencyEncourage consistency



Elements of Standardized Elements of Standardized ProceduresProcedures

Emphasizes counting and computingEmphasizes counting and computing Multiple estimation approachesMultiple estimation approaches ReestimationReestimation Approach changes over the course of a Approach changes over the course of a

projectproject Clear description of estimate’s inaccuracyClear description of estimate’s inaccuracy When it can be used as a basis for internal When it can be used as a basis for internal

and external commitmentsand external commitments Archiving estimation data for later reviewArchiving estimation data for later review Deviations should be documented and rareDeviations should be documented and rare Define when estimation occurs in SDLCDefine when estimation occurs in SDLC



Estimating SizeEstimating Size

Role of LOC in size estimationRole of LOC in size estimation Single dimensionSingle dimension Have to measure somethingHave to measure something

Function Points GUI ElementsWhat Size, Features Size, FeaturesSize S M L S M LStage Early-Middle EarlyStyle Seq. Seq.Accuracy High Low



Estimating SizeEstimating Size

Function PointsFunction Points External Inputs, External Outputs, External Inputs, External Outputs,

External Queries, Internal Logical Files, External Queries, Internal Logical Files, External Interface FilesExternal Interface Files

GUI ElementsGUI Elements Screens, Reports, Files, Interfaces, Screens, Reports, Files, Interfaces,

DialogsDialogs



Estimating EffortEstimating Effort

Informal ComparisonInformal Comparison Estimation Software ToolsEstimation Software Tools Industry Average Effort GraphsIndustry Average Effort Graphs

Informal Comparison to Past Projects

Estimation Software Tools

Industry Average Effort Graphs

What Effort Effort EffortSize S M S M L S MStage Early-Middle Early-Middle EarlyStyle Iter. & Seq. Iter. & Seq. Seq.Accuracy Medium High Low-Medium



Estimating ScheduleEstimating Schedule

The Basic Schedule EquationThe Basic Schedule Equation ScheduleInMonths = 3.0 x StaffMonthsScheduleInMonths = 3.0 x StaffMonths1/31/3

Informal Comparison to Past ProjectsInformal Comparison to Past Projects EstimatedSchedule = PastSchedule x EstimatedSchedule = PastSchedule x

(EstimatedEffort / PastEffort)(EstimatedEffort / PastEffort)1/31/3

Estimation Software ToolsEstimation Software Tools

The Basic Schedule Equation

Informal Comparison to Past

Estimation Software Tools

What Schedule Schedule ScheduleSize M L S M L M LStage Early Early EarlyStyle Seq. Iter. & Seq. Iter. & Seq.Accuracy Medium Medium High



Schedule CompressionSchedule Compression Shortening the schedule increases effortShortening the schedule increases effort Trade OffsTrade Offs

Schedule | EffortSchedule | Effort -15% | +100%-15% | +100% -10% | +50%-10% | +50% -5% | +25%-5% | +25% 0% | 0%0% | 0%

Team SizeTeam Size AverageTeamSize = EffortEstimate / AverageTeamSize = EffortEstimate /

ScheduleSchedule



Estimating Planning Estimating Planning ParametersParameters

Estimating total activityEstimating total activity Converting estimated effort to Converting estimated effort to

planned effortplanned effort Estimating defect production and Estimating defect production and

removalremoval



Estimate Presentation Estimate Presentation StylesStyles

Communicate estimate assumptionsCommunicate estimate assumptions Expressing uncertaintyExpressing uncertainty Risk qualificationRisk qualification AccuracyAccuracy RangesRanges



Politics, Negotiation and Politics, Negotiation and Problem SolvingProblem Solving

External constraintsExternal constraints Negotiating a commitmentNegotiating a commitment Problem solvingProblem solving

Separate the people from the problemSeparate the people from the problem Focus on interests, not positionsFocus on interests, not positions Invent options for mutual gainInvent options for mutual gain Insist on using objective criteriaInsist on using objective criteria



Outstanding IssuesOutstanding Issues

Counting lines of codeCounting lines of code Estimates vs actualEstimates vs actual Contractual software developmentContractual software development



RecommendationsRecommendations DevelopersDevelopers

Use rangesUse ranges Present estimates with assumptionsPresent estimates with assumptions

ManagementManagement Use historical dataUse historical data Archive estimates in an easily accessible placeArchive estimates in an easily accessible place Publish a standardized estimation procedurePublish a standardized estimation procedure

BothBoth Use checklistsUse checklists phymata.blogspot.comphymata.blogspot.com



ConclusionConclusion

What am I doing about it?What am I doing about it? What’s next?What’s next?

construx.com/Page.aspx?nid=297construx.com/Page.aspx?nid=297

the practice of software estimation

Technology