unclassified approved for public release 07-mda-2965 (26 oct 07) load bearing walls: early sizing...

UNCLASSIFIED

UNCLASSIFIEDApproved for Public Release 07-MDA-2965 (26 OCT 07)

Load Bearing Walls: Early Sizing Estimation In The NOSTROMO Tool (U)

Dan StricklandDynetics Program Software Support

[email protected]

Ashley MathisTHAAD Project Office System Software Engineering

[email protected]

Brandon GautneyDynetics Program Software Support

[email protected]

DISTRIBUTION STATEMENT A – Approved for public release; distribution is unlimited.

07T-1552.01

UNCLASSIFIED

2UNCLASSIFIEDApproved for Public Release 07-MDA-2965 (26 OCT 07)

Overview (U)

• Background

• Area of Opportunity

• Heuristic Model

• Subcomponent Weight

• Complexity Weight

• NOSTROMO Concept

• NOSTROMO Additions

07T-1552.02

UNCLASSIFIED


Background (U)

1-800-PSY-CHIC

UNCLASSIFIED


Boehm Horn Curve (U)

[Ref (1) - SOFTWARE COST ESTIMATION WITH COCOMO II (P 10) - BOEHM 2000]

• Size (and effort) should converge over time relative to development phase

• During Concept of Operation, estimates can be 50% to 200% of the actual final size

07T-1552.04

UNCLASSIFIED


4x

2x

1.5x

1.25x

0.25x

0.5x

x

Concept of Operations

Requirements Spec

Preliminary Design Spec.

DetailedDesign Spec.

Qualified Software

Area of Opportunity (U)

• Area between Concept of Operations and Requirements Specification when early size estimation is key

• Early sizing method needs to have accuracy within this area

07T-1552.05

UNCLASSIFIED


Roetzheim Heuristic Model (U)

Function 1Object 2Object Library 4Proof of Concept 5Evolutionary Prototype 6Internal Application 8External Application 9Shrink-Wrap Application 10Component of System 11New System 12Compound System 13

Individual Use 1Shareware 2Academic/Engineering 3Single Location -Internal 5Multilocation - Internal 6Contract Project - Civilian 7Contract Project - Local Government 8Marketed Commercially 9State Government 11State Government - Federally Funded 13Federal Project 14Military Project 15

No Programming (4GL/Drag and Drop) 1Batch 23GL Programming 4Embedded - Single Board 5Database Oriented 6Client - Server 8Mathematical 9Systems 10Communications 11Process Control 12Embedded - Multi Board 13Embedded - Complete System 14Social Service 15

Project Scope Table Project Class Table Project Type Table

• Project taxonomies are a method for estimating Function Points (FP)• Function Points are a language independent approach for estimating software development effort

Approximate Value for Function Points (FP): FPs = (ValueSCOPE + ValueCLASS + ValueTYPE)

2.35

Roetzheim, William. “Estimating and Managing Project Scope for New Development.”Cross Talk April 2005

07T-1552.06

UNCLASSIFIED


Heuristic Model Results

Projects Parameters

Results

Language Project Scope Project Class Project Type

Project 1 C++ Component of System Military Project SystemsProject 2 Ada 95 Component of System Military Project Embedded - Multi Board Project 3 C++ Component of System Military Project Embedded - Multi Board

• Most large, complex programs use SLOC as system of delivery for their sizing metrics

• “Backfiring” is a method for converting FPs to an estimated corresponding number of Software Lines of Code (SLOC)

Actual New KSLOC Estimated New KSLOC Fraction DeltaProject 1 75.00 240.76 3.21 221%Project 2 403.65 268.66 0.67 -33%Project 3 455.75 290.59 0.64 -36%

Total 934.40 800.00 0.86 -14%

07T-1552.07

UNCLASSIFIED


Heuristic Model Results (cont’d)

• Projects 2 & 3 are within acceptable boundaries of the Area of Opportunity.

• Project estimated totals are within 14% of the actual size.

4x

2x

1.5x

1.25x

0.25x

0.5x

x


Requirements Spec



Qualified Software

ProjectTotalsArea of Opportunity


22

11

33

07T-1552.08

UNCLASSIFIED


Subcomponents (U)

Project Parameters


2.35 * (WeightSUBCOMPONENT)

Subcomponents Description WeightFew 1 - 4 Software Subcomponents 0.67

Nominal 5 - 8 Software Subcomponents 1.00Many 9+ Software Subcomponents 1.33

Language Project Scope Project Class Project Type Subcomponents

Project 1 C++ Component of System Military Project Systems FewProject 2 Ada 95 Component of System Military Project Embedded - Multi Board ManyProject 3 C++ Component of System Military Project Embedded - Multi Board Nominal

07T-1552.09

UNCLASSIFIED


Heuristic Model With Subcomponents Results


Total 934.40 809.21 0.87 -13%

• Heuristic Model performance improves when coupled with Subcomponent Weight.



4x

2x

1.5x

1.25x

0.25x

0.5x

x


Requirements Spec



Qualified Software



22

11

11

3322

33

07T-1552.10

UNCLASSIFIED


Complexity (U)

Problem Complexity

Code Complexity

Data Complexity

Sum of Problem, Data, and Code Complexity Scores

Code Size Adjustment Calculators

3 0.704 0.755 0.806 0.857 0.90

8 0.95

9 1.0010 1.0511 1.1012 1.1513 1.2014 1.2515 1.30

Complexity Adjustments


2.35 * (WeightCOMPLEXITY)

Jones, Capers T. Estimating Software Cost. New York: McGraw Hill,1998.

1-5

1-5

1-5

07T-1552.11

UNCLASSIFIED


Heuristic Model With Complexity Results

Project Parameters

Results


Total 934.40 886.38 0.95 -5%

Data Complexity Sum of Complexity Complexity Adjustment Multiple files, switches and data interactions 8 0.95

Complex data elements and complex data interactions 12 1.15Very complex data elements and complex data interactions 13 1.20

Problem Complexity Code ComplexityProject 1 Majority of simple algorithms and calculations Well Structured (small models and simple paths)Project 2 Some difficult and complex calculations Fair structure, but some complex paths or modelsProject 3 Some difficult and complex calculations Fair structure, but some complex paths or models

07T-1552.12

UNCLASSIFIED


4x

2x

1.5x

1.25x

0.25x

0.5x

x


Requirements Spec



Qualified Software



22

11

3322

11

33

Heuristic Model With Complexity Results (cont’d)

• Heuristic Model performance is slightly improved when coupled with Complexity Weight.



07T-1552.13

UNCLASSIFIED


Project Parameters

Complete Heuristic Model - Combined Heuristic Model with Subcomponent Weight and Complexity Weight

Language Project Scope Project Class Project Type

Project 1 C++ Component of System Military Project SystemsProject 2 Ada 95 Component of System Military Project Embedded - Multi Board Project 3 C++ Component of System Military Project Embedded - Multi Board


2.35 * (WeightSUBCOMPONENT) * (WeightCOMPLEXITY)

Subcomponents Complexity Adjustment

Few 0.95Nominal 1.15

Many 1.20

07T-1552.14

UNCLASSIFIED


Complete Heuristic Model Results - Combined Heuristic Model with Subcomponent Weight and Complexity Weight

Actual New KSLOC Estimated New KSLOC Fraction DeltaProject 1 75.00 153.24 2.04 104%Project 2 403.65 410.91 1.02 2%Project 3 455.75 348.71 0.77 -23%

Total 934.40 965.38 1.03 3%

• Heuristic Model performance improves when coupled with Subcomponent Weight and Complexity Weight.



4x

2x

1.5x

1.25x

0.25x

0.5x

x


Requirements Spec



Qualified Software



22

11

3322

11

33

07T-1552.15

UNCLASSIFIED


Original NOSTROMO Concept (U)

• NOSTROMO first takes normal COCOMO II inputs• NOSTROMO accounts for uncertainty in the settings• NOSTROMO uses Monte Carlo to simulate hundreds of COCOMO II estimates using the inputs and uncertainties• NEWT – NOSTROMO Entry Writing Tool – a Delphi polling tool that captures Uncertainty

NEWT

DATAPAGE

REPORTS

NOSTROMO

Notional Obscurity STatistical Risk Observation MOdel

07T-1552.16

UNCLASSIFIED


New NOSTROMO Concept (U)

NEWT

DATAPAGE

REPORTSNOSTROMO

DEFAULT UNCERTAINTY

COMBINATION

AGC CALCULATOR

NOSTROMO 0.3.16 offers:• multiple methods for inputting Uncertainty• combination of subcomponents• AGC methodology• Pessimistic SLOC estimation• Early Size Estimation

PESSIMISM

EARLY SIZE ESTIMATION

07T-1552.17

UNCLASSIFIED


SI Name

Input Rating Uncertainty UncValPREC 2 2

FLEX 2 2

RESL 2 2

TEAM 2 2

PMAT 2 2

NEW CODE 284161 2 2 FALSE

REVL 2 2

ADAPTED CODE 2 2

% DESIGN MOD 2 2

% CODE MOD 2 2

% TEST MOD 2 2

Assessment and Adaptation 2 2

Software Understanding 2 2

Unfamiliarity with Software 2 2

TIME 2 2

STOR 2 2

PVOL 2 2

RELY 2 2

DATA 2 2

CPLX 2 2

RUSE 2 2

DOCU 2 2

ACAP 2 2

APEX 2 2

PCAP 2 2

PLEX 2 2

LTEX 2 2

PCON 2 2

TOOL 2 2

SITE 2 2

SCED 2 2

Number of Iterations 500

Putnam Technology Rating 2

Run SI

Clear Page

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Plans and Requirements

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Low Uncertainty

Combine Project

AGC Calculator

Default Uncertainty

Delete Project

NEWT

PESSIMISM

Pessimism Worksheet

Early Sizing

NOSTROMO Input Sheet – Notional Data (U)

Software Item Name

Scale Factors

Code Size

Cost Drivers

Iterations

Putnam Productivity

Function Buttons

Conditional Formatting

Pessimism Toggle

Early Size Estimation

07T-1552.18

UNCLASSIFIED


NOSTROMO Early Size Estimation Worksheet – Notional Data (U)

• Allows the user the option to use the Weighted Subcomponents and Weighted Complexity with the Heuristic Model in NOSTROMO to calculate Function Points.

• Heuristic Model estimates the number of Function Points and Backfires them into SLOC.

• Allows the user to copy the results into NOSTROMO Input Sheet.

Language Ada 95

Project Scope Component of SystemProject Class Military ProjectProject Type Communications

Weight Subcomponents Many (9+ Subcomponents) TRUE 1.33

Problem Complexity Majority of simple algorithms and simple calculations

Code Complexity Well structured (small modules and simple paths) TRUE 0.9Data Complexity Several data elements, but simple data relationships

Approximate Value for FPs: 5,799Estimated New SLOC: 284,161

Instructions: Enter Projected development language, heuristic model data, subcomponent data, and complexity data in drop down boxes above.To use weighting, click appropriate box(es). Select "Add To Estimate" to use Early Sloc estimate or "Return" to return to data sheet without.

Heuristic Model

Weight

Early Sizing

Complexity Weight

Subcomponent Weight

Return to MainCopy to Main

07T-1552.19

UNCLASSIFIED


NOSTROMO Outputs – Notional Data (U)

410 KSLOC – Estimated SLOC

Percentile Effort Schedule Size Productivity EAF99% 6637.14 58.47 1054990 158.95 1.6095% 5345.51 54.48 890723 166.63 1.4490% 4763.02 52.47 813874 170.87 1.3675% 3927.86 49.27 699971 178.21 1.2470% 3745.03 48.51 674359 180.07 1.2160% 3436.15 47.17 630457 183.48 1.1650% 3170.52 45.95 592011 186.72 1.1140% 2925.42 44.76 555909 190.03 1.0725% 2559.20 42.85 500702 195.65 1.0020% 2426.91 42.11 480344 197.92 0.9710% 2110.46 40.24 430628 204.04 0.915% 1880.49 38.75 393474 209.24 0.861% 1514.53 36.11 332209 219.35 0.77

Default 3028.34 45.27 603525 199.29 1.04

Language Ada 95

Project Scope Component of SystemProject Class Military ProjectProject Type Embedded - Multi Board

Weight Subcomponents Many (9+ Subcomponents)

Problem Complexity Some difficult and complex calculations

Code Complexity Fair structure, but some complex paths or modulesData Complexity Complex data elements and complex data interactions

Approximate Value for FPs: 8,386Estimated New SLOC: 410,911

Heuristic Model

Weight

Project _Data - ESLOC vs. Effort

0

1000

2000

3000

4000

5000

6000

7000

8000

0 200000 400000 600000 800000 1E+06 1E+06

ESLOC

Eff

ort

410 KSLOC – ESLOC vs. Effort

07T-1552.20

UNCLASSIFIED


Demonstration

07T-1552.21

UNCLASSIFIED


Future NOSTROMO Additions and Efforts (U)

• Addition of Exception Conditions from Ray Madachy’s Expert COCOMO

• Transition to an application environment (MS-Access or .NET)

• Expansion of the NOSTROMO tool to address other COCOMO Family models with uncertainties

• COSYSMO – especially relevant in development of complex DoD systems

• COSOSIMO – System of Systems models are becoming increasingly prevalent in DoD efforts

• COQUALMO – Defect Prediction Models

• Expansion of the NOSTROMO methodology and tool into Readiness Level models

• Software Readiness Levels (SWRL)

• Technology Program Management Model (TPMM)

07T-1552.22

UNCLASSIFIED


Conclusion (U)

Early and accurate estimation of software size metrics provides a method of true software cost estimation

• Estimating software size accurately during the Area of Opportunity will increase the fidelity of the estimate

• When analogous data is unavailable or unreliable, use of a heuristic model can produce accurate metrics for size

• New NOSTROMO methodology incorporates code size estimate heuristic model to provide better answers earlier

07T-1552.23

UNCLASSIFIED


BACKUP

07T-1552.24

UNCLASSIFIED


Distributions of Uncertainty (U)

Setting n-1 Setting n Setting n+1

Level 1 - Certain

Level 2 - Low Uncertainty - Normal

Level 3 - Medium Uncertainty - Triangle

Level 4 - High Uncertainty - Uniform

NOSTROMO assumes ceiling and floor limits of the highest and lowest default values for each COCOMO II Scale Factor and Cost Driver

NOSTROMO changes the distribution and sets the high and low points based on the level of uncertainty with each factor

07T-1552.25

UNCLASSIFIED


NOSTROMO Application (U)

• Microsoft Excel worksheet with attached macros and Visual Basic code

• Currently on version 0.3.16 (as of 10/17/07)

• Does not use outside applications for Monte Carlo simulation (standard random number generation from Visual basic)

• Limited testing performed on a proof-of-concept application

• Suggest limiting Monte Carlo run size to 500-2000 runs

• Generates two pages of output – Data and Charts

• Data page contains all outputs from Monte Carlo simulation, charting data, and histograms

• Charts page contains Confidence Intervals for output of COCOMO II and Putnam models, charts, and histograms

• NOSTROMO – Data Sheet and CM Control Sheet are Developer pages and should not be removed

07T-1552.26

unclassified approved for public release 07-mda-2965 (26 oct 07) load bearing walls: early sizing...

Documents