1© Total Metrics

Implementing aImplementing a‘Mature’ ‘Mature’ FPA FPA

ProcessProcess

Pam MorrisManaging Director

Total Metrics (Australia)Pam.Morris@Totalmetrics.com

WWW.Totalmetrics.com

2© Total Metrics

Pam Morris ProfilePam Morris Profile

• CEO of Total Metrics• Member of the IFPUG Counting Practices

Committee 1993 - 2000• International Workgroup convenor and project

editor ISO/IEC 14143 Functional SizeMeasurement Standards

• President Australian Software MetricsAssociation (ASMA)

• Australian Representative ISBGS Committee• Core project member COSMIC

3© Total Metrics

Measurement is a key toMeasurement is a key tosuccessful softwaresuccessful software

development ………...development ………...

“ You cannot manage what you cannotmeasure.”

“Without objective data you are justanother person with an opinion”

“ If you do not know where you are then amap is no use.”

True?

4© Total Metrics

But … what has historyBut … what has historyshown us?shown us?

• 1993 - “80% of all measurementprograms fail” Source : Howard RubenAssociates 1993

• 2000 - “Metrics usage continuesto decline, with a steeper fall inthe U.S. than in Non-U.S”companies. Source Howard Rubens ITPerformance Trends 2000 (Meta Group)

Why arent we gettingbetter?

5© Total Metrics

Reasons or Excuses?Reasons or Excuses?Functional Size Measurement is a great

idea but….– “It was too hard”– “We did not have enough resources”– “No-one used the results”– “Never had time to do it”– “Were not sure if we were doing it right”– “The results were meaningless, looked

like random numbers”– “Not sure we measuring the same as

everyone else”– “No one cared about what we did”– “We did not have any tools”– etc.

Where did we go

6© Total Metrics

Other IT ProcessesOther IT Processes• The ‘capability’ of an IT organisation is

measured by the maturity of its ‘ITprocesses’

• IT Processes :– Requirements Management– Project Planning– Configuration Management etc

• Mature processes are ones that are:– defined– repeatable and predictable– controlled, measured and monitored– optimised for improvement

• SEI Capability Maturity Model

Initial

Repeatable

Defined

Managed

Optimised

Level 1

Level 2

Level 3

Level 4

Level 5

SEI - CMM MaturitySEI - CMM Maturity Level Level

Adhoc

Disciplined

StandardConsistent

Predictable

Continuouslyimproving

Maturity Levels wheremeasurement can startto become effective

Focus for ImprovementFocus for Improvement– 5 Optimised

– 4 Managed

– 3 Defined

– 2 Repeatable

– 1 Initial

– Process improvement isinstitutionalised. IncludesChange management anddefect prevention

– Product and processes arequantitatively controlledwith detailed measurement

– Software engineering andmanagement practicesdefined and integrated, plustraining

– Project managementsystem in place;performance is repeatable

– Process is informal andadhoc; performance isunpredictable

$ / fp delivered

target

Prob

ality

$ / fp delivered

target

Prob

ality

$ / fp delivered

target

Prob

ality

$ / fp delivered

target

Prob

ality

$ / fp delivered

target

Prob

ality

9© Total Metrics

Was failure our fault orWas failure our fault orthe times we lived in?the times we lived in?

• Predicting the future by measuring chaoswas not viable

• Often the measurement process was asunrepeatable as the process we weremeasuring - double jeopardy!

• Comparing ‘apples’ and ‘oranges’ was‘fruitless’

• We did not have a ‘culture’ that supported– pro-active improvement– acceptance and adherence to

standards– adoption of a repeatable disciplined

approach– use of our results for continuous

improvement

10© Total Metrics

2002 - Now we 2002 - Now we cancan make makeFPA Work!FPA Work!

• IT development processes arematuring

• IT culture is more accepting ofstandards and procedures

• IT management needsmeasurement to support theirmaturity assessment

• IT performance is being questionedand is becoming accountable

But how do we make itwork?

11© Total Metrics

Measurement is also anMeasurement is also an‘IT process’ so to treat it‘IT process’ so to treat it

like one!like one!• Purchasing FPA Training and FPA software tools

is NOT enough!

• Mature sustainable processes need to :– identify and allocate responsibilities for all

components of the the FPA process– implement standards and procedures for:

• collection• validation• storage• analysis• reporting• use of FPA results

– document local interpretations andapplications of industry rules

– continually measure , monitor and improvethe process

12© Total Metrics

Process ManagementProcess Management

4. Improve the process

1. Define the process

2. Measure the process

Execute theProcess

3. Controlthe process

13© Total Metrics

‘Process’ ‘Process’ DefinitionsDefinitionsprocessA system of operation or series of actions, changes,or functions, that bring about an end or resultincluding the transition criteria for progressingfrom one stage or process step to the next. [Reference :IEEE P1220]

process ownershipAll managed processes must be assignedownership that includes responsiblities for design,for establishing and implementing mechanisms formeasuring the process and taking correctiveaction where necessary.( Reference :SEI Guidebook HB003 97)

We also need to measure the measurement

Process

We need to understand the

measurement Process

14© Total Metrics

FPA Measurement as aFPA Measurement as aProcessProcess

We as Measurement Professionals need to:• become as ‘mature’ as the processes we

measure• be able to provide industry standard

‘predictable and repeatable’ results• be able to provide realistic resource

estimates of the measurement effort andcosts

• document the outcome of measurementto an agreed industry wide standard.

• measure the ‘measurement process’and optimise our own improvement

15© Total Metrics

Current Standardisation ofCurrent Standardisation of FPA Method and Process FPA Method and Process

INDUSTRYISO FSM Methods(IFPUG,COSMIC,MKII, NESMA)

ISO 15939Measurement Process Framework)

IT ORGANISATION

FPA Method(Local Guidelines)

Local FPA Process

(?)

✔✔✔

✔

✗

✗

ISO 14143FSM Method

Framework Standards

✔✔✔

ISO ???FPA PROCESS

✔✔✔

16© Total Metrics

Software Process Software Process MaturityMaturity

– The extent to which a specific process isexplicitly defined, managed, measured,controlled, and effective.

– Maturity implies a potential for growthin capability and indicates both therichness of an organization's softwareprocess and the consistency withwhich it is applied in projectsthroughout the organization

( Reference : SEI:SW-CMM)

17© Total Metrics

Level 2 - CMMILevel 2 - CMMIKey Process AreasKey Process Areas

Project FocusProject Focus

– Requirements Management– Project Planning– Project Monitoring and Control– Supplier Agreement Management– Measurement and Analysis– Process and Product Quality

Assurance– Configuration Management

18© Total Metrics

ISO/IEC - 15939ISO/IEC - 15939

Software MeasurementSoftware MeasurementProcess FrameworkProcess Framework

• Defines the activities and tasks that arenecessary to successfully :– identify– define– select– apply– improvesoftware measurement within an overall project

or organisational measurement structure• provides standard definitions for

measurement

19© Total Metrics

ISO - 15939ISO - 15939MeasurementMeasurement

ProcessProcess

•15939 Activities:1.Establish and SustainMeasurementCommitment

2. Plan MeasurementProcess

3. Perform theMeasurement Process

4. Evaluate Measurementand improve

•FPA ProcessApprove FPAProcess

Set up FPAprocedures

Function PointCounting

Review andImprove FPA andIT process

20© Total Metrics

1.1. Establish and Sustain Establish and SustainMeasurement CommitmentMeasurement Commitment•15939 Activities:•managementcommitment to supportmeasurement

•requirements formeasurement areaccepted

•competent peopleare assigned•adequate resourcesare assigned

•FPA Process

Requirements , scope& stakeholders forFPA results areagreed and approved

FPA Trained Counters,Validators, Analystsassigned

Budget is allocated toFPA Process

FPA Process approval

21© Total Metrics

22. Plan Measurement Process. Plan Measurement Process

•15939 Activities:organisational unit

Identify informationneeds

Select measures

Define data collection,analysis, and reportingprocedures(continued….)

•FPA ProcessWhich software to bemeasured

Frequency, Accuracy,type of Counts

Documented Rules forselected Version FPA,

FPA CountProcedures

IFPUG CPM 4.1

Levels of counting

22© Total Metrics

What do we Mean by ‘LevelsWhat do we Mean by ‘Levelsof Counting’of Counting’

Standardized descriptions of exactly howthe count will be conducted and itsdeliverables.

Defines Count:– level of detail– type of count documentation– extent of comments and notes– maintainability– valid uses– error margin– counting rates– benefits and limitations– input requirements

23© Total Metrics

‘Levels of Counting’‘Levels of Counting’

Count Level

Increases

➨ count recording➨ accuracy➨ quality specifications➨ maintainability➨ count cost➨ count effort➨ count usefulness

Decreases1

6

➨Count efficiency

(fps counted per day)

Increases600 -6,000

100

24© Total Metrics

TM Definitions of TM Definitions of‘Levels of Counting’‘Levels of Counting’

LEVEL 1 = Detailed Linked and Labelled Count

LEVEL 2 = Detailed Linked Count

LEVEL 3 = Detailed Count

LEVEL 4 = Default Complexity Count

LEVEL 5 = Rough Count

LEVEL 6 = Size Approximation.

Download full details of Count Levels fromTotal Metrics WWW Site -WWW.Totalmetrics.com

Need to define and document when each typeof Count Level is appropriate for the purposethat the count result will be used

25© Total Metrics

22. Plan Measurement Process. Plan Measurement Process

•15939 Activities:organisational unit

Identify informationneeds

Select measures

Define data collection,analysis, and reportingprocedures(continued….)

•FPA ProcessWhich software to bemeasured

Frequency, Accuracy,type of Counts

Documented Rules forselected Version FPA,Levels of counting

FPA CountProcedures

26© Total Metrics

FPA CountFPA Count ProceduresProcedures• Formal documented standardised procedure

manuals for :– FPA Count Activity– FPA Count Validation Process– FPA Count Issues and Resolutions– FPA Result Reporting– FPA Count Repository Management

• Each Includes:– Document Configuration Control– Roles and responsibilities of participants– Deliverables and outcomes– References to relevant standards– Mapping of FPA process to IT processes– Definitions for each task and activity– Resource, tools and infrastructure requirements

27© Total Metrics

FPA Result Reporting - standards?FPA Result Reporting - standards?• Project Productivity Reporting

– Project FPA Count = 200 or 2000 ? fps– Total Effort = 2000 or 4000 ? hours– Productivity = 2 ?or 10 ? or 20? hrs/ fp

• Whose hours and which FPs?– is the count adjusted or unadjusted?– were the FPs, the ones developed, customised

and/or delivered by the package or a combinationof all three?

– does the count include all functionality delivered bythe package or just the fps actually required by thebusiness?

– did the count include re-used functionality fromanother application not developed by the team?

– does the count only include the businessapplication functionality or does it include changesto functionality of middleware software?

– Do the hours include DBA, QA, Users?

Business Applications A

DeveloperUser

Developers Tools

ExternalBusinessUser

UtilitySoftware

Users Tools

Operating System SoftwareOpsUser

AdminUser

Business Applications B

Screen DriversPrinter Drivers

Applications and their UsersApplications and their Users

TARGET APPLICATION= 1000 fps

External User =primary user

Utility Software=1200 fps

Users Tools = 400fps

Operating System Software = 3000fps

PEER APPLICATION= 700 fps

Operating System Software

COMPLETESYSTEM

MIDDLEWAREAPPLICATIONS

External User =primary user

COMPLETESYSTEM

Project ScopeBusiness UserViewpoint = 600fps

A=500 B=100

C=30 D = 40

E = 50

Project Scope -Project ManagersViewpoint = 720fps

31© Total Metrics

FPA Result Reporting - IndustryFPA Result Reporting - Industrywide Standardswide Standards

• Comparisons need to know each ofthe ‘parameters’ influencing how thesize was derived

• Size Parameters :– type of count– purpose for count– scope of count– users– viewpoint

• Need a standard set of ‘types’ ofreports that include a standard ‘set’of parameters for externalbenchmarking

FPA Result ReportingFPA Result Reporting Standard Set of Report Types Standard Set of Report Types

Financial Reporting (dollars)Gross Sales = $1,000,000Gross Profit = $100,000Net Profit = $55,000

FPA Reporting (fps)End User Impacted Size = 600 fpsProject Impact Size = 720 fpsEnd User Delivered Size = 2050 fps

SIZE PARAMETERS METRICS REPORTINGREPORTTYPE

Viewpoint Users Purpose Scope Count Type ReportedSize

ReportedEffort

Productivity

End-userImpactedsize

ProjectSponsor

Business User Measurefunctionalitydeliveredfor use

ImpactedEnd Userfunctions

EnhancementProject

Project Size600 fps

(A+B)

*ISBGS L46000 hrs 10 hrs/fp

ProjectImpactSize

SoftwareDeveloper

Business UserDevelopersAdministrationOperations

Estimatetotal projectEffort

Allimpactedfunctions

EnhancementProject

Project Size720 fps

(A+B+C+D+E)

*ISBGS L14000 hrs 5.6 hrs /fp

End-userDeliveredSize

Accountant Business User NewBusinessSoftwareAsset Size

Net sizeof enduserfunctions

ApplicationBaseline 2050 fps

(A+B)

N/A N/A

SupportedNet Size

SupportManager

Business UserDevelopersAdministrationOperations

Totalportfoliosizesupported

Net sizeof allfunctions

ApplicationBaseline 5920 fps

(A+B+C+D+E)

N/A N/A

33© Total Metrics

22 Plan Measurement Process Plan Measurement Process(continued)(continued)

•15939 Activities:Define criteria forevaluating the informationproducts and themeasurement process

Review, approve,and staff measurementtask

Acquire and deploysupporting technologies

•FPA Processdefine criteria forrepeatability andaccuracy of counts,counting rates,resource costs

Review proceduresand standards. Planand allocate FPAresources

Schedule training ,select FPA recording,repository, analyissand reporting tools

34© Total Metrics

33. Perform Measurement Process. Perform Measurement Process

•15939 Activities:IntegrateMeasurementprocedures into currentprocesses

Collect data

Analyse data

Communicate the results

•FPA ProcessIntegrate FPA intoProject process, Maprequirementsdocumentation to FPA

Identify count SizeParametersPerform and validatecount

Interpret and AnalyseFPA results for Metricsreporting

Formally documentFPA metrics resultsand report

35© Total Metrics

44. Evaluate Measurement Process. Evaluate Measurement Process

•15939 Activities:Evaluate Measures andMeasurement process

Identify PotentialImprovements

•FPA ProcessMeasure FPA :–efficiency–effectiveness–accuracy–repeatability againstdefined criteria

Identify– strengths andweaknesses– strategies forimprovement FPA & IT–feedback to processand stakeholders

Management

Application Experts

User

Counter

1. Establish1. Establish

People

Recommendations4. Measure and4. Measure andEvaluate ProcessEvaluate Process

Review the Count Process

Review the Count Result

Validation ProcessValidation Process

Validation Review Report

Validation ResultValidation Result

3. Perform FPA3. Perform FPA MeasurementMeasurement

Approved Validated Count

•Plan the Count

•Define Size Parameters

•Establish Application Boundary

•Identify functions and assign points

•Calculate Value Adjustment Factor

CountCount

Analyse and ReportAnalyse and ReportFunction Point Count

• Documented results from Each Step

• Notes, decisions and Assumptions

•Local Count Standards•IFPUG CPM 4.1•Counting Procedures•Validation Methodology•Issues and Resolutions•Reporting Standards•FPA Training Notes•Software ProductFunctional UserRequirements

Training

2. Plan2. Plan

37© Total Metrics

Recommendations forRecommendations forMaturing your FPA ProcessMaturing your FPA Process

• Perform a ‘gap analysis’ - FPA Processversus activities in ISO 15939

• Review ‘Planning Activity’- developstandards for– count process– count validation– count issues and resolutions– count reporting

• Review ‘Evaluation’ (analysis andfeedback) activities in your FPA process– measure FPA process– implement process improvement stragegies

FPA Measurement without a MatureFPA Measurement without a MatureProcessProcess

Successful FPA Process

ReportingStandards

FeedbackEvaluation

Planning

FPAProcedures

39© Total Metrics

Total Metrics Pty LtdSuite 1, 667 Burke RoadCamberwellVictoria 3124 AustraliaPhone +613 9882 7611Fax +613 9882 7633Pam.Morris@Totalmetrics.com

ThankThank You GoodYou Good Luck with your Luck with yourFPA Measurement ProcessFPA Measurement Process ! !

Download full details from : WWW.Totalmetrics.com

1. Article - Implementing a ‘Mature’ FPA Process

2. Article - Infrastructure and Resources required for the FPAProcess