managing complex projects using evm

8/8/2019 Managing Complex Projects Using EVM

1/38

Managing Complex Projects Using

Earned Value Management

Abba Consulting

DEFENSE+INDUSTRY 07Adelaide

20-23 August 2007

Wayne Abba


2/38

2007 Abba Consulting 2

Agenda

History of EVM (abridged version) DoD Experience

EVM as Key Element of US GovernmentProgram Management Transformation

Office of Management and Budget Government Accountability Office

Federal Aviation Administration

Role of Industry and ProfessionalAssociations

Complexity: Project, Agency, Executive and Legislative Branches

Complexity: Project, Agency, Executive and Legislative Branches


3/38


Earned Value Management: Origins

Industry BestPractices

GovernmentRequirements

Criterion-based Management

Brief statements of attributes Not how-to

Not a system Minimum acceptable standard

1967: DoD Instruction 7000.2Cost/Schedule Control Systems Criteria (C/SCSC)

1997: DoD Regulation 5000.2-REarned Value Management Systems (EVMS)

1960s Complex defense programs Multiple customers Need for improved management

Solution PERT & PERT COST 10 versions by 1964 Industry How to Manage


4/38


EVM Around the World

United States

Australia

Canada Sweden

United Kingdom

Japan

Brazil

France

Israel

Italy Portugal

and others

International Performance

Management Council


5/38


Earned Value ManagementSystems Definition

EVMS will effectively integrate the workscope of a project with the schedule and costelements for optimum program planning andcontrol. The primary purpose of the system isto support program management.

ANSI/EIA 748-98B

Industry StandardGuidelines for EVMS

EVM provides for effective planning and measurement of resource use.

EVM provides for effective planning and measurement of resource use.


6/38


American National Standard for EVMS

ANSI/EIA Standard 748-98B, Earned ValueManagement Systems Published 1998, reissued 2007

Incorporated in governmentregulations

Global Engineering Documents,

http://global.ihs.com/ (US$ 62) 32 Guidelines:

Organization

Planning, Scheduling &Budgeting

Accounting Considerations

Analysis & Management Reports

Revisions & Data Maintenance


7/38


CostVariance

Approx. Time VarianceJ F M A

Time

Resources

400

300

200

100

Time Now

ACWP(ActualCost)

BCWS(Planned

Value)BCWP

(EarnedValue)

Cumulative Trend Analysis

M J J A

Schedule Slip

ScheduleVariance

Risk

Management Reserve

AtCompleteVariance

ProgBudget


8/38


Cost/Schedule Variance TrendAnalysis

Time

Cost Varianceat Completion

ScheduleVariance atCompletion

0

Cost Variance ---Schedule Variance

Management Reserve

Unfavorable

Fa

vorable

ManagementReserve

Consumption

Schedule Slip

Standard chartsStandard charts


9/38


DoD Experience 1982-99

A-10 A-12 B-1 B-2

C-17 DDG-51 F-18 F-22 C-130H & J

Chinook Apache Longbow Blackhawk Kiowa Seahawk Comanche T-45 T-46

CVN-68 SSN-21 SSN-688 SSN-774

T-AOE-6 LHD-1 Crusader M1 M2/M3 FVS

MK-48 ADCAP MK-50 ALWT SSBN-726 CG-47 MH-53 V-22 Osprey P-3 Orion RCAS

LIF Hawk

B-1 Engines

F-414-GE-400 Engines

Chem. Demilitirization

D-5 TRIDENT II

Javelin

Pershing II

FAMECE/UET

Satellites

AN/BSY-2 THAAD

ABL

GBL

* Aggregate overrun 5.5% on115 largest DoD contracts($3.0B on $84.8B @ 60%complete), adjusted for overtarget baselines a/o 4/30/2000

* OUSD(AT&L)ARA/AM, 1/16/01

* OUSD(AT&L)ARA/AM, 1/16/01


10/38

2007 Abba Consulting

A-12 Contract Actual Cost vs. EAC

Contractor and PM EACsboth imply no cost accrualfor several years during keymanufacturing/test phases


11/38


Secretary of Defense on A-12

In canceling the A-12 program in 1991, Mr. Cheney said:

"This program cannot be sustained unless I ask Congress for moremoney and bail the contractors out. But I have made the decision that Iwill not do that. No one can tell me exactly how much more it will costto keep this program going. And I do not believe a bailout is in thenational interest. If we cannot spend the taxpayers' money wisely, wewill not spend it."

Failure or crisis changesbehavior in bureaucracies

Largest contract terminationcase in history

16 years in litigation 5 trials Another decision May 3, 2007

And another appeal Millions of documents $2.5 Billion at stake


12/38

2007 Abba Consulting

Lessons of the A-12

Abiding cultural problem Existing controls, properly applied, would have been sufficient

Too often, earned value insights remain the sole province ofthe supporting program control staff of both contractors andthe government Earned value must be an integral part of the performing design

and manufacturing organizations

Only when program technical staffs are held accountable forearned value analysis, will they begin to understand itsimplications.

Facing empty carrier decks, Navy developed theF/A-18E/F Super Hornet

Awarded to McDonnell Douglas (now Boeing) A-12 cancellation spurred the DoD EVM initiative

The Beach Report, A-12 Administrative Inquiry, 28 Nov 1990The Beach Report, A-12 Administrative Inquiry, 28 Nov 1990


13/38


F/A-18E/F Contract Performance

No major variances Scope control Effective planning EVM used effectively Excellent performance


14/38


F/A-18E/F in the Fleet

Navy Admiral John B. Nathman, vice chief of naval operations,commenting on the smoothness of the Super Hornet fighter jetacquisition program. Explaining what makes the airplane so lovable,he said "It's an efficient, effective platform... under cost, onschedule.

* National Defense Magazine (March 2005)

"I wake up every morning"I wake up every morning

and I want to kiss the Superand I want to kiss the SuperHornet on the lips.Hornet on the lips. **


15/38


Performance Management Laws

Government Performance

and Results Act of 1993 Federal AcquisitionStreamlining Act of 1994,Title V

Information TechnologyManagement Reform Act of1996 (Clinger-Cohen)

Implemented by ExecutiveOffice of the President,

Office of Management andBudget (OMB)

Audited by GovernmentAccountability Office (GAO)


16/38


Executive Office of the President,Office of Management and Budget

Implemented performancemanagement laws in late 1990s Worked with Defense Department

Issued Circular A-11 Part 7 Agencies expected to achieve, on

average, 90% of cost, scheduleand performance goals Program budgets require strong

business case tied to projectplanning using EVM per ANSI/EIA

748-98 Performance is graded Required for government and

contractors


17/38


CIRCULAR NO. A-11

PART 7

PLANNING, BUDGETING,

ACQUISITION, AND

MANAGEMENTOF CAPITAL ASSETS

EXECUTIVE OFFICE OF THE PRESIDENTOFFICE OF MANAGEMENT AND BUDGET

JULY 2007

http://www.whitehouse.gov/omb/http://www.whitehouse.gov/omb/

A-11 Part 7 Updated annually

constantly raising the bar

Exhibit 300 Business Case

Capital Programming Guidereissued 2006

Federal Acquisition Regulationcontract clauses issued 2006

FAR has force and effectof law

A major transformation of US

Government management

PM Certification requirementsissued 2007


18/38


Executive

BranchManagementScorecard

www.results.gov

2001 BaselineEvaluation


19/38


Executive

BranchManagementScorecard

www.results.gov

Quarterly ReportJune 2007


20/38

20

Using EVM as an Audit Tool

GAO began using EVM analysis in 2002 for selectedaudits

Since that time GAO has issued 6 reports using EVManalysis as a basis for findings

Programs are far ranging covering major Automated InformationSystems, Shipbuilding, Missile Defense and NASA

Results vary among programs--Examples include:

Realism of the Performance Measurement Baseline (PMB)

Integrated Baseline Reviews (IBRs) data validation Number of rebaselines Optimistic Estimates at Completion (EACs) Frequency of reporting


21/38

21

Next steps for GAO and EVM

Select members of GAO have been extensively trained touse EVM for audits

These members delivered in-house training to various GAOteams to get the word out about the benefits of using EVM

Ability to quantify historical cost and schedule performance Provide a means for forecasting future costs and schedules Identify areas of poor performance and technical concern Ability to forecast potential problem areas and report to Congress

GAO has ordered Winsight software and will be trained in thenext few months

Next area of focus includes scheduling techniques and tools,with an emphasis on the critical path method

Cost Guide developed 2006-7Cost Guide developed 2006-7


22/38


GAOs Cost Guide

Exposure Draft Issued August 13, 2007 1 year public comment

www.gao.gov GAO-07-1134SP

Comprehensive 20 chapters

17 Cost Estimating 3 EVM

14 Appendices 56 Tables 42 Figures 47 Case studies 17 Checklists 333 Pages


23/38


Why GAO? New name reflects expanded role

Old - General Accounting Office Financial audits 15% of workload

New - Government Accountability Office Program evaluations

Policy analyses and legal opinions

Advocate for truth and transparency in government

Not just what is wrong but best practices

Cost guide intended for auditors and as a resource for agenciesthat lack capabilities of major agencies such as DoD, NASA

OMB requires agencies to use the GAO guide Appendix 9, OMB Capital Programming Guide

This Appendix is based on the GAO's guide to their auditors on howto evaluate an Agency's cost estimating process, and the reliabilityand validity of the data used to develop the cost estimates.


24/38


Preface GAO Cost Guide To effectively use public funds, the government must meet the

demands of todays changing world by employing effectivemanagement practices and processes, including the measurementof government program performance. Legislators, government

officials, and the public want to know whether government programsare achieving their goals and what their costs are. We developed theCost Guidein order to establish a consistent methodology, based onbest practices, to be used across the federal government fordeveloping and managing its program cost estimates.

As resources become scarce, competition for them will increase. It isimperative, therefore, that government acquisition programs deliveras promised, not only because of their value to their users butbecause every dollar spent on one program will mean one lessavailable dollar to fund other efforts. To get better results, programswill need higher levels of knowledge when they start andstandardized monitoring metrics such as EVM so that betterestimates can be made of total program costs at completion.


25/38


EVM at the Federal AviationAdministration (FAA)

OMB budget reduction 2004 GAO High Risk List

Began major transformation 2005 Policy Guidance Training Program assessments

Working teams Systems Engineering, Risk

Management, Cost Estimating,Contracting, Finance, Operations

EVM Council

Executive review EVM at Program Level

Prime/support contractors Government employees


26/38

FAA EVM Assessment Summary(Baseline 2005)

WBS

OBS

WBS/OBSIntegration

WBS/OBSReporting

Avera

ge

Sched

ule/interdependencies

Sched

uleMilestones

TimePhasedBudgetBaseline

BudgetElementsofCost(EOC)

ControlAccountWP&PP

ControlAccountbudgeting

LOEb

udgeting

MR&

UB

Programbudgeting

Avera

ge

Overh

eadresponsibility

Overh

eadbudgeting

Recor

ddirectcosts

Summ

arizeCAsintoWBS

Summ

arizeCAsintoOBS

Recor

dindirectcosts

Identifyunitcosts

MMAS

Indire

ctCostVariances

Avera

ge

Cost&ScheduleVariances

VarianceAnalysisReports

WBS/OBSVariances

CorrectiveActions

EstimatesatCompletion

Avera

ge

TimelyChangePlanning

Recon

cileChange

ControlRetroactiveChanges

LimittoAuthorizedChanges

DocumentPMBChanges

Avera

ge

Guideline 1 2 3 5 Sum 6 7 8 9 10 11 12 14 15 Sum 4 13 16 17 18 19 20 21 24 Sum 22 23 25 26 27 Sum 28 29 30 31 32 Sum

Prog 1

Prog 2 1 1 1 2 1.3 1 1 3 3 3 3 1 2.1 1.5 3 3 2.5 3 3 3 3 2 2.8 2 2 3 2 1 2.0

Prog 3 0.0 0.0 0.0 0.0 0.0

Prog 4 2 2 3 2 2.3 2 1 3 3 3 3 1 2.3 1.5 3 3 1 2.1 3 3 3 3 2 2.8 2 2 2 2 3 2.2

Prog 5 3 3 3 3 3.0 3 3 3 3 3 3 3 3.0 3 3 3 3.0 3 3 3 3 3 3.0 3 3 3 3 3 3.0

Prog 6 3 3 3 3 3.0 3 3 3 3 3 3 3 3.0 3 3 3 3.0 3 3 3 3 3 3.0 3 3 3 3 3 3.0

Prog 7 2 2 3 3 2.5 2 1 3 3 3 3 1 2.3 1.5 3 3 2.5 3 3 3 3 2 2.8 2 2 3 3 3 2.6

Prog 8 0.0 0.0 0.0 0.0 0.0

Prog 9 3 3 3 3 3.0 1 2 3 3 3 3 3 3 3 2.7 2 3 3 2 2.5 3 3 3 3 3 3.0 3 3 3 3 3 3.0Prog 10 1 1 1 1 1.0 1 1 1 1 1 1 1.0 1.5 1 1 1.2 1.5 1.5 1 1 1 1 .2 1 1 1 1.5 1.1

Prog 11 1 1 1 1 1.0 1 1 1 1 1.5 1.5 1 1.1 1.5 2 1 1 1 1.3 1.5 1.5 1 1 1.5 1.3 1 1.5 2 1.5 1.5 1.5

Prog 12 1 2 3 2 2.0 3 2 3 1 3 3 3 3 2.6 3 2 2 1 2.0 3 3 3 1.5 3 2.7 3 3 1.5 3 3 2.7

Prog 13 2 1 1 2 1.5 2 1 1 1 2 2 2 1.5 1.8 1 1 1 1.0 1.5 1 1 1 1 1 .1 1 1 1 1 1 1.0

Prog 14 1 1 1 1 1.0 1 1 1 1 1 1 2 1 1 1.1 2 1 1 1 1 1.2 1 2.0 1 1 2.0 1.4 1 1 1 1 1 1.0

Prog 15 3 3 3 3 3.0 3 3 3 3 3 3 3 3.0 3 3 3 3.0 3 3 3 3 3 3.0 3 3 3 3 3 3.0

Prog 16 0.0 0.0 0.0 0.0 0.0

Prog 17 0.0 0.0 0.0 0.0 0.0

Prog 18 1.5 3 3 3 2.6 2 2 3 3 3 3 3 3 3 2.8 2 3 3 2.7 2 2 2 2 2 2.0 3 3 3 3 3.0

Prog 19 0.0 0.0 0.0 0.0 0.0

Prog 20 0.0 0.0 0.0 0.0 0.0

Prog 21 1 1 1 1 1.0 1 1 1 1 2 1 2 1 1 1.2 1 1 1.5 2 1 1.8 2 1.5 1 1 2 1.2 1 1 1 1 1 1.0Prog 22 1.5 1 1.5 1 1.3 1.5 1 1 1 2 2 2 1 1.4 1 1 2 1 1.3 1.5 1.5 1 1 1 1.2 1 1 1 1 1 1.0

Prog 23 1 2 2 2 1.8 1 1 1 2 3 3 2 2.2 3 3 3 3.0 2 2 2 2 3 2.2 2 2 2 2 3 2.2

Prog 24 3 3 3 3 3.0 3 2 3 2 3 3 3 3 3 2.8 3.0 3 3 2 2.8 3.0 3.0 3 3 3 3.0 3 3 3 3 3.0 3.0

Prog 25

Prog 26 0.0 0.0 0.0 0.0 0.0

FAA Average 1.72 1 .83 2.03 2 .00 1.90 1.75 1 .50 2.06 1.63 2 .28 2.38 2 .42 2.20 1.81 2 .02 1.94 2 .22 2.17 1.29 1.00 2 .04 2.22 2.22 2.06 1 .97 2 .08 2.09 1.94 1.97 2.03 2.03 2 .11 2.02

Agency-Level Guideline not assessed at the Program Level

Based on Unique Aspects of the Program this Guideline was Not Applicable

Program Completely in Either t he Planning or O&M Phase of t he Life Cycle (Guidelines Not Assessed- Transition Plan Provided)

This Pattern Indicates that the Program i s Between a Green and Yellow Assessment Rating

Assessment has NOT been completed

Analysis & ReportingOrganizing Performing & AccountingPlanning Change Management


27/38

27

FAA EVM Assessment Summary(June 1, 2007)


28/38

28

Program Performance

Program NameColor Code -

EVMS Quality:

Green: MeetsYellow: PartiallyMeets

Red: Does NotMeetBlack: NotAssessedCost Variance %

Sc

heduleVarian

ce%

-5%

-10%

5%

10%

Over SpentAhead ofSchedule

Under SpentAhead ofSchedule

Over SpentBehind Schedule

Under SpentBehind Schedule

-5%-10% 5% 10%

>-15%

>15%

XXXXXXXXXXXX

XXXXXXXXXXXX

XXXXXXXXXXXX

XXXXXXXX

XXXXXXXXXXXXXXXXXXXX

XXXXXXXXXXXX

XXX reporting Jan 2007 dataXXX not reporting data

XXXXXXXXXXXX

XXXXXXXXXXXX

XXXXXXXXXXXX

XXXXXXXXXXXXXXXXXXXXXXXX

XXXXXXXXXXXX

not to scaleCV% = 23%SV% = 52%

not to scaleCV% = 23%SV% = -3%

not to scaleCV=4%SV=24%

XXXXXXXXXXXX

XXXXXXXXXXXX

XXXXXXXXXXXX

XXXXXXXXXXXX

XXXXXXXXXXXX


29/38


Management Transformation in Civilian

Agencies

FAA Transformation more than just EVM

Setting an example for public administration

A project-managed agency

Measuring objectively and reporting honestly

Accountable for results With Social Security Administration, leading civilian agencies

Improving the practice of EVM

Managing all program resources Effective and efficient implementation

Consistent among business units and programs


30/38


FAA Transformation Progress

"It takes focus on performance and efficiency measures, dedication,

candor, accountability, and these are all things I see exhibited at the

FAA. You basically are there. You are where I would like thefederal government to be on all 1,200 of its programs, and yourun an operation that is more measurable than others."

Clay Johnson III

Deputy Director for Management

Office of Management and Budget

November 15, 2005

Clay Johnson III

Deputy Director for Management

Office of Management and Budget

November 15, 2005

Communications Plan

On track to meet goals andcommitment to OMB/GAO

Working with contractors

Center of Excellence

Pilot for PMI Research Study

on value of PM


31/38


Meanwhile, Back at the Pentagon -

EVM emphasis declined in early 2000s afterAcquisition Reform era

Leadership changes and inertia Reverted to reporting system

Related skills declined

Cost estimating Systems engineering (MIL-STD-499)

Oversight relaxed in favor of contractor responsibility

Problems emerged in programs

Expeditionary Fighting Vehicle, Ships etc. EVM misapplied

Perverse reporting incentives

Gaming the system


32/38


Naval Audit Service Report N2007-0002

October 24, 2006

Oversight of Earned Value Management for NavalAcquisition Programs

DCMA, the executive agent for EVM, has yet to take any significantactions to address the weaknesses of the implementation andoversight of EVM within DoD and DON identified by our audits andother audit agencies reports. As noted in this and other reports, thelack of action continues to impact key aspects of integratedmanagement processes and practices necessary for successfulmanagement of cost, schedule, and performance of Navalacquisition programs. These continuous and long standingperformance issues can also be attributed to USD (AT&L) not

providing sufficient guidance and oversight to DCMA, as theexecutive agent for EVM.

Devastating critique of Defense managementDevastating critique of Defense management


33/38


Pentagon Response to Audits

Beefed-up Defense ContractManagement Agency EVM Center

More reviews, more data, morebureaucracy, more cost Balance is shifting back to

oversight with no industry input Course correction is needed

Executive understanding Integrated management systems engineering, cost

estimating, contracting, risk management

Hold contractors responsible and accountable with minimaloversight Meet project managers needs for timely, reliable information


34/38


Role of NDIA

National Defense Industrial Association Subject Matter Experts for ANSI/EIA 748-98

Issued companion documents Application Guide

Intent Guide

System Acceptance Guide

Surveillance Guide IBR Guide

OMB cites documents in Circular A-11 Part 7 At OMB request, NDIA opened meetings to civilian

agencies and their contractors

http://www.ndia.org/http://www.ndia.org/


35/38


NDIA EVM Application Guide


36/38


Role of PMI-CPM

PMI College of Performance Management

Education and training conferences United States Spring and Fall

Australia

Japan

Korea

Earned Value Practice Standard Earned Value Community of Interest CoEye

PMBOK(R) Guide--Fourth Edition Project Team

A resource for professional colleagues worldwide Government, Industry, Academia

http://www.pmi-cpm.org/http://www.pmi-cpm.org/


37/38


New AlliancesSubject Matter Experts for EVM

Association for Project Management (UK)

APM EVM Guide, May 2002

National Defense Industrial Association (USA) ANSI/EIA EVMS Guidelines

Agreed as of July 1, 2004:

Documents are equivalent in intent

Review every 3 years or earlier if needed Publicize to government and industry

Encourage recognition in public contractsand other business transactions

New model for international cooperation PMI-College of Performance Management

Practice Standard for Earned ValueManagement 2005


38/38


Summary

For forty years, as countless management fadshave come and gone, Earned Value

Management has endured as the tool of choicefor integrated cost, schedule and technicalperformance management and oversight ofcomplex programs

EVM is the cornerstone of major US governmentmanagement initiatives Openness, transparency and accountability are

essential Synergistic executive and legislative branch interests