Key Issues CausingProgram Delays inDefense Acquisition

J. Michael Gilmore, Ph.D.

Director, Operational Test and Evaluation, Office of the Secretary of

Defense, The Pentagon, Washington, D.C.

Irecently released a memorandum with theUnder Secretary of Defense for Acquisition,Technology and Logistics (USD[AT&L])summarizing two independent assessments ofthe key issues causing program delays. The

USD(AT&L) had chartered a team to assess concernsfrom the acquisition community suggesting that testingdrives undue requirements, excessive cost, and addedschedule into programs. Concurrently, I conducted asystematic review of recent programs to address thequestions of whether testing delays programs, whatother causes create program delays, what is the durationof the delays, and what is the marginal cost ofoperational test and evaluation. The results of bothstudies indicated that testing and test requirements donot cause major program delays or drive undue costs.Our Joint memorandum addressed other problems thatwere identified in the two studies.

The USD(AT&L) study, based on interviews withsenior leaders within the Office of the Secretary ofDefense (OSD) and Service leaders with responsibilityfor program management and oversight, developmen-tal testing, and operational testing, found no signif-icant evidence that the testing community typicallydrives unplanned requirements, cost, or schedule intoprograms. The study team found that tensions areoften evident between programs and the test com-munity and that for the most part these are normaland healthy; however, the study identified fourpotential improvements to these relationships andinteractions:

1. stronger mechanisms for a more rapid adaptationto emerging facts,

2. a requirements process that produces well-defined and testable requirements,

3. alignment of acquisition and test strategies (i.e.,programs lack the budgetary and contract flexi-bility necessary to accommodate discovery), and

4. open communications between programs andtesters, early and often, with constructive in-volvement of senior leaders.

At the time of writing, we are working withUSD(AT&L) to implement changes to the Depart-ment of Defense (DoD) acquisition policy (DoD2008), which we expect will help realize some of thepotential improvements listed above.

Causes of program delaysMy review examined 67 major programs that

experienced significant delays and/or a NunnMcCurdy breach. (The study is available at http://www.dote.osd.mil/pub/presentations.html.) Thirty-sixof the 67 programs experienced a Nunn McCurdybreach, and six programs were ultimately canceled.Two of the 36 Nunn McCurdy programs experiencedno delays to their schedule. We characterized theprograms as exhibiting any of five categories ofproblems that caused delays:

1. manufacturing and development (to includequality control, software development, and inte-gration issues),

2. programmatic (scheduling or funding problems),3. performance in Developmental Testing (DT),4. performance in Operational Testing (OT), and5. conducting the test (such as range availability,

test instrumentation problems, and test executionproblems).

Of the 67 programs, we found that 56 programs (84percent) had performance problems in testing (DT,OT, or both) while only eight programs (12 percent)had issues conducting the tests that led to delays. Onlyone program had delays solely attributed to the test.(The U.S. Army’s Force XXI Battle CommandBrigade and Below [FBCB2] operational test wasdelayed for 1 year because the test unit was deployed.)

DoD T&E ExecutiveITEA Journal 2011; 32: 389–391

32(4) N December 2011 389

Figure 1 shows the distribution of delay cause catego-ries for the 67 programs. There were 158 instances ofissues that caused delays for the 67 programs. Many ofthe programs had multiple problems that fell into morethan one of the five categories of reason for delays.There were eight test conduct problems and 82program performance problems discovered duringtest—an order of magnitude difference. Clearly,programs are most often delayed because of the resultsof testing, not the testing itself.

Length of delaysThe length of delays for the programs examined varied

from none (for two of the Nunn McCurdy programs) to15 years. Thirty-seven programs were delayed by morethan 3 years. The delays were measured against the mostrecent previously published schedule, so in a sense thetotal delays could be even longer relative to the originalplanned schedule. Six of the programs were eventuallycanceled, and one had the Milestone B rescinded.

Cost of OTBoth the USD(AT&L) and the Office of the

Director, Operational Test and Evaluation (DOT&E)studies noted that the marginal cost of testing is a smallportion of the overall program budget; however, the costcan be a large percentage of the budget in the year(s) inwhich it occurs. Because the testing occurs at the end ofthe development process, programs typically have fewdegrees of freedom (and resources) left to work issues.

We evaluated marginal cost to programs of Oper-ational Test and Evaluation (OT&E) as a percentageof total acquisition cost. A review of 78 recent testprograms in the U.S. Army, Air Force, and Navyshowed that the average marginal cost of OT&E is0.65 percent of the total acquisition cost. It alsoappears that some programs truly have negligibleOT&E costs relative to program acquisition costs(OT&E ,0.1 percent) and that most program OT&Ecosts are less than one percent. Few programs that wereviewed (seven out of 78) required more than 1.5percent of program acquisition costs for OT&E. Forprograms with OT&E costs above average, we foundthat low program acquisition cost, expense of testarticles, and test article expendability were thedominant drivers of high relative OT&E cost. Figure 2shows the distribution of the marginal cost of OT&Efor the 78 programs we examined.

SummaryThe Decker Wagner report commissioned last year by

the Secretary of the Army addressed the Army’s failurerate of procuring new development programs (Army2010). The study found that between 1990 and 2010, theArmy terminated 22 Major Defense Acquisition Pro-grams (MDAPs) and that 15 of those terminations haveoccurred since 2001. Further, excluding the FutureCombat System (FCS), the Army spent greater than

Figure 1. Reasons behind program delays: Programs are more

likely to be delayed because of the results of testing vice the

testing itself.

Figure 2. Marginal cost of operational test and evaluation relative to program acquisition cost.

Gilmore

390 ITEA Journal

one billion dollars per year since 1996 on programs thatwere eventually canceled before completion. The studycited many reasons for the failed programs, includingunconstrained requirements, weak trade studies, anderosion of the requirements and acquisition workforce.However, none of the reasons cited included T&E. Infact, earlier and more robust T&E may have revealedproblems and solutions earlier in the program when theywould have been less costly to fix, or allowed decisionmakers to cancel or restructure before wasting billions ofdollars.

Finally, in his recent testimony before Congress, Dr.Carter stated his opinion on reducing the averageacquisition timeline:

‘‘…[acquisition] time is best reduced by ensuringreasonable requirements are set, by being willingto trade away requirements that prove to beexcessive, and by controlling requirements creepso that development time can be constrained. Isupport rigorous developmental and independentoperational test and evaluation to provideaccurate and objective information on thecapabilities and limitations of defense systemsto both acquisition executives and warfightersand to ensure contractors deliver products thatmeet requirements.’’ (Carter 2011) C

DR. J. MICHAEL GILMORE was sworn in as director ofOperational Test and Evaluation on September 23, 2009.

A presidential appointee confirmed by the United StatesSenate, he serves as the senior advisor to the Secretary ofDefense on operational and live fire test and evaluation ofDepartment of Defense weapon systems. Previously, Dr.Gilmore was the assistant director for National Security atthe Congressional Budget Office (CBO). Dr. Gilmore isa former Deputy Director of General Purpose Programswith the Office of the Secretary of Defense, ProgramAnalysis and Evaluation (OSD[PA&E]). Dr. Gilmoreserved with Program Analysis and Evaluation for11 years. Earlier, Dr. Gilmore worked at the LawrenceLivermore National Laboratory; Falcon Associates; andMcDonnell Douglas Washington Studies and AnalysisGroup where he became manager, electronic systemscompany analysis. Dr. Gilmore is a graduate of Massa-chusetts Institute of Technology, Cambridge, Massachu-setts, where he earned a bachelor of science degree inphysics. He subsequently earned master of science anddoctor of philosophy degrees in nuclear engineering from theUniversity of Wisconsin, Madison, Wisconsin. E-mail:mike.gilmore@osd.mil

ReferencesU.S. Army. 2010. ‘‘Army Strong: Equipped, Trained

and Ready,’’ Final Report of the 2010 Army Acqui-sition Review, January 2010. Washington, D.C.:Secretary of the Army.

Carter. 2011. Senate Armed Services Committee.Online at http://armed-services.senate.gov.

Defense Acquisition Program Delays

32(4) N December 2011 391