USC CSSE USC CSSE Annual Research Annual Research

ReviewReviewLos Angeles, CA

March 2006

Agile Defense Agile Defense Acquisition:Acquisition:

Possibility or pipe dream? Possibility or pipe dream? Richard TurnerRichard Turner

Systems and Software Systems and Software ConsortiumConsortium

Herndon, VA USAHerndon, VA USAturner@systemsandsoftware.orgturner@systemsandsoftware.org

©2005 USC-SSCI2

The Systems and Software The Systems and Software ConsortiumConsortium

• A not-for-profit collaboration of member companies

• Objective is to improve business performance by enhancing software and systems engineering

• Members benefit by leveraging SSCI Staff’s– technical talent – broad industry-wide

perspective

     

                                

     

Rule #4 : The best companies are the best collaborators

Thomas L. Friedman, The World is Flat. 2005, p. 352

Rule #4 : The best companies are the best collaborators

Thomas L. Friedman, The World is Flat. 2005, p. 352

©2005 USC-SSCI3

TopicsTopics

• The difficulty of line dancing with elephants (a tale of agility)

• What’s new• What’s the same• What are our opportunities

©2005 USC-SSCI4

Elephants Dancing Elephants Dancing IndependentlyIndependently• Individually

fairly reliable• Competitively

differentiated• Don’t do the

same steps• May not be

listening to the same music

• Not particularly attractive as a group

©2005 USC-SSCI5

Elephants Line Dancing – sort Elephants Line Dancing – sort ofof• Looks good early, but…

• The line can break down

©2005 USC-SSCI6

Elephants ResynchingElephants Resynching

• Need ways to check the steps and resynchronize with the leader

©2005 USC-SSCI7

Elephants Resynched and Elephants Resynched and Dancin’!Dancin’!

And off they go!

• Elephants are easy, think about all the humans involved with acquiring complex systems!

• Need a new (or different) paradigm for acquisition

• Evolving value-based risk-driven process– Humans are part of both the value equation and

the risk evaluation

©2005 USC-SSCI8

Acquisition is primarily a human Acquisition is primarily a human activityactivity• Done by humans, with human activities

– Creativity (concepts, requirements)– Negotiation (discussion, trade-offs)– Intimidation (power, authority, process)– Bureaucracy (aversion to change, careers)

• Acquisition success is human-driven– Focus on end user/warfighter– Economic drivers– Political vectors

• Barriers to success are also human– Laws to prevent graft– Regulations that establish and

protect turf– Budget decisions (+ or -)

• Communication is key

©2005 USC-SSCI9

What’s New in Defense What’s New in Defense AcquisitionAcquisition

• Quadrennial Defense Review

• Defense Acquisition Performance Assessment (The Kadish Review)

• Business Transformation Agency

©2006 SSCI10

Quadrennial Defense ReviewQuadrennial Defense Review

• More “jointness” between Services and among warfighter, acquisition and resource communities – Portfolio management– Solution assessment informed by

“technological feasibility and cost-per-increment of capability improvement”

• Emphasizes agility, flexibility, responsiveness and effectiveness– Agile decision making– Risk-based (vice cost-based)

source selection– Time-certain approach

(more later)

©2005 USC-SSCI11

Defense Acquisition Performance Defense Acquisition Performance ReviewReview• Latest in a long line of major

reviews• Led by Gen. Ronald Kadish, USAF

(ret)– Vice President, Booz Allen Hamilton– Former Director, Missile Defense

Agency• Insightful analysis of current

system• Acknowledges disconnects and

instability– Actions taken locally without

consideration of impact– “Gov’t-induced and long-standing

cycle of instability”• Recommends radical changes

“[The Acquisition System] must be agile – to an unprecedented degree – to respond quickly to urgent operational needs from across the entire spectrum of potential conflicts.”

©2005 USC-SSCI12

Current Acquisition System Current Acquisition System (DAPA)(DAPA)

©2005 USC-SSCI13

Pertinent RecommendationsPertinent Recommendations• Create Stable Program Funding Account

and Management Reserve to mitigate budget perturbations

• Delegate authority to PMs to defer non-KPP requirements to future blocks or spirals

• Time-certain Delivery requirements• DDR&E coordinate Service S&T transition

plans• DDR&E actively participate in process at

Joint level• Move Milestone B (decision to move into

System Development and Demonstration) to PDR

©2005 USC-SSCI14

Time-certain DevelopmentTime-certain Development• Timeboxing approach (when

coupled with new PM authority to defer requirements)

• Enforces evolutionary acquisition by making time the focus of the up front requirement statement– Maximum number of years is

mandated (nominal 6 years to MS A)

– Start and end dates are defined– Driving processes (requirements,

budget, source selection, etc.) are revamped to support

• Applies after MS B

©2005 USC-SSCI15

Business Transformation Business Transformation AgencyAgency• Evolved from Business Management

Modernization Program (BMMP) program– Reports to OUSD(AT&L)

• Responsible for DoD Business Enterprise Architecture and Enterprise Transition Plan– BEA is DODAF based (IBM Lead)– Goal: Integrate all DoD business systems into

a single architecture– Compliance to the BEA required for funding

(enforced by Investment Review Boards)

©2005 USC-SSCI16

What’s the sameWhat’s the same

• Emphasis on systems engineering– SEP– Program Support Assessments

• Aversion to using “spiral” and “evolutionary”– Seen as too complex– DAPA report could impact this

• Shrinking industrial sources– Emphasis on COTS-based solutions (even

ships)

©2005 USC-SSCI17

Key technical issues remainKey technical issues remain• Estimation and progress measurement at

the system and system of systems level• Management approaches for complex SoSs

– Supplier chain acquisition and monitoring– Large-scale integration facilities– Simulation-based testing effectiveness, coverage

• Software and information assurance• Integration of SwE and SE within current

acquisition system• Net-centric systems

– Infrastructure availability, reliability

– Information overload

©2005 USC-SSCI18

Opportunities: Agile Opportunities: Agile AcquisitionAcquisition• Rehabilitate spiral

– Revisit USC-SEI workshop results– Reinterpret in light of agility– Leverage QDR and DAPA reports– Make it simpler to implement and recognize

• Disciplined agility• Embrace (or at least acknowledging) change• Interpret and integrate OSD systems engineering

acquisition lifecycle for agility• Develop ways to effectively pilot (simulate?) and

compare/evaluate new acquisition and development lifecycle approaches – including human components– Boehm three-tier Value-based Risk-driven approach– DAPA Time-certain– OSD Multi-V

©2005 USC-SSCI19

Backup Slides And InformationBackup Slides And Information

©2005 USC-SSCI20

The Need for Network-centric The Need for Network-centric Complex Systems of SystemsComplex Systems of Systems• Lack of integration among stovepiped

systems causes– Unacceptable delays in service– Uncoordinated and conflicting plans– Ineffective or dangerous decisions– Inability to cope with fast-moving events

• Increasing NCSOS benefits– See first; understand first; act first– Network-centric operations coordination– Transformation of business/mission potential– Interoperability via Integrated Enterprise

Architectures

©2005 USC-SSCI21

Complexity of Solution SpacesComplexity of Solution Spaces• Large software size: 10-100 MLOC• Number of external interfaces: 30-300• Number of “Coopetitive” suppliers: 20-200

– Even more separate work locations• Depth of supplier hierarchy: 6-12 levels• Number of coordination groups: 20-200

– Reviews, changes, risks, requirements, architecture, standards, procedures, technologies, -ilities, integration, test, deployment, personnel, infrastructure, COTS,…

– Key personnel spend 60 hours/week in meetings

• Multi-platform, hybrid comms, intelligent components

• Unprecedentedness• Emergence• Rapid change• Multi-cultural globalization

©2005 USC-SSCI22

Effect of Waterfall SEMP and Spiral Effect of Waterfall SEMP and Spiral SDPSDP• Delays in starting critical software

infrastructure– OS, networking, DBMS, transaction

processing, …

• Infeasible infrastructure– Premature performance requirements (e.g., 1

second)

• Premature hardware selection overconstrains software– Can also induce premature COTS commitments

• Waterfall-based progress payments undermine-spiral tasks– Develop prototypes or get paid for

specifications

©2005 USC-SSCI23

Emergent Requirements and Emergent Requirements and Rapid ChangeRapid Change• Global connectivity and competition accelerate

emergence and change– More ripple effects of technology, marketplace changes

• Increased need for agility, continuous learning– Need to balance agility and plan-driven dependability– Decline of THWADI (That’s how we’ve always done it)– Beware! Avoid technical agility + administrative THWADI– Keep good THWADI: Your corporate memory

• Hybrid agile/plan-driven processes needed for larger systems

• Need for incremental processes, methods, tools, skills• Need for pro-active technology, marketplace

monitoring– Example: Internet spiral

• Education: Need to learn how to learn

©2005 USC-SSCI24

Rapid Change and High Rapid Change and High Dependability: Need Simultaneous Dependability: Need Simultaneous Agility and DisciplineAgility and Discipline• Discipline for planning, structure,

assurance– Foundations (architecture, organizations)

• Planning for foreseeable change– Thorough V&V

• Agility to handle the environment– Rapid, continuous, often unforeseeable

change– Concurrent engineering

• Of requirements, architecture, hardware and software

• Of development and evolution• Of many suppliers and integrators

– Simultaneous agile and disciplined change management is critical

©2005 USC-SSCI25

A Value-based, Risk-driven A Value-based, Risk-driven ApproachApproach• Framework for integrating systems, hardware,

and software engineering• Distinguishing intellectual content of systems

engineering– Vs. physical-science content of AE, ChE, EE, ME, …

• Clean mapping onto traditional, legacy process milestones (DoD, RUP, MBASE,…)

• Synchronizes and stabilizes concurrent engineering

• Enables both high assurance and agile adaptation to rapid change

• Distinguishes project termination from project failure– Termination: cost-effective windup of calculated

risk• Scalable up to NCSOS and down to small web

applications

©2005 USC-SSCI26

NCSOS Acquisition: More Like NCSOS Acquisition: More Like Doing C4ISR - than purchasing Doing C4ISR - than purchasing fruitcakefruitcake• C4ISR: Command, Control, Communications,

Computers, Intelligence, Surveillance, and Reconnaissance

• No detailed plan survives the first engagement• Acquisition C4ISR via spiral OODA loops

– Observe, Orient, Decide, Act– Vs. Requirements, Delay, Surprise

• Concurrent tasking, collaboration technology essential– Spanning deep chains of command

• Customer, LSI, IPT’s (C4ISR), Decision Support, COP Refresh, Sensor Fusion, Sensors, Sensor components

• Common strategy essential; micro-planning risky• Competition, technology, marketplace ISR essential• Rapid adaptability essential, but• Stable development increments essential as well• How the Government purchasing agents buy

fruitcake...

©2005 USC-SSCI27

Acquisition C4ISR Via Spiral Acquisition C4ISR Via Spiral OODA LoopsOODA Loops

Life Cycle Architecture Milestone for Cycle

Decide on next-cycle capabilities, architecture upgrades, plans

• Stable specifications, COTS upgrades

• Development, integration, V&V, risk management plans

• Feasibility rationale

Act on plans, specifications

• Keep development stabilized

• Change impact analysis, preparation for next cycle (mini-OODA loop)

Orient with respect to stakeholders priorities, feasibility, risks

• Risk/Opportunity analysis

• Business case/mission analysis

• Prototypes, models, simulations

Observe new/updated objectives, constraints, alternatives

• Usage monitoring

• Competition, technology, marketplace ISR

Operate as current system

Accept new system

©2005 USC-SSCI28

Next-Development Increment (DI) Next-Development Increment (DI) Definition Review Preparation: A Definition Review Preparation: A Concurrent Agile SE OODA Loop Concurrent Agile SE OODA Loop • Assess changes during previous DI

– Changes in Operational Concept, key scenarios, COTS, interoperating systems

– Previous-DI scope changes– Feedback from assessments, users– Determine feasible scope of next DI

• Coordinate scope changes with affected stakeholders

– Users, assessors, interoperators, SISOS elements, suppliers, future-DI managers, upper management

• Prepare review content– Scope definition and rationale, SISOS element

requirements, change summary, top-N risk list, candidate pilot users and major external assessments

©2005 USC-SSCI29

Emerging VBRD Method: Emerging VBRD Method: ImplicationsImplications

• Concurrent incremental develop/V&V/rebaseline enables agile/plan-driven process solution

• SOS inception and elaboration phases generate agile/plan-driven product solution

• D/V/R activities – Need different contract incentives– Need different skill strengths, but some rotation is

healthy

• Enables risk-driven tailoring to different solutions

• Technology/Competition/Marketplace monitoring enables value-based competitive agility

• Method scales-up to Future Combat System/JSF-type projects, down to campus web-services

©2005 USC-SSCI30

Increment N Baseline

Increment N+1 Baseline

The Emerging VBRD Approach:The Emerging VBRD Approach: Increment Activites Increment Activites

Rapid Change

High Assurance

Agile Rebaselining for Increment N+1

and V&V

Short, Stabilized Development of Increment N

V&Vof Increment N

Increment N Transition/O&M

Increment N-1 V&V

Unforeseeable Change (Adapt)

Short DevelopmentIncrements

Increment N+1 V&V

Stable Development Increments

Continuous V&V

ConcernsArtifacts

Deferrals

Foreseeable Change (Plan)

©2005 USC-SSCI31

The Emerging VBRD Approach:The Emerging VBRD Approach:A Complete ProgramA Complete Program

SOSInception

SOSElaboration

Agile DI2 (OOD) Re-baseline

DI1 Construction (A)

DI1 V&V

Agile DI3 (OOD) Re-baseline

DI2 Construction (A)

DI2 V&V

Agile DI4 (OOD) Re-baseline

DI3 Construction (A)

DI3 V&V

DI1 Transition, O&M

DI2 Transition, O&M

SOS LCOSOS, DI1, DI2 B/L

LCADI2 LCA

ChangesDI3 B/L LCA

Changes

DI3 B/L LCA

Changes

Capability Drop

DI3 Transition, O&M

Capability Drop

Capability Drop

©2005 USC-SSCI32

The Internet Spiral ProcessThe Internet Spiral ProcessRef: USAF-SAB Information Architectures Study, 1994Ref: USAF-SAB Information Architectures Study, 1994

ApprovedInternet

Standard

ApprovedInternet

Standard

Approved Draft

Standard

Approved Draft

Standard

ApprovedProposedStandard

ApprovedProposedStandard

UnapprovedProposedStandard

IESGApproval

IESGApproval

IESGApproval

IETFReview

IETFReview

IETFReview

Working GroupReview

Working GroupReview

Working GroupReview

FullImplementation

WidespreadImplementation

and Test

MultipleImplementation

and Test

Equipment

Working GroupEvolution

Working GroupEvolution

Working GroupEvolution

UnapprovedDraft

Standard

UnapprovedInternet

Standard

IESG = Internet Engineering Steering GroupIETF = Internet Engineering Task Force

©2005 USC-SSCI33

Functional Solution Analysis

Objectives• Candidate IOC capabilities

and priorities• KPP ranges• Candidate evolution

capabilities

Constraints• Environment: threats,

doctrine, ext. systems• Cost, schedule• Legacy systems• Operational scenarios

Alternatives• Architecture options• Candidate suppliers,

capabilities• Acquisition, support

options

Feasibility evidence and risks

Spiral A IPPD Plans• Required resources• Risk mitigations

Evaluation Frameworks• Prototypes, models• Simulations and facilities• Exercises

Success-critical Stakeholder IPTs

CD

CR Risk?

CD

CR Risk?

A

TD Risk?

A

TD Risk?

B

SDD Risk?

B

SDD Risk?

OC1RROC1RR

SPIRAL AIPPD, Monitoring and

Control

Technology Environment Monitoring

Opportunity, Risk and Change management

RefinedObjectives …

Constraints ...

Alternatives …

Feasibility evidence and risks …

Spiral B IPPD Plans …

Evaluation frameworks …

Success-critical Stakeholder IPTs…

SPIRAL BIPPD, Monitoring and

Control

Technology Environment Monitoring

Opportunity, Risk and Change management

SPIRAL OC1Opt. Engineering Increments:

EI1 IDR, IPR, IRREI2 IDR, IPR, IRR

…EI NAR

Technology Environment Monitoring

Opportunity, Risk and Change management

SPIRAL OC2

…

No

HighLow

No

HighLow

No

HighLow LowHigh

Concept Refinement Technology Development System Development and Demonstration

SDD Cont.

Baselined IOC, requirements

Evaluation Objectives

Life Cycle Architecture

Feasibility evidence and risks

IOC, SDD Plans• Risk mitigation• Acquisition• Support preparation

Initial infrastructure

Evaluation frameworks

Success-critical Stakeholder IPTs

???

OC2RROC2RR

…

The Emerging VBRD Approach: The Emerging VBRD Approach: A Notional Spiral DOD 5000A Notional Spiral DOD 5000

Continue VBRD for Capability Increments

©2005 USC-SSCI34

SISOS Acquisition: Critical SISOS Acquisition: Critical Success FactorsSuccess Factors• Risk-driven spiral processes and

organizations– Project manager’s risk/opportunity team

• Stabilized evolutionary builds– Concurrent plan-driven construction, agile

rebaselining– Anchor point milestones and Feasibility

Rationales

• Rethinking supplier management– Teambuilding and plans/architecture

participation– Balanced agile/plan-driven contracts, award fees

• Knowing when not to system engineer– Example: Product line architecture vs. best-of-

breed supplier selection