Acquisition of Software IntensiveSystems Conference

Mike NicolASC/EN

937-255-9566Michael.Nicol@wpafb.af.mil

28 January 2003

Revitalizing the SoftwareAcquisition Process

Air Force Aeronautical Systems Center, Engineering Directorate

228 January 2003

ASCASCASCASC

Outline

n Our History

n Today's Environment

n Plan for Improvement

n Next Steps

n Conclusions

328 January 2003

ASCASCASCASC

A Decade Ago...

n DoD 5000.2, Part 6-D, Computer Resourcesn Computer Resources Life-Cycle Management Plan (CRLCMP),

Integrated system development, Software metrics, Software testmanagement, Ada language policy, Software engineering practices

n Air Force Regulation (AFR) 800-14, Life CycleManagement of Computer Resources in Systems

n AFMC Pamphletsn Software IV&V, Software Risk Abatement, Review of Software

Requirements and Interface Requirements Specifications, SoftwareManagement Indicators, Software Quality Measurement, SoftwareDevelopment Capability Assessment

n SAF/AQ Memosn Software Engineering, Software Maturity Assessment, Ada,

Metrics, Software Estimating, Software Reuse, Best Practices, Useof Software Development Capability Evaluation in SourceSelection, Etc.

428 January 2003

ASCASCASCASC

A Decade Ago…(Cont.)

n Development standardsn DOD-STD-2167/2168, MIL-STD-498, MIL-STD-1803n MIL-STD-882, MIL-STD-490, MIL-STD-499, DOD-STD-1521

n Senior software engineer in each program office,supported with additional help, as necessaryn Depending on magnitude of software development effort, program

phase, etc.

n Air Force Systems Acquisition School trainingn Computer Resources Acquisition Course (CRAC)

In spite of all this, success was not guaranteed...

528 January 2003

ASCASCASCASC

Today...

n Limited policy / guidance specific to the acquisitionof software intensive systemsn Almost none of it mandatory

n No standard way of doing businessn Processes across the acquisition enterprise have divergedn Decreasing oversight / insight

n Lack of appreciation for processn Demands for reduced cycle time

n Training available through SAM coursesn Data indicates limited exposure

n Aging and diminishing workforcen 10 year gap for new hiresn Acquisition workforce being rapidly downsized

628 January 2003

ASCASCASCASC

ASC Workload and EN Staffing

0

2000

4000

6000

8000

10000

12000

14000

FY98FY99FY00FY01FY02FY03FY04FY05FY06

$ in

Mil

lio

ns

02004006008001000120014001600

EN

Au

tho

riza

tio

ns

ASC Workload EN Authorizations

728 January 2003

ASCASCASCASC

The Message(s) to PMs...

Be morecredible...

Transform...

Deliver thecapability NOW!Reduce

cycle times...

Take risks...

The budget is $x. We need the system

in n months...

Use a disciplined systemsengineering process...

Implement softwareacquisition process

improvements...

828 January 2003

ASCASCASCASC

A Sample of Findings fromASC Program Reviews

n Incompatible / optimistic performance, effort, andschedule baselines

n Deficiencies in requirements managementn Inadequate risk identification and managementn Processes set aside due to program pressuresn Planned reuse not achievedn Program staffing problemsn Failure to identify and react to problemsn Inadequate program office insightn Labs not fully capable or not in place when neededn Fixed price development contracts with uncertain

requirements or other significant risks

928 January 2003

ASCASCASCASC

An Issue Seen Too Frequently…

n Why?n Programs come with defined cost, schedule, and performance

baselines, often (optimistically) determined without adequateinsight into what actually needs to be done

n All participants challenged to reduce cycle times, take risks, etc.n Requirements are not fully defined / stablen Difficult to estimate the size of a software development effort for

unprecedented systems or where requirements are notcompleten Hence, difficult to estimate development effort and schedule

n History indicates software size estimate grows significantlyduring development

Inability to establish compatible effort, schedule,and performance baselines

1028 January 2003

ASCASCASCASC

Embedded Software Size Growth

0

200

400

600

800

1000

1200

1400

A B C D E F G H I J K

Programs

Siz

e (K

Wo

rds

of

Mem

ory

)

Proposed Actual

119%160

320

576

767516

299

700

387

110

22

531

240

1272

577

48

121%

120%

81%

175%105%

163%84%

93%

100%

238%54

138200

45105

40

1128 January 2003

ASCASCASCASC

Embedded Software Schedule Growth

0 10 20 30 40 50 60 70

1

2

3

4

5

Sch

edu

le D

ura

tio

n (

Mo

nth

s)

Programs

Contract Award

Completion

48

1525

2511

48

2713

127%

36

63 (Est)

67%

31%

108%

28%

1228 January 2003

ASCASCASCASC

Plan for Improvement

n ASC/EN initiative to document and improvesystems engineering processes

n Identify, define, & document technical processesn Combine and simplify current processesn Fully integrate internal processesn Focus on government responsibilities

n Deployn Training, guidance, and monitoring

n Also considering independent lookn Validation of selected processes by outside

organization

1328 January 2003

ASCASCASCASC

Software Acquisition Approach

n Document critical processesn Enterprise support activitiesn Program-level activities (planning and

execution)

n Develop trainingn Target to all who need to know - not just

organic engineering

n Deploy and monitor application ofprocesses

1428 January 2003

ASCASCASCASC

Software Acquisition Approach(Cont.)

n Organizationn Enterprise (Center Level) Support (3)n Acquisition Program Planning Processes (4)n Acquisition Program Execution Processes (14)

n Key practices addressed in acquisition strategy

n Each process documented with briefdescriptionn Purposen Roles and Responsibilitiesn Key stepsn Inputs

n Appendices with additional detail as needed

n Outputs / Productsn Available Tools / Techniquesn Potential Problem Areas / Pitfallsn Lessons Learned

1528 January 2003

ASCASCASCASC

Key Software Acquisition Practices(To Be Addressed in Acquisition Strategy)

n Establish Realistic and Compatible ProgramBaselines

n Provide System Development and Demonstration(SDD) Phase Source Selection Support

n Identify and Manage Computer System andSoftware Risks

n Establish and Manage Software Requirements

n Accommodate High-Assurance Systems

n Ensure Application of Mature DevelopmentProcesses

n Maintain Technical Insight

1628 January 2003

ASCASCASCASC

Enterprise Support

n Provide Advice and Counseln Pre-Acquisition Strategy Panel (ASP) Supportn Source Selection Consultation and Advicen Ensure achievable baselinesn Program Executionn Independent Reviews

n Manage Software Acquisition Training and Experiencen Software Acquisition Engineering Training (Guidebook)n Software Estimation Trainingn Other special-topic training as required

n Collect and Disseminate Lessons Learnedn Collect lessons learned at project/build completionn Establish and maintain lessons learned databasen Disseminate lessons learned through briefings, training, etc.n Implement needed process improvements

1728 January 2003

ASCASCASCASC

Acquisition Program Planning

3.1 Develop Software Acquisition Strategyn Develop program approach to key software acquisition practicesn Get informal, independent review prior to issuing RFP

3.2 Establish Realistic, Compatible Program Baselinesn Estimate software development size (factoring in growth)n Estimate software development effort and schedulen Develop realistic estimate that balances risk, cycle time, etc.

3.3 Support Request for Proposal (RFP) Preparationn Provide key software considerations for RFP Sections L and Mn Solicit and evaluate software size, effort, and schedule estimatesn Solicit software development process documentation

3.4 Provide Source Selection Supportn Evaluate developer capabilityn Evaluate proposed development processesn Evaluate proposed development plann Assess compatibility of processes, plan, effort, and schedule

1828 January 2003

ASCASCASCASC

Program Level Execution

4.1 Identify and Manage Software-Related Risksn Ensure effective risk management process is in placen Ensure all software-related risks are identified and managed

4.2 Establish and Manage Software Requirementsn Ensure software requirements are defined, complete, verified,

consistent and traceable

4.3 Address Training System ConcurrencyRequirementsn Provide for the most efficient method to meet training system

concurrency requirements

4.4 Establish Software Build Plann Ensure there is a plan to define, develop, integrate, and deliver

software increments in response to system requirements

1928 January 2003

ASCASCASCASC

Program Level Execution (Cont.)

4.5 Accommodate Application and Sustainment of Non-Developmental Software (NDS)n Address COTS and other NDS integration

4.6 Accommodate Security Certification & Accreditation(C&A)n Ensure that confidentiality, integrity, and availability is maintained

throughout the life-cycle of the systemn Preclude compromise, exploitation, sabotage, and intentional

damage and destruction

4.7 Accommodate Safety-Critical and High-AssuranceSystemsn Define the process, including what is expected of the developer,

to specify, design, develop, integrate, and verify flight-critical andsafety-critical systems

2028 January 2003

ASCASCASCASC

Program Level Execution (Cont.)

4.8 Establish System / Software EngineeringEnvironment (S/SEE) and Development andIntegration Laboratoriesn Ensure development, integration, and verification environment

requirements are fully definedn Ensure environments are in place when needed and can

provide the required throughput

4.9 Ensure Application of Mature DevelopmentProcessesn Assess developer team process capability prior to contract

award to identify strengths, weaknesses, and risksn Support disciplined application of processes throughout the

development effort

2128 January 2003

ASCASCASCASC

Program Level Execution (Cont.)

4.10 Maintain Technical Insight and ResolveDevelopment Issuesn Implement effective means of communication on program

status and issuesn Take corrective action when necessary

4.11 Establish Software Product Engineering Datan Ensure the minimum set of engineering data and

documentation required for the weapon system software isdeveloped, acquired (or escrowed), and maintained

4.12 Conduct / Support Technical Reviewsn Determine the types of reviews to be accomplished and the role

of the acquisition organizationn Establish relevant entry and exit criteria

2228 January 2003

ASCASCASCASC

Program Level Execution (Cont.)

4.13 Plan for Post Deployment Software Supportn Identify source of support for all software elementsn Determine expected rates of change and expected workloadn Establish required support resources and facilities

4.14 Identify and Collect Lessons Learnedn Survey project participantsn Collect objective datan Share the results

2328 January 2003

ASCASCASCASC

Process Development Schedule

n Complete guidebook draft January 2003

n Complete coordination and February 2003review by SPOs andAFMC SISSG members

n Publish guidebook Version 1 February 2003

n Complete development of March 2003guidebook training

2428 January 2003

ASCASCASCASC

Next Steps

n Consider extending scope to AFMCn Address concerns of other domainsn Add sustainment processes

n Get leadership buy-in

n Work with Air Force Institute of Technology(AFIT) to enhance trainingn Software Professional Development Program (SPDP)n Acquisition-specific training

n Address Section 804 requirements

n Incorporate improvements identified byindependent process validation activities

2528 January 2003

ASCASCASCASC

Conclusions

n We understand the issues and are takingpositive steps to set programs up tosucceed

n Revitalizing our processes is a crucialfirst step

n We can't solve the problem by ourselvesn Balance risk and credibilityn Support disciplined application of processes