1

LaRC Software Process Improvement Effort Sponsor Briefing

July 8, 1997

Presented To:Kristin Hessenius Pat Dunnington

Rob Kudlinski Ron Baker

2

Agenda The Business Case for Software Process

Improvement NASA Langley’s Software Process Improvement

Initiative What we need from our sponsors Summary

3

In light of shrinking budget and staff we must increase productivity and quality of our software development

Software process improvement is a sound investment

Ties to SQF Organizational Value Critical Success Factor

However...Software process improvement will not happen unless you provide the leadership

Why SOFTWARE process improvement?

The Business Case

4

Importance of Software at LaRC

Cross cutting function (under the CIO) In the background across all core competencies Profile of Software at the National Aeronautics and

Space Administration, 1995 Report– Software managers and technical personnel make up over

10% of the total NASA workforce– Software development and maintenance cost NASA more

than $1 billion for its FY93 budget of $14 billion

– These numbers reflect the lower bound Tech Transfer of software products increasing

The Business Case

5

LaRC’s Core Competencies Provide Vital Support to NASA’s Strategic Enterprises

Aeronautics& Space Trans.Technology

HEDS MTPE Space Science

LaRC Core Competencies

Mission & Systems Analysis/Integration/Assessment (Org. Codes - BA, CB)

Aerodynamics, Aerothermodynamics &Hypersonic Propulsion

(Org. Code - DA)

Structures &Materials

(Org. Code - DS)

Airborne Systems & Crew Station

Design & Integration

(Org. Codes - DC, DI)

AtmosphericSciences &

Remote Sensing(Org. Codes - CA, GL)

(Orgs. based on 1994 survey)The Business Case

Computer Wind Labs Simulators Flight AircraftReliances Tunnels Experimental Systems Reliance

(Org. Code - GG) (Org. Code - GM) (757 Project)

6

Software Improvements Based on Capability Maturity Model Framework

Many organizations have had significant software process improvement results / ROI using the Capability Maturity Model (CMM) as a framework

The CMM has become the de-facto standard for measuring improvement in the software industry

Over 477 organizations from 127 companies have conducted SEI software process assessments as of December 1995

The Business Case

7

Software Improvements Based on Capability Maturity Model Framework

(cont.) This framework is widely used by organizations

to:– Characterize the current state of software practice

– Set objectives and priorities for process improvement

– Establish a plan for process improvement and technology insertion

– Assign dedicated resources

– Assess progress against improvement goals

The Business Case

8

Software Engineering InstituteCapability Maturity Model (CMM)

LevelsLevel Characteristics Key Challenges Result

Productivity& Quality

High

RiskHigh

Managed

Defined

Repeatable

Initial

QuantitativeMeasured Process

QualitativeProcess defined and

institutionalized

IntuitiveProcess dependent

on individuals

Ad Hoc

Continuous ProcessImprovement

Changing TechnologyProblem Analysis

Problem Prevention

Process MeasurementProcess Analysis

Quantitative Quality Plans

TrainingTechnical Practices

Process Focus (stds, SEPG)

Project PlanningProject Management

Configuration ManagementSoftware QA

Improvement routinely fedback into the process

Optimizing

The Business Case

9

The Business Significance of Maturity Levels

Level 1: typically are late and over budget Level 2: have generally mastered project

planning and commitment Level 3: significantly improve product

quality Level 4 & 5: make major improvements in

productivity and cycle time

The Business Case

10

Evidence of Capability Maturity Model Impact

Level 1 Level 2 Level 3 Level 4 Level 50

10

20

30

40

50

60

Level 1 Level 2 Level 3 Level 4 Level 5

Cost of Poor QualityRework as % of Development

Effort

Level1

Level2

Level3

Level4

Level5

0

50

100

150

200

Level1

Level2

Level3

Level4

Level5

Schedule

Cost

Schedule/CostOverrun (%)

The Business Case

11

What is the Expected Return On Investment

Examples of Cost Savings– Air Force Logistics Command

» Invested $1,070,000

» Direct savings of $4,792,527

– Raytheon invested $1,000,000 a year for four years for a savings of $15,800,000

– Hughes Aircraft invested $445,000 for a one-year savings of 2,000,000

Typical return on investment between 5:1 and 8:1

The Business Case

12

What is the Expected Return On Investment (cont.)

Process improvement is a sound investment– save more than it costs– improve project cost and schedule performance – improve productivity– produce higher quality products

The Business Case

13

How the SPI Initiative Supports LaRC Current Improvement Efforts

The LaRC Strategic and Quality Framework is the basis for our SPI Initiative

Since much of the improvement activity will involve defining software processes to produce quality products, this effort will help LaRC achieve the SQF strategic goal: “All processes efficiently deliver quality products and services”

By implementing software process improvements many of the quality system requirements under ISO 9001 will be fulfilled (See appendix for table mapping ISO to the CMM)

SPI Initiative

14

SPI Initiative Based on CornerStone Approach

HighHighPerformancePerformance

ModelModel

HighHighPerformancePerformance

ModelModel

CurrentCurrentStateState

CurrentCurrentStateState

Improvement Infrastructure

SPI Initiative

Our SPI High Performance Model is a hybrid of the CMM, ISO 9000, and Baldrige Award Criteria

The combination of the three models will provide a holistic approach for software process improvement in support of LaRC’s SQF

The outcome of achieving the High Performance Model is increased productivity and improved quality

15

How do we increase software development productivity?

Productivity improves because – staff is working to established plans

– development is more orderly

– crises are fewer

– engineers work processes are more efficient

– focus on management and planning reduces schedule and cost overruns

SPI Initiative

16

How do we increase software development quality?

In moving up the levels of maturity, engineers learn techniques to remove defects earlier in the development process– Less errors are present in delivered products

– Since defects are more expensive to fix later in the development process, this saves time and money

SPI Initiative

17

Example of software productivity and quality improvements at NASA

As organizations begin to measure and analyze their processes, they are able to determine error injection points and actually prevent defects– Saving time and money

– Higher quality products » Shuttle avionics software decreased error rates to less than .2

errors per thousand lines of code as compared to the commercial rate of approximately 8 to 10 errors per thousand lines of code

Shuttle software development organization is a Level 5 Causal analysis is performed on defect data to determine injection

points and eliminate them

SPI Initiative

18

LaRC’s SPI Roadmap

ApplyingDefiningPlanningAssessingImprovedDelivery

EffectivenessEstablishing

SQF Organizational Value“Improvement Implementation”

CornerStone “Lay the Foundation”

SPI Initiative

19

CornerStone Schedule Lay the Foundation

– Establish» Plan for the Assessment 6/4/97

» CornerStone Planning and Validation 7/30/97*

– Assess» Workshop Training 8/12/97

» Customer Workshops 8/18/97

» Supplier Workshops 8/25/97

» Follow-Up Interviews (as needed) 8/25/97

» Best Practices Documentation Review 8/18/97

» Solution Analysis 9/10/97

» Prepare and Present Findings 9/11/97 *

– Plan» Develop SPI Plan 9/25/97

» Review SPI Plan 10/2/97 *

» Present SPI Plan 10/14/97 *

Improvement Implementation ….

SPI Initiative* Indicates Sponsors participation

20

CornerStone Participants Candidate organizations

– Atmospheric Sciences Division

– Aero- and Gas- Dynamics Division

– Structures Division

– Flight Dynamics and Control Division

– Flight Electronics Technology Division

– Information Systems and Services Division

– Facility Systems Engineering Division

– Aerospace Electronic Systems Division

(Refer to organization chart in appendix and achieve consensus)

Appraisal Team Members Workshop Participants

SPI Initiative

21

Participant’s Commitment

Assessment Team Members– Participate in all scheduled CornerStone activities

(approx. 25% of time between July 15, 1997 - October 16, 1997)

Workshop Participants– 2-4 members from each division attend a 4 hour

workshop– Attend the Final Findings Briefing and SPI Plan

Presentation

SPI Initiative

22

Products of CornerStone

Outcome for FY 97– Identify and catalogue LaRC best practices– Develop a list of improvement initiatives based

on workshop results– Establish a standing Software Engineering

Process Group (SEPG) to facilitate software improvement initiatives

– Produce Software Process Improvement Plan

SPI Initiative

23

Implementing Improvements

FY 98 and beyond– Assign staff to Technical Working Groups to

implement the highest priority improvements– Processes will be defined and put into place and

training provided– Software Engineering Process Group track and

report progress against plans– Execute supporting RTOP (if approved)

SPI Initiative

24

What we need from our sponsors Provide LEADERSHIP for Software Process Improvement

Initiative Identify target organizations Secure division chief level sponsorship Assign Appraisal Team Members Designate workshop participants Participate in Software Process Improvement Initiative product

reviews Establish a standing Software Engineering Process Group

(SEPG) Provide rewards and incentives for participation

Sponsor’s Role

25

What we need from our sponsors (cont.)

Participate in prioritizing improvements Commitment to staff the Technical Teams to implement the

improvements outlined in the findings Invest in improvement efforts

– Work load will have to be adjusted to accommodate the learning curve associated with the insertion of new technology and processes & additional contractor support will be needed to compensate for the temporary decline in work flow

– Some tools may need to be purchased (For example: Configuration Management COTS product)

Assurance activities to insure compliance with defined processes Sustained support for the SPI effort

Sponsor’s Role

26

Summary Why do we need software process improvement?

– In light of shrinking budget and staff we must increase productivity and quality of our software development

How do we increase software development productivity and quality?– Know where you’re at, know where you need to be and develop a plan to

move the organization forward

What is the expected Return On Investment?– Software process improvement based on the CMM is well documented as a

sound investment

What do we need for long term success?– Your commitment, leadership and support for Langley’s Software Process

Improvement Initiative

Summary

27

Backup Slides

28

How do we increase software development productivity and quality?

Their is an abundance of industry and government data showing quality increases as maturity levels increase

0%

20%

40%

60%

80%

Defects Identified Before Unit TestHughes data on defects found earlier in development

•Maturity Level 2 - 21% before unit test

•Maturity Level 3 - 79% before unit test

Backup Slides

29

How do we increase software development productivity and quality?

Rockwell % post-release defects drooped substantially

– 1986 - 1989 = 28 %

– 1990 - 1993 = 6%0%

5%

10%

15%

20%

25%

30%

Post Release Defect Reduction

Backup Slides

30

Results of Software Process Improvement Initiatives

Schlumberger % on-time deliveries increased from 1990 - 51% to 1992 - 94%

Hughes budgeted to actual cost increased from 1988 - .91 to 1992 - 1.02

Backup Slides

31

What is the expected Return On Investment

Median results from a Software Engineering Institute study of 13 software organizations– Productivity gain per year 35%– Yearly reduction in time to market 19%– Yearly reduction in post-release defects 39%– Business return per dollar invested $5

Backup Slides

32

Appendix

LaRC Organization Chart CornerStone MSProject Schedule Summary Mapping Between ISO 9001 and

CMM