risk based quality

Protecting the Public, Astronauts and Pilots, the NASA Workforce, and High-Value Equipment and Property

Mission Success Starts With Safety

Risk Based Quality

July 17, 2001

Quality Leadership Forum

Tom Whitmeyer

Manager, Agency Quality Program

(1)


07/17/2001 (2)

What is the Risk Based Quality Approach?

AnalyzeResultsRe-Deploy

Involvement

Planning

Implementation

Stand alone -self sustaining Supporting project/contract management

Add on, after the fact

IntegratedDiscipline Unique

All or nothing

Active, proportional,” partnering”

Quality

Engineering

ReportFile

Coordination

Communication

Procurement

Finance Property

W B S

1 .1 .1V eh ic le I& T

1 .1 .2 .1P ayload I& T

1 .1 .2 .2 .1S u n S en sor

1 .1 .2 .2 .2M oon S en sor

1 .1 .2 .2 .3S ig n a l C on verte rs

1 .1 .2 .2P ayload S en sors

1 .1 .2 .3A n ten n a G rou p

1 .1 .2 .4R F P rocessor

1 .1 .2P ayload S u b

1 .1 .3S tru c tu re S u b

1 .1 .4E lec trica l S u b

1 .1S p ace V eh ic le

1 .2S ys tem s E n g

1 .3G rou n d E q u ip

1 .4P rog am M an .

1 .5C on trac t D ata

1 .0P rog ram

Built into contract/project

Static, “cut-paste’, “us-them”

Levels of Surveillance in Specific Areas

Surveillance where needed, when needed, to the degree needed

0%100%or

High RiskMore Definition

Low Risk

Increasing Emphasis


07/17/2001 (3)

What are Areas of Increasing Emphasis for Risk Based Quality?

• Based on Government “Requirements”

• Emphasis on “How”

• Sometimes “Cut and Paste”

• Somewhat “Static”

• Quality Compliance

• Government Requirements and Industry Consensus Practices Tailored to Unique Risk

• Some “How”, and Additional Emphasis on “What” and “Why” Based On Risk

• More Requirements Tailored to Risk

• More Evolution as Risk Changes

• Quality Compliance and Additional Focus on Defining and Managing Quality Related Risk

Increasing Emphasis


07/17/2001 (4)

Operate

Build

Test

Maintain

DeploySystem-wide life-cycle analysis and reasoning to identify and eliminate risks

Monitoring RiskIncreasing the level of surveillance as necessary

Awareness, Supplier Assessment System, Improved Delegation Process, Outreach to University/Non-Profits, Conferences, Industry Coordination,

Management Meetings, Risk Based Surveillance Training

Understand “Why” and “How”Ensure knowledge is captured, integrated, and utilized continuously to improve quality

PBMA, Pictorial Reference Book, Technical Reports, Web-sites, Application Notes, Points of Contact, Contract Writing Tool, Capability Models,

Standards Cross Index, Component/System Reliability Data, Surveillance Writing Tools

Policy Risk Based Surveillance

Supplier AssessmentQuality

Problem Reporting and Corrective ActionEEE Parts

Process Control

Sys

tem

Lif

e-C

ycle

What is the NASA Risk Based Quality Strategy?


07/17/2001 (5)

What is the Risk Based Quality Process?

Risk of Performance

2

Performance

1

Risk of Requirements

Solicitation Development

W B S


1 .1 .2 .1P ayload I& T

1 .1 .2 .2 .1S u n S en sor

1 .1 .2 .2 .2M oon S en sor











1 .4P rog am M an .

1 .5C on trac t D a ta

1 .0P rog ram

Acquisition Team

Solicitation RevContract Award

Contract/Grant

W B S


1 .1 .2 .1P ayload I& T

1 .1 .2 .2 .1S u n S en sor

1 .1 .2 .2 .2M oon S en sor







1 .3M an ag em en t

1 .4 .1 .1 .1S /W IV V

1 .4 .1 .1N ew S /W

1 .4 .1 .2N ew L og ic

1 .4 .1C om m an d


1 .5C on trac t D a ta

1 .0P rog ram

Project Team

Surveillance

Contract/Grant Evolution

W B S


1 .1 .2 .1P ayload I& T

1 .1 .2 .2 .1S u n S en sor

1 .1 .2 .2 .2M oon S en sor







1 .4P rog am M an .

1 .1 .2 .2 .3S ig n al C on verters





1 .0P rog ram

Ongoing Technical/Program Risk

Assessments

Cost, Tech, Schedule

“Level” of Surveillance Based on Risk

W B S


1 .1 .2 .1P ayload I& T

1 .1 .2 .2 .1S u n S en sor

1 .1 .2 .2 .2M oon S en sor







1 .3M an ag em en t

1 .4 .1 .1 .1S /W IV V

1 .4 .1 .1N ew S /W

1 .4 .1 .2N ew L og ic

1 .4 .1C om m an d



1 .0P rog ram


07/17/2001 (6)

WB

SS

Formulation ImplementationSRR PDR CDR FRR OperationsProduction

WB

S

WB

S

Top-Level

Rqmts .

Trades

Concepts

SystemSpec.

EndItem

Specs.

Analysis

Prelim.Drawings

DetailedDrawings

Solicitation

Production/Manufacturing

Package

H/W & S/WVerification

Plan

Acceptance/COFR

Evolving RiskProfile

Identified Risk

Identified Risk

How Does Risk Based Quality Evolve During the Product/Process Life Cycle?

CloseOut


07/17/2001 (7)

How Do We Evaluate “Risk of Requirements” ?

Accepted Risk

OrganizationalEvaluation

Process Evaluation

Tech ReportsLessons Learned

PRACAComponent

Failure History

HeritageProcurementFinance Risk

Start with Solicitation Development

Data

Requirements

Risk

Evaluation

Technical Risk

Cost Risk

Acquisition Team - Project Team

System Evaluation

Schedule Risk

TRL Evaluation

Fault Tree - FMEA

PRA

Other Models

Contract Risk Profile


07/17/2001 (8)

How Do We Evaluate Risk of Performance”?

Accepted Risk

ContractorCapability

ContractorPerformance

ProjectReportsReviews

FinancialReporting

ProcurementData

Audits DCMADCAA

Start with Contract Award

Data

Performance

Risk

Evaluation

Technical Risk

Cost Risk

Acquisition Team - Project Team

SupplierPerformance Evaluation

Schedule Risk

GIDEPAlerts

Assessments

Analysis

Reporting

Property Data

Contractor Risk Profile


07/17/2001 (9)

Where Does Risk Based Quality Fit Within NASA Policy?

NASA Policy for Safety and Mission SuccessNPD 8700.1

SMA for Spacecraft, Instrument,

ELVs

NASA OrganizationNPG 1000.3

NASA Strategic PlanNPD 1000.1

NASA Strategic Management Handbook

NPG 1000.2

Safety and Health

R & M Quality

http://www.hq.nasa.gov/office/codeq/doctree/qdoc.htm


07/17/2001 (10)

Why Do Risk Based Quality Initiatives Focus on Acquisition/Awards?

$13.6 Billion Total Appropriation

NASA FY 2000 Appropriations vs. Procurement Awards

19%

81%

All other NASA cost such as payroll and

facilities

Cost of Contracts, Grants and Cooperative Agreements

(Includes $1.3B for JPL)

Over 90% of NASA’s FY 2000 Contract

Obligations Utilized Performance Based

Contracting


07/17/2001 (11)

Why is Industry Involvement Important?

US Aerospace Industry Sales by Product Group (2000)

Aerospace Spending is $144 Billion

NASA is about 40% of U.S. space sales,

and 9% of the overall U.S. Aerospace spending

Aviation is 55% percent

of U.S. Aerospace Spending

Space22%

Mil Air23%

Missile7%

Civil Air32%

Other16%


07/17/2001 (12)AEROSPACE SUPPLY CHAIN

Tier 2Integrators /

Source Control/Software Dev./

Major Assemblies

Tier 3 - Integrators /Source Control / Specialty

Electronics / Wiring / Compnts

Shared Supply Base

Tier 4 - Make to Print / Machine Shops/ Processors

Tier 5 - Distributors (Fasteners, Raw Materials, Commodities, Adhesives, Special Materials)

Tier 6 - Raw Materials (Castings, Forgings, Sheet, etc.)

What Are the Challenges for Contracts?

Tier 1Vehicle / Airframe /Propulsion Manufacture

NASA Major Primes

60% to 80% of activity is sub-contracted

Eight ThousandActive Contracts

Products and Services

Requirements


07/17/2001 (13)

What Are the Challenges for Non-Profits and Educational Institutes?

0

50,000,000

100,000,000

150,000,000

200,000,000

250,000,000

300,000,000

350,000,000

400,000,000

BOSTON UNIV

COLUM

BIA U

NIV

GEORGIA T

ECH APPLI

ED RES C

ORP

MASS IN

STITUTE T

ECHNOLOGY

OREGON STATE U

NIV

RESEARCH TRIA

NGLE IN

STITUTE

SOUTHWEST R

ESEARCH INSTIT

UTE

SUMM

A TECHNOLO

GY INC

UNIV A

LABAM

A HUNTSVIL

LE

UNIV A

RIZONA

UNIV C

ALIF B

ERKELEY

UNIV C

ALIF IR

VINE

UNIV C

ALIF S

ANTA BARBARA

UNIV C

OLORADO B

OULDER

UNIV C

OLORADO B

OULDER

UNIV F

LORID

A

UNIV M

ARYLAND C

OLLEGE P

ARK

UNIV M

IAM

I

UNIV N

EW H

AMPSHIR

E

UNIV T

EXAS AUSTIN

UNIV W

ISCONSIN

MADIS

ON

UNIVERSIT

IES S

PACE RESEARCH

FY 2000

Act

ive

R&

D A

war

ds

66 University Awards above

$5million

7,375 Grants

$2.0 Billion Total

1,414 Cooperative Agreements

$2.9 billion total

FY 2000 Active Awards

Estimated Value at Non-Profit and Educational Institutes


07/17/2001 (14)

What Are Some of the Risk BasedQuality Task In Work?

Supplier Assessment System

Capability Models

Standards Index

Check sheets

Maintain

Workmanship

Pictorial Reference

Application Notes

Testing

Quality Planning

Policy

Training

Interviewing Tool

Writing ToolW B S


1 .1 .2 .1P ayload I& T

1 .1 .2 .2 .1S u n S en sor

1 .1 .2 .2 .2M oon S en sor











1 .4P rog am M an .


1 .0P rog ram

Risk Based Surveillance

Surveillance web site

Surveillance training

NEPAGEEE Parts Risk Management

Vendor StatusProblem Advisory

Coordination

Quality Leadership Forum

Industry / Associations

(AIA, ADQMC, AAQG)

EMC, PMC, S&MA, Procurement

DCMA

ISO TC 20 SC 14

NASA/ESA/NASDA

KnowledgePBMAFilming

Technical Questions

Outreach

Process Control AwarenessQuality “Road Show”


07/17/2001 (15)

What are the QLF Expectations?

• Today we will be discussing many of these risk based quality tasks.

• Many have just started.

• There are many more at the Center level that have not been mentioned.

• The Quality Leadership Forum will continue to play an active role in guiding the development of these activities and sharing ideas.

risk based quality

Documents