environmental, safety, and occupational health (esoh) programmatic risk tool aeronautical systems...

Environmental, Safety,

and Occupational Health

(ESOH)

Programmatic Risk Tool

Aeronautical Systems CenterAeronautical Systems Center

Amy Mercado VinceASC/RAPDSN [email protected]

Dominant Air Power: Design For Tomorrow…Deliver Today

2

Birthplace, Home and Future of Aerospace

Overview

• New Acquisition Program Guidance

• ASC’s Response to New Guidance: ESOH Programmatic Risk Tool

• Benefits, Teaming, and Summary

• Questions/Comments

3


ESOH Programmatic Risk ToolBackground

Requirement: ASC should more effectively manage ESOH-related risk, keep program managers informed of ESOH issues, and comply with new direction in DoDI 5000.2

Purpose: - To inform you of the tool ASC is developing to address this requirement- To show you what to expect from your ESOH professionals with respect to the systems engineering process

4


ESOH Programmatic Risk ToolSystems Engineering Application

12 Sep 04 USD (AT&L) Policy Memo, Defense Acquisition System Safety, requires PMs to:

1) Use MIL-STD-882 to integrate ESOH risk management into SE

2) Report status of residual risk acceptance decisions at technical and program reviews

DoDI 5000.2, in concert with the application of the MIL-STD-882 system safety methodology, make up the DoD's systems engineering (SE) approach for eliminating ESOH hazards or minimizing ESOH risks across the entire system life cycle.

5



Q: Why Establish an ESOH program early?

A: Reduces program risk

•Reduces costs: – Improves system design early – Reduces cost of implementing needed changes (i.e.,

technical manuals and training)•Reduces schedule risk:

– Lowers potential for major redesign – Lowers potential for additional testing

•Reduces performance constraints: – Minimizes operational limitations – Avoids environmental restrictions

6


DoD Acquisition Process

PESHE: A living management/decision making document that guides identification and management of ESOH risks.

A PESHE is the ONLY DoD-required ESOH document!

PESHE

Concept Refinement

Concept Decision

System Development & Demonstration

Design Readiness Review

Production & Deployment

FRP DecisionReview

Operations & Support

Technology Development

A B C FOCIOC

Start

Initial Completion

Update Required

Starts as a planning document

ESOH Risks

Becomes an ESOH Risk Management Tool

7


Overview





8


Past ESOH Metrics/Standards ReportingASC Reconnaissance Wing, CY 2000

EnvironmentalQuality Perf. Indicators (EQPIs)

CurrentNote: RAP and Sr. Year will be combined for the next EWG

EnvironmentalQuality Perf. Indicators (EQPIs)

CurrentNote: RAP and Sr. Year will be combined for the next EWG

NEPA Doc. ODS Elim. HRASenior Yr. Green Green GreenDet 3 Green Green GreenRAB Green Green GreenRAJ Green Green GreenRAV Green Yellow GreenRAP Green Green GreenRAI Green Green Green

Prg. Doc. (Rev.) Prg. Doc. (Inp.) ESH Plan. ESH Integ. ESH TrainingSenior Yr. Green Green Green Green GreenDet 3 Green Green Green Green GreenRAB Green Green Green Green GreenRAJ Green Green Green Green YellowRAV Yellow Green Green Yellow YellowRAP Green Green Green Green YellowRAI Green Green Green Green Green

Program Ofc.

ASC’s old metric and reporting system

• Issues could be masked in the roll up within topics

ASC’s old metric and reporting system

• Issues could be masked in the roll up within topics

9


ASC’s Plan ESOH Programmatic Risk Tool

• Provide program managers an assessment of the “health” of their ESOH program(s)

• Quantify and qualify the ESOH programmatic risks

• Facilitate development of the PESHE

• Provide program managers an assessment of the “health” of their ESOH program(s)

• Quantify and qualify the ESOH programmatic risks

• Facilitate development of the PESHE

• Provide a ready means of addressing the ESOH-related regulatory requirements

• Provide reports to highlight issues

• Make more effective ESOH programmatic decisions

• Provide a ready means of addressing the ESOH-related regulatory requirements

• Provide reports to highlight issues

• Make more effective ESOH programmatic decisions

Goals

Bottom line: Give Program Managers a tool to effectively manage ESOH risks and resources

10


ESOH Programmatic Risk Tool

– Uses reverse failure tree analysis to assess program’s management effectiveness

– Applies 5M model construct– Weighted questions developed from Success Tree

Three Primary Components:

Success Tree, ESOH Risk Manager, and PESHE Template

Program Assessment

11


ESOH Programmatic SuccessTool Work Breakdown Structure

ESOH Program Success(Planning & Management

Template)

Man(Team)

Organic ESOHProfessionals

IPTs/WorkingGroup

FunctionalTeam

Support

MissionSuccess

Goals/Objectives/

Targets

Management (Guidance,Procedures Standardization,

& Enforcement

Machine(Tools)

Media

Legal/Regulatory

Reqmts/Stds

PESHEProcess

AcquisitionStrategy

Integration

ContractorESOH Staff

Regulators/Stakeholders

Results/Actions

SystemSupport

ESOHInformation,

Data, Reports

Responsibil-ities/

AuthoritiesFunding

Techniques

Coordination

ManagementOversight/

Audit

ESOH SystemEngineeringIntegration

ESOHCompliance

P2/HAZMATReduction

NEPAProcess

Life CycleCost

Identification

ESOH RiskMin/MgtProcess

DEMILIntegration

Hazard I MgtProcedures

ProgressTracking

OSS&E

HazardAnalysis

12


Mission Program Success


Success Tree, ESOH Risk Manager, and PESHE Template

– Extracts answers from questionnaire to formulate risk– Uses the six ORM Steps– Identifies benefits associated with addressing risks– Program management tool for ESOH issues

ESOH Risk Manager

13


Identifying & Assessing Risk

Questions provide basis to evaluate compliance with ESOH requirements and implement ORM process

Questions provide basis to evaluate compliance with ESOH requirements and implement ORM process

14


Mitigation Plans/Tracking Progress

Mitigation plans can be built to achieve compliance for all required actions


Allows continuous review process to mitigate risks, make decisions, and keep progress on track

15


MPS Summary ReportExample

16


4.3 Preliminary Risk Assessment4.4 Design Analysis4.6 Part and Material Selection4.7 Concept Studies and Analysis4.19 Design for Assembly6.11 NEPA Schedule10.3 Occupational Health Issue10.4 Technical Risk Assessment10.7 Air Conformity Issues10.10 HAZMAT Issues

Impact

0.90.70.50.30.0100%

Low Risk80%

60%

Medium Risk40%Compliance

20%

High Risk0%

Impact

0.90.70.50.30.0100%

Low Risk80%

60%

Medium Risk40%Compliance

20%

High Risk0%

Com

plia

nce

. 4.3

. 4.4

. 4.6

. 6.11

. 4.19

. 10.4

.10.3

. 10.10

. 10.7

. 4.7

Risk Matrix ReportExample

17




5M Model--Assess Program & Identify Risks

Man Media

Machine

Management

Goals/Thresholds/Objectives

Results/Analysis/Actions

MISSION





Operational Mission

5M Model Implementation

The ESOH Program Success portion is based in part on the 5M Model

The ESOH Program Success portion is based in part on the 5M Model

The 5-M model includes Man, Machine, Media, Management, and Mission

• Man, Machine, and Media interact to produce a successful Mission

• Management provides the policies and procedures to govern the interactions between Man, Machine, and Media

• Management is often the controlling factor in operational success or failure

The 5-M model includes Man, Machine, Media, Management, and Mission

• Man, Machine, and Media interact to produce a successful Mission

• Management provides the policies and procedures to govern the interactions between Man, Machine, and Media

• Management is often the controlling factor in operational success or failure

AcqESOH

Mission

18




Success Tree, Risk Manager, and PESHE Template

– Award-winning “cookbook” format that reduced PESHE development cycle time and cost by 97%

– Achieves format and content standardization across ASC

19


Overview





20


ESOH Programmatic Risk Tool Benefits

Supports resource allocation: funding, manpower, schedule

Assesses ESOH program health through planning and execution phases

Quantifies programmatic ESOH risks

Assesses regulatory compliance

Provides reports to highlight issues

Builds mitigation plans

Provides progress tracking methodology

Measures performance: ESOH goals/objectives/standards

Will promote detailed understanding of ESOH requirements to facilitate decision making

21


ESOH Programmatic Risk Tool Benefits

Promotes standardization across Acq Program Offices– Greater E, S, and OH synergy – Facilitates system life cycle updates by crossfeeding information to

PESHE

Provides examples of risk minimization and acceptance benefits and burdens from similar weapon systems– E.g., HAZMAT substitution success; NEPA issue

Ensures compliance with DoDI 5000.2 and MIL STD 882– Provides a solid risk mitigation and tracking mechanism– Records decision making process: lessons learned will result in fewer

ESOH burdens

Will enhance the Acquisition and Operational missions

22


Teaming

• Strong support from Pentagon: SAF/AQRE, SAF/IEE, AF/ILEV

• Space and Missiles Center and Missile Defense Agency joined ASC’s team– Space and missiles requirements will be incorporated into work

breakdown structure and questions

– Alpha phase will include Space-Based Space Surveillance, Patriot, and Airborne Laser programs plus select ASC programs

– Kickoff team meeting planned for May in DC

• Electronic Systems Center and Air Armament Center will be invited to participate in beta testing

• Army and Navy will be invited

23


Milestones

• Initiate teaming with other centers• Initiate teaming with other centers

• Beta Test– Open to all program offices– Populate and assess tool and users’ guide

• Beta Test– Open to all program offices– Populate and assess tool and users’ guide

• Alpha Test– Train users & populate the tool– Assess effectiveness

• Alpha Test– Train users & populate the tool– Assess effectiveness

• Refine tool and user’s guide• Refine tool and user’s guide

• Finalize tool, user’s guide, and deploy (ECD Spring 06)• Finalize tool, user’s guide, and deploy (ECD Spring 06)

• Pre-Alpha Evaluation– Evaluate/rate the templates/questions

• Pre-Alpha Evaluation– Evaluate/rate the templates/questions

24


Summary

• Historically, PESHE used for ESOH management resource decisions

• ASC has defined, documented results based upon PESHE application

• ASC’s response to new DoDI 5000.2 and MIL STD 882 guidance will result in a more hard-hitting, meaningful evaluation processes based on risk, integral to the systems engineering process

• Alpha and beta testing with new team members will refine program

25


Overview


• ASC’s Response to New Guidance: Acquisition Mission Program Success and PESHE Template



Questions/Comments?

28


BACKUP

29


1. Identifythe Hazards

2. Assessthe Risks

3. Analyze Risk Control

Measures

4. MakeControl

Decisions

5. Risk ControlImplement

6. Superviseand Review

AFPAM 91-215, Section A, Figure 2

The ORM 6-step Model

30



12 May 03, DoDI 5000.2, Operation of the Defense Acquisition System:

Requires Program Managers (PMs) to provide a Programmatic Environment, Safety, and Occupational Health Evaluation (PESHE). Required at Milestones B & C and Full-Rate Production Decision Review.


31


New PESHE Guidance

DoDI 5000.2 revised and released in May ’03

Provided new Program Environmental Safety Health Evaluation (PESHE) requirements and direction.


DoDI 5000.2

PESHE format• Risk Management• Acquisition Strategy

Summary Report

PESHE format• Risk Management• Acquisition Strategy

Summary Report

Stresses:

• Documented Risk Management Process: ASC uses Operational Risk Management (ORM)

• National Environmental Policy Act (NEPA) scheduling

• ESOH Integration into Systems Engineering

• Less prescriptive in format

Stresses:

• Documented Risk Management Process: ASC uses Operational Risk Management (ORM)

• National Environmental Policy Act (NEPA) scheduling

• ESOH Integration into Systems Engineering

• Less prescriptive in format

32


PESHE History

Original guidance in DoD 5000.2-R

– Detailed explanation of PESHE format

– Liberal guidance on risk tracking and reporting

Original guidance in DoD 5000.2-R

– Detailed explanation of PESHE format

– Liberal guidance on risk tracking and reporting

The PESHE: A single, live document summarizing the Single Manager’s ESOH strategy

The PESHE: A single, live document summarizing the Single Manager’s ESOH strategy

DoDI 5000.2-RDoDI 5000.2-R

33


ESOH Programmatic Risk ToolThree Primary Components

ESOH Program Success Tree Questionnaire

ESOH Program Success Tree Questionnaire

ESOH Risks

ESOH Risk Manager

ESOH Risk Manager

ESOH Risk Management

• Apply ORM

• Reports

• Action Plans

PESHEPESHE

ESOH Grade Card

• Program Integration

• System & Health Safety

• NEPA• Compliance• HAZMAT• P2

34


ESOH Programmatic Risk Tool’s Software Platform

TRIMS: Technical Risk Identification and Mitigation System (US Navy)

• Process-oriented tool– Evaluate ESOH requirements– Identify hazards– Assess risks– Analyze risk control

measures– Make control decisions– Risk control implementation– Supervise and review– Assign responsibilities– Integrate with PESHE

• Windows application software

• Process-oriented tool– Evaluate ESOH requirements– Identify hazards– Assess risks– Analyze risk control

measures– Make control decisions– Risk control implementation– Supervise and review– Assign responsibilities– Integrate with PESHE

• Windows application software

35


ESOH Programmatic Risk ToolModel Development and Tailoring

Categories: Process StepsCategories: Process Steps

Templates: Sub-processes where risk levels are assigned

Templates: Sub-processes where risk levels are assigned

Questions: Methodology to accomplish ORM:

• Compliance and risk level determination

• Responsibility assignment • Mitigation plans and

progress tracking

Questions: Methodology to accomplish ORM:

• Compliance and risk level determination

• Responsibility assignment • Mitigation plans and

progress tracking

36


Template/Question Development for TRIMS

We are finalizing the weighting

QuestionsReference/ Source

Document Why it is important/otherQuestion Weight (1 to 10)

1.Has the PM, in concert with systems engineering and associated system safety professionals, determined what system safety effort is necessary to meet program and regulatory requirements?

MIL-STD 882d, para A.4.3

Before formally documenting the system safety approach, the program manager, in concert with systems engineering and associated system safety professionals, must determine what system safety effort is necessary to meet program and regulatory requirements.AFI 92-202, para 9.2 also reads, “The PM shall ensure that system safety engineering is an integral part of the systems engineering process and receives proper management attention.”

2. Has the program chief engineer incorporated a robust systems engineering program, including configuration management to attain OSS&E assurance?

MIL-STD 882d, para 11.5, subpara 7

37


ESOH Programmatic Risk Tool Process

ManagementReports

PESHE Data &

Information

ESOH Program Success

ESOH Risk Manager

Program Mgt Assessment

Concept Refinement

Risks

Ops & Spt Risk

Prod & Deploy Risk

Sys Dev & Demo Risk

Tech Devel Risk

ORM Process


Benefits of ESOH Risk Min

Success

Costs of ESOH Risk Min Failure

ESOH Pgm Plan & Mgt

ManagementAction Plans

38


Summary Report Example

39


Sample WBS in TRIMS

40


ESOH Programmatic Risk ToolResource Benefit

How will the Program enhance/sustain resources?

Identifies ESOH risks

Identifies ESOH risks

Provides tools for managing risks

Provides tools for managing risks

Identifies potential solutions

Identifies potential solutions

The Program should enable management to find cost effective solutions in less time

Identifies Life Cycle Costs

Identifies Life Cycle Costs

Facilitates Risk vs. Cost Decisions

Facilitates Risk vs. Cost Decisions

41


Sample Question in TRIMS

42



43



44


Sample Action Plan in TRIMS

45



46


Sample Action Plan in TRIMS

47


Sample Question with

Why is this important? Screen in TRIMS

48


Mitigation Plans/Tracking Progress



Allows continuous review process to mitigate risks, make decisions, and keep progress on track

49


Generated Reports

Summary Report

Summary Report

A graphical depiction of risk levels

Detailed Report

Detailed Report

Data from TRIMS’ questions and mitigation plans

Risk Matrix Report

Risk Matrix Report

A conventional probability vs. consequence risk matrix

Data in these reports can be tailored to provide managers the information necessary to make decisions

50


Example NEPA Issue

• A major aircraft program planned part of its OT&E at an Army test range -- It was believed an existing test range EIS on the would cover the aircraft OT&E

• It was determined that a new EIS was required on the test range to support the OT&E

– This determination was ascertained 7 months before certification and 9 months out from start of testing

– A new EIS would have taken at least one year

– The program's OT&E plan was restructured to eliminate the need for that test range

• This example shows how external forces can affect a weapon system program and that continual attention to ESOH issues minimized the adverse impacts on the program

Source: PESHE Development Guide for PMs, Dec 03, ESC

51


HAZMAT Substitution Success

• Aircraft Radome Chemical Depainting involved MEK, an EPA-listed chemical solvent

• Searched for alternative process• Selected a chemical alternative, compatible with the radome

materials & capable of removing the protective coatings– Solvent is a blend of dibasic esters [DBE] w/ low vapor pressure and

low toxicity

• Benefits– Easy to use– Goes further [less expensive to use]– Compatible with radome materials– Not an EPA-listed material -- Environmentally compliant– Lowers health risks to workforce– $0 implementation cost and Reduces operating costs by $30K– Reduces HAP emissions by 78,000 lbs– Eliminates abatement requirements [$2M]

52


Integrating with PESHE

Reports exported to PESHEReports exported to PESHE

Summary Report

Summary Report

Detailed Report

Detailed Report

Risk Matrix Report

Risk Matrix Report

Promotes updated risk review and decision making

environmental, safety, and occupational health (esoh) programmatic risk tool aeronautical systems...

Documents