observation process may 2005 revision the key to effective review - the performance element of the...

Observation ProcessMay 2005 Revision

The Key to Effective Review -

The Performance Element of the Performance-Based Review

Process!

10T10L

Upon successful completion of this topic, you should be able to:

Describe how to prepare to observe an ORR/RA evolution.

Explain the principle of observation vice participation (Heisenberg Principle).

Describe requirements based observation techniques.

List several types of ORR/RA evolutions.

Explain the need for a facility to run a drill vice only respond to a drill.

10T10L May 2005 Revision 2


Evolution Purpose

• To perform, to the maximum extent possible, a representative sample of anticipated operations and evolutions, and responses to credible casualties and “off-normal events” in order to demonstrate the facility operational readiness . . .


Types of EvolutionsSome Samples

• Exercise

• Drills

• Operations

• Maintenance

• Surveillances

• Tours

• System startup

• Briefings

• Training session

• Critique

• Meetings

• Turnovers

• Walk downs

• Alarm response


Steps for Conducting an Observation

• Preparation . . .

• Actual observation . . .

• Follow-up . . .

• Reporting . . .


Step 1 - Preparation

• Single most important step in the process:– Serves as a foundation;

– Team must select representative operations;

– Surrogate material use . . .

Steps for preparation:– Select the activity to be observed;

– Determine appropriate standards of performance;

– Become familiar with standards/knowledge required;

– Integrate appropriate upset conditions.


Selecting the Activity• Important to plant, personnel, procedures, management

system:– Plant-people-procedure-process relationship;– Surveillance versus routine preventive maintenance;– Critical process versus routine operation;– Minimize simulations!

Known or suspected problem area:– ORPS review;– CORR or MSA results;

• Corrective actions to rectify previous problems . . .


Selecting the Activity (continued)

• Existing facility condition:

– Material condition of equipment;

– Housekeeping practices;

– Labeling;

– Industrial safety conditions;

– Surrogate material available and use . . .



• Targets of opportunity:– Drills / Dry-runs;

– System operability checks;

– Maintenance work in progress;

• Teamwork:– Keep other members informed of your observation;

– Deploy Team to Achieve Complete Coverage!



• Review of off-normal condition preparedness:– process upset conditions;

– emergency conditions;

– Facility integrates upsets into selected evolutions . . .

• Practical considerations:– Time availability;

– Accessibility of activity.


Preparation

• Be a prepared and knowledgeable observer:– Obtain and study the procedure;

– Understand what actions you should see and where;

– Mark-up the procedure in advance to enhance notation;

– Walk down the area, understand the systems;

– Understand ancillary processes . . . log keeping, pre-job briefing processes, authorization/control mechanisms;

– Have clear understanding as to who is to watch what;

– Prepare for copious notes including time of happening.


Rules for Observations

• Do not act as a Safety Observer!• Do not violate rules!• Do not interfere with operations!• Do not impose conditions, simulations, or

anomalies!• Be transparent . . .• Request Facility to communicate roles and rules!


Step 2 – Actual Observation• Interaction with observed personnel:

– Establish rapport;

– Explain purpose of observation;

– Outline your role as an observer:• Questioning techniques;• Will not interfere with work;• Be Transparent.


Step 2 – Actual Observation (continued)

• Take thorough notes in real time:– Times actions occurred;

– Procedure details- have procedure in hand;

– Nomenclature and physical details;

– Be quantitative, record pertinent data;

– Include questions and follow-up items;

– Follow procedure as conducted.


Step 2 – Actual Observation (continued)

• Clearly understand what actually happened:– Don't assume . . .

– Ask clarifying questions without interfering;

– Seek additional details as needed;

– Full understanding is imperative to accuracy of the observations;

– Coordinate with other Team Members who observed.


Observe beyond the Activity

• Start at the true beginning:– Observe Pre-job briefings;

– Observe system alignment checks;

– Observe daily checks;

• Note the peripheral activity:– People;

– Environment;

– Preparation/set-up activities.

• Become creatively inquisitive . . .

• Keep attention to important items . . .


Heisenberg Uncertainty Principle

• Difficult to make an accurate measurement without affecting what is being measured . . .

• Difficult to measure people performance without influencing their performance . . .

• Do not ask prompting type questions that could change performance . . .


Problems

• Often, if not always, symptomatic of problems elsewhere:– keep Team Leadership informed of problems that have broad

impact (training, procedures, supervision, safety culture, etc.)!

• Isolated problems not likely . . .

• Often recurring:– those where management has taken corrective action . . .


Step 3 – Follow-up

• Resolve open questions . . .

• Primary purpose of follow-up:– What you observed are indicators . . .

– Follow-up will define real problem:• need for improvement;

• underlying causes.


Primary Follow-up Questions

• What is the problem?

• How big is the problem?

• Where else might this problem exist?

• Other Contributors ? – Program Problems?– Training Problems?


Step 4 – Recording the Observation

• Perspective . . . – Clearly understood?

– What does it really mean?

• Record immediately after observation:– (Observer notes / Form 1 . . . ).

• Many indications/deficiencies lead to a finding – identified in Form 1 and summarized on a Form 2 . . .


Step 4 -- Recording the Observation

• Avoid speculation . . .

• Stick to specifics when describing observed strengths and weaknesses . . .


Step 4 (continued)

• Avoid superlatives such as:– "is the worst I've ever seen” . . .

– "is the ________ ever seen” . . .

• Stay focused on whether the objective is being met as defined by the criteria . . .


Other Lessons Learned for Conducting Observations

• The ORR Team observes the drill process: – The Team does not run the drill;

– DO NOT act as Safety Observers or Drill Initiators!

• Observe pre/post-job/setup activities . . .• Have an “observation contingency plan” ready . . . • Use more than one observer for most activities . . .• “Off-normal events” are not “emergency drills” . . .


Observations to Findings

• Observations provide the basis for:– Findings;– Observations;– Correlation with Interviews/Document Reviews.

• Combine several observations if appropriate to draw conclusions concerning program status:– Rarely should a single observations result in a finding;– Adequacy is the standard, perfection is the goal.

observation process may 2005 revision the key to effective review - the performance element of the...

Documents

activity continuedreview

observation techniques

observation processmay

actual observation

normal condition preparedness

performance element

selected evolutions

safety observer