Project Review Questions

Ops ReviewProject Facilitation Process Excellence

Project Charter (Define)

Questions• What is the expected business impact?• What is the business impact to date?• What is the problem and where is it

focused?• Has this been worked on before and

how are you leveraging that effort?• What are the boundaries of the

process?• Who is on the team and who is

sponsoring the effort?• How effective is the team and the

facilitator (Black Belt)?• What are the important lessons learned

(both good and bad)?• What actions were taken to correct for

any issues uncovered?• What barriers to success exist?

OPS REVIEW

Project Plan (Define)

Questions• What has been completed since the last

review?• What are the necessary deliverables

from the Measure Phase and have they been completed?

• Have Performance Reports been used to communicate with the rest of the business?

• Is a Visual Workplace established and current?

• How detailed is the plan going forward?

Monthly Report

Step Key ActionWho'sResp. Date Status

Estimated Impact Dent Rate Reduction

(BSL = 3.3%)

1 Improve Plate Cleaner Water Quality Bockstanz Feb-97Complete - Monitoring

water quality every load0.10%

2 Cover Exposed Pallet Boards with Tack ClothCho Smith and 1D

Layup TeamMay-97

Complete - all shifts following procedures. Need to put audit into

control plan

0.10%

3 Treater 6 and 7 edge seal improvements B. Griebler, P. HajjarMay-97

and Sep-97

Complete -Edge seals complete, better edge sealing reducing dust

0.50%

4 Layup Cleanroom Upgrade P. Smith, Bockstanz May-97Complete - Room

complete, Class 1000 or better near layup area

0.50%

5 B-stage Sealing with Hand Held Heat Guns C. Dunlap, Bockstanz Jun-97

Complete - Sealing cross cut and previously sealed

edges in the stack. Greatly reduces epoxy

dust

0.50%

6 Improved Cleanroom Practices Bockstanz Jun-97

Ongoing - Operator training, awareness.

Gowning, housekeeping and procedures improved

0.10%

7 New Release Sheets C. Dunlap, Bockstanz Jul-97Replaced Fluorglass with Chemfab - no more PTFE

delamination0.10%

8 Plate Cleaner Improvements Bockstanz Aug-97

Complete - reversed top brush to elliminate plates

from sticking. Also allowed more pressure to

be put on plates, thus cleaner plates.

0.30%

9 New Tack Wipe MachinesE. Shrewsbury,

BockstanzDec 97 and

????

First new machine due December. Next ????.

Will improve wipe efficiency.

0.50%

10 Central Vacuum in Prebuild D. Smith, Bockstanz Nov-97Used for Cleaning and vacuuming dust during

prebuild.0.10%

11 Install Tec Nec Rollers E. Seay, Bockstanz Jun-98

Will improve plate cleanliness along with

modifying plate conveyors to reduce black inclusion

dents

0.20%

Report Date: October 31 Project Baseline = 3.3%. Goal = 2.0% then 1.5% in Q1 1998. Current dent rate = 1.93%. Savings to date = $106,000. Annualized savings = $350,000.

Dent Rate

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar

Month

De

nt

Ra

te (

%)

Dent Rate Goal

ID Task Name

1 Project Kickoff

2 Process Mapping

3 C&E Matrix

4 Control plan review

5 MSA

6 Capability studies

7 FMEA

8 Multi-vari study

9 DOE 1

10 DOE 2

11 DOE 3

12 Control plan updates

13 SPC implementation

1/4 1/11 1/18 1/25 2/1 2/8 2/15 2/22 3/1 3/8 3/15 3/22 3/29 4/5 4/12 4/19 4/26 5/3 5/10 5/17 5/24January February March April May

OPS REVIEW

Voice of the Customer (Define)

• Did you use a cross-functional team to interview the customers?• How many customers did you interview? What market segments?• How were the customers selected?• Have you used the KJ process to sort and organize and rank their

inputs into the most significant customer desires, needs and expectations?

• Have you examined latent customer expectations?• Have you used QFD to translate significant customer desires, needs

and expectations into functional product requirements? • Have you incorporated the significant customer requirements into

rating criteria in the Pugh Concept selection matrix? • Have you used the Pugh Matrix Process to evaluate and select the

most promising design concept(s) that will meet or exceed customer expectations?

OPS REVIEW

Process Map (Measure)

Questions• Who helped develop the map and what

organizations do they represent?• Does it reflect current state or the

desired process?• Are all non value-added steps included?• What quick-hits did you find from this

effort?• What process steps does the team feel

can be eliminated or combined to reduce opportunities for scrap and increase rate?

• What characterizes an Uncontrolled and a Controlled variable?

• How do we measure the product for defects within the process?

PAPERWORK TURN STEAM ON TO DICY TANK

LOAD DMF LOAD DICY LOAD 2MI

BILL OF MATERIALS

ISO PROCEDURES

REWORK

SCALE ACCURACY

PREHEATING

LOAD ACCURACY

CLEANLINESS

RAW MATERIAL

LOAD ACCURACY

ENVIRONMENT (HUMIDITY)RAW MATERIAL

MIXER SPEED

LOAD ACCURACY

ENVIRONMENT (HUMIDITY)RAW MATERIAL

MIXER SPEED

GET PORTABLE CLEAN IF NECESSARY

LOAD SOLVENT TO PORTABLE

LOAD DRUMSTOCK TOPORTABLE

2

PROPER SIZE

REWORK

CLEANLINESS

CROSSCONTAMINATION

LOAD ACCURACY

ENVIRONMENT

RAW MATERIAL

LOAD ACCURACY

ENVIRONMENT

RAW MATERIAL

MIXER SPEED

LOAD BULK MATERIAL TOPORTABLE

2PUMP DICYSOLUTION TO PORTABLE

DIGEST STAGE BATCH TO TREATER

LOAD ACCURACY

ENVIRONMENT

RAW MATERIAL

MIXER SPEED

PUMP RATE

MIXER SPEED

FILTERING

TEMPERATURE

COMPLETE SOLUTION

TIME

MIX SPEED

VENTING

ENVIRONMENT

TIME

MIX SPEED

VENTING

ENVIRONMENT

COMPOUNDINGINPUTSTechniquesProceduresRaw MaterialsReworkEquipmentEnvironmentCleanlinessDicy Solution Temp

OUTPUTSReactivity (Gel)ViscosityCleanlinessColorHomogeneityConsistency Batch to BatchDigestion TimeTemperature

OPS REVIEW

Process Map (Measure)

Questions to Ask• Who was on your team?• How were the Customer Requirements identified?• What does the team feel are the most important Uncontrolled

KPIVs in the process?• Does this map reflect the current process or the one we want?• Where are the rework loops identified?• What are the non-value added processes?• Where are the measurement points in the process?• What does your Control Plan include at this time?• How many KPIVs did you identify per step (on average)?• What is the team going to do next?

OPS REVIEW

Cause and Effect Matrix (Measure)

Questions• Who provided inputs to the Customer

Requirements for this matrix?• What groups determined the

relationship ratings for the Inputs and Outputs?

• What method was used to determine the final relationship score?

• What actions are being taken on the top ranked Input Variables?

• Are there any quick hits that can be assigned to lower ranking Input Variables?

• Do the current Control Plans reflect the need to monitor these top KPIVs?

Rating of Importance to Customer

7 9 4 10 8 10

1 2 3 4 5 6

Pa

int

thic

kn

ess

Pa

int

ha

rdn

ess

Ra

te

Co

ve

rag

e

Su

rfa

ce

q

ua

lity

Pro

pe

r sc

he

me

Process Step Process InputTotal

1 Paint Nozzle type 9 0 9 9 9 9 3512 Paint Paint viscosity 9 9 9 9 3 3 3243 Paint Air pressure 9 3 9 9 9 3 3184 Prime Primer age 9 9 9 3 9 0 2825 Paint Paint age 9 9 9 3 9 0 2826 Paint Surface contamination 1 9 3 9 9 1 2727 Prime Air pressure 9 1 9 9 9 0 2708 Prime Nozzle type 9 0 3 9 9 0 2379 Prime Surface contamination 1 3 3 9 9 0 20810 Prime Relative humidity 1 9 9 1 9 0 20611 Paint Ambient temp 1 9 9 0 9 0 19612 Paint Surface roughness 3 1 3 3 9 3 17413 Paint Lot number 0 1 0 3 1 9 13714 Prime Surface roughness 0 1 3 3 9 0 12315 Prime Lot number 1 3 0 3 1 3 10216 Prime Ambient temp 1 3 9 0 3 0 94

Total 504

630

384

820

928

310

Lower Spec

Target

Upper Spec

Cause and Effect Matrix

This table provides the initial input to the FMEA. When each of the output variables (requirements) are not correct, that represents potential "EFFECTS". When each input variable is not correct, that represents "Failure Modes".

1. List the Key Process Output Variables2. Rate each variable on a 1-to-10 scale to importantance to the customer3. List Key Process Input Variables4. Rate each variables relationship to each output variable on a 1-to-10 scale

OPS REVIEW

Cause &Effect Matrix (Measure)

Questions to Ask• Who was on your team?• Who determined the Customer Requirements weighting?• Did you use the C&E Matrix to prioritize Process Steps to

address first?• How did the team determine the correlation values?• Does the prioritization make sense?• How many Uncontrolled Variables rated highly in the sorted

matrix?• How did the team determine the cut-off for variables to

investigate further?• Did your team include internal customer requirements?• What are your next steps?

OPS REVIEW

Measurement Systems Analysis (Measure)

Questions• What are the major sources of

measurement error?• How much measurement error

exists in comparison to the process variation? In comparison to the customer requirements?

• Is the measurement system acceptable for the process improvement efforts?

• If not, what actions do you suggest?

• How were the samples chosen?• Was the team sensitive to sub-

grouping for sample selection?

Misc:Tolerance:Reported by:Date of study:Gage name:

0

120011001000

900800700600500400300

321Xbar Chart by Operator

Sam

ple

Mea

n

X=725.73.0SL=797.1

-3.0SL=654.3

0

150

100

50

0

321R Chart by Operator

Sam

ple

Ran

ge

R=37.97

3.0SL=124.0

-3.0SL=0.000

10 9 8 7 6 5 4 3 2 1

11501050

950850750650550450350

Part ID

OperatorOperator*Part Interaction

Ave

rage

123

321

120011001000

900800700600500400

Oper ID

By Operator

10 9 8 7 6 5 4 3 2 1

120011001000

900800700600500400

Part ID

By Part

%Total Var%Study Var%Toler

Part-to-PartReprodRepeatGage R&R

100908070605040302010

0

Components of Variation

Per

cent

Gage R&R (ANOVA) for Output

OPS REVIEW

Gage R&R

Source VarComp StdDev 5.15*Sigma Total Gage R&R 1727 41.558 214.021 Repeatability 1407 37.507 193.162 Reproducibility 320 17.895 92.160 Operator 320 17.895 92.160Part-To-Part 24407 156.228 804.573Total Variation 26134 161.660 832.552

Source %Contribution %Study Var %Tolerance Total Gage R&R 6.61 25.71 10.70 Repeatability 5.38 23.20 9.66 Reproducibility 1.23 11.07 4.61 Operator 1.23 11.07 4.61Part-To-Part 93.39 96.64 40.23Total Variation 100.00 100.00 41.63

Number of Distinct Categories = 5

Measurement Systems Analysis (Measure)

Questions to Ask• Who performed the MSA?• What are the %P/T and %R&R values?• What was the main source of measurement error?• What actions are necessary for improvement, and what impact will

they have?• How were the samples selected for the MSA?• What is the measurement system acceptable for use in assessing -

Process Shift or Acceptance to Specification?• What variable(s) have you completed MSA on - KPOVs or KPIVs?• What effect will this measurement system have on Capability

assessments, Multi-vari studies, and DOEs?• How is the measurement capability going to be monitored into the

future?

OPS REVIEW

Baseline (Initial) Process Capability (Measure)

Questions• What variables were

evaluated?• Are they stable?• Are there explanations for

out of control points?• How long was the process

monitored to determine stability?

• What are the largest types of variation over time - shift to shift, week to week, etc.?

250200150100500

6

4

2

0

Individual and MR Chart

Obser.

Indi

vidu

al V

alue

11

111 1

X=1.611

3.0SL=3.349

-3.0SL=-0.1277

4.5

3.0

1.5

0.0

Mov

.Ran

ge

11

11

11

R=0.6537

3.0SL=2.136

-3.0SL=0.000

245235225

Last 25 Observations

2.5

2.0

1.5

1.0

Observation Number

Val

ues

40

3.34939-0.12766

Cp: 1.15 CPU: 1.37 CPL: 0.93 Cpk: 0.93

Capability PlotProcess Tolerance

Specifications

StDev: 0.579508

III

III

5.02.50.0

Normal Prob Plot

5.02.50.0

Capability Histogram

Process Capability Sixpack for sqrt_psf

0.00%

Fiscal WeekOPS REVIEW

Registration ScrapRegistration Scrap

1.00%

2.00%

3.00%

4.00%

5.00%

6.00%

Process Step/Input

Potential Failure Mode Potential Failure EffectsSEV

Potential CausesOCC

Current ControlsDET

RPN

Load DMF/DMF Load Accuracy Mischarge of DMF Viscosity out of spec 7 SOP not Followed 5

Operator Certification/ Process Audit

5 175

Steam to DICY/Scale Accuracy

Scale Not Zeroed Mischarge DMF 3 Faulty Scale 2 None 9 54

Load DMF/DMF Load Accuracy Mischarge of DMF Viscosity out of spec 7 Equipment Failure 2

Maintenance Procedure (SOP 5821)/Visual Check

3 42

Steam to DICY/Scale Accuracy

Scale > 0 Low DMF Charge 3 Water in Jacket 2 Visual Check of Jacket (SOP 5681) 4 24

Steam to DICY/Scale Accuracy

Scale Inaccurate High DMF Charge 3 Tank Hanging Up 2 Visual Check (SOP 5681) 4 24

Failure Modes Effect Analysis (Analyze)

Questions• Who helped develop the

FMEA and what organizations do they represent?

• Does the tool reflect current state or the desired process?

• What quick-hits did you find from this effort?

• What type of ranking system did you use ?

• Did you complete the actions recommended section of the tool?

OPS REVIEW

Failure Modes Effect Analysis (Analyze)

Questions to Ask• Who was on your team?• How were the Severity, Occurrence and Detection values

determined by the team?• What was the source of the rating scales for Severity,

Occurrence and Detection?• What are the top KPIVs?• What actions are being taken and by whom?• What barriers exist to completion of the top ranked items?• Are there any ‘quick fixes’ to be completed?• Were Safety items included?• How did the team decide what items to investigate further?• How did the actions affect the process performance?

OPS REVIEW

Design of Experiments (Improve)

• What significant factors and interactions exist?• How many full factorial designs were completed and why?• What methods have been utilized to optimize the process?• What actions have been taken to verify the results of the

experimentation?• How have the results been used to impact the design and

control plan?• What is the underlying model / equation that relates the

KPIVs to the KPOV(s)?• What next steps should be taken?

OPS REVIEW

Design of Experiments Proposal / Report (Improve)

Questions - Plan• Who developed and reviewed the DOE plan?

Who approved it?• What strategy was used in determining the

scope of the experiments?• How were experimental factors and levels

selected? Data?

Questions - Report/Results• What was concluded from the experiments?• What are the recommended changes based

on the studies?• What analysis techniques were used to

determine significant terms?• What percentage of the error is described by

the statistically significant terms?• How will the results be validated?

Experiment Planning Sheet

Date:_________________________ Product:______________________

Team Leader: _________________ Process(es): ___________________

Expected Start: ________________ Expected Completion:______________________________________________________________________________

Problem Statement:

Objective:

List of Experiment Parameters:

Response _________________________ Type: Quantitative QualitativeUnit of Measure ____________________Specification ______________________

Input Variables Levels Specifications

1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

Noise Variables Measurable?__________________________1. 2. 3. 4. 5.

Brief Outline of the Experimental Design to be used:

OPS REVIEW

-1 1

-1 1

1 1-1-1

8

7

6

5

Solvent

AirPress

Mea

n

Interaction Plot for Thicknes

Design of Experiments Statistical Analysis (Improve)

Questions• What proportion of the process

variation is accounted for by the model?• How large are they in comparison to

current levels?• What interactions, if any, are important?• How will you control interaction terms?• What changes are recommended based

on the results?• What is the predicted level of the

response based on the model from the DOE?

210

1.5

1.0

0.5

0.0

-0.5

-1.0

-1.5

Effect

Nor

mal

Sco

re

AC

A

Normal Probability Plot of the Effects(response is Thicknes, Alpha = .10)

A: AirPressB: ViscC: Solvent

Fractional Factorial Fit

Estimated Effects and Coefficients for Thicknes

Term Effect Coef StDev Coef T PConstant 6.4250 0.02500 257.00 0.002AirPress 2.3000 1.1500 0.02500 46.00 0.014Visc -0.5000 -0.2500 0.02500 -10.00 0.063Solvent 0.1500 0.0750 0.02500 3.00 0.205AirPress*Visc 0.1500 0.0750 0.02500 3.00 0.205AirPress*Solvent 1.0000 0.5000 0.02500 20.00 0.032Visc*Solvent -0.0000 -0.0000 0.02500 -0.00 1.000

Analysis of Variance for Thicknes

Source DF Seq SS Adj SS Adj MS F PMain Effects 3 11.1250 11.1250 3.70833 741.67 0.0272-Way Interactions 3 2.0450 2.0450 0.68167 136.33 0.063Residual Error 1 0.0050 0.0050 0.00500Total 7 13.1750

OPS REVIEW

Designed Experiments (Improve)

• Who was on your team and what were their roles?• How did you select the factors (KPIVs) to include?• How were the levels for each factor determined?• Did you complete a screening DOE?• How did you determine the sample size requirements for each

experimental run?• What risk levels did you assume when you designed your experiment?• What did you learn from your Pilot Run?• How much variation in the response (KPOV) was due to the factors

(KPIVs) you investigated?• What actions would you recommend based on the experimental

results?• How much of your experimental budget did you consume with each

experiment?

OPS REVIEW

Final Capability Summary (Control)

Questions• What are the long and short

term process capability values?

• How does this compare to the customer’s perspective of performance?

• Is there a significant opportunity to improve beyond current levels (i.e., how large is the gap between Cp and Ppk)?

• What are the definitions of defects and opportunities?

• Are the capabilities established for KPIVs and KPOVs from the C&E?

C u sto m er R e q u irem en t

(Ou tp u t V ariab le )

M easu rem en t T ech n iq u e

% R & R o r P /T R atio

U p p er S p ec L im it

T arg etL o w er S p ec L im it

C p C p kS am p le

S iz eD a te S tatu s

F ire R etard en cyU L 700 25% 3 1.5 n a

N o D ata Ava ilab le

S elvag e E d g e C o n sis ten cy

On -lin e P h ys ica l M easu rem en t

15% 4.5 4 3 .5 1 .15 0 .85 50 S ep -95Im p ro vem en t P lan in P la ce

M em b ran e S tab ility

Ov en T e st N o n e 0.75 0 .5 n a 1 .1 0 .65 25 Au g -95M easu rem en t

S tu d y S ch ed u led

150100500

14.5

13.0

11.5

10.0

Individual and MR Chart

Obser.

Indi

vidu

al V

alue

1

1111111

1 1

11

1

X=12.64

3.0SL=14.18

-3.0SL=11.10

4.5

3.0

1.5

0.0

Mov

.Ran

ge

1

1

1

1

1

1 1

1

R=0.5799

3.0SL=1.895

-3.0SL=0.000

155150145140135130

Last 25 Observations

13

12

11

10

Observation Number

Val

ues

16 8

14.184811.1004

Cp: 2.59 CPU: 2.18 CPL: 3.01 Cpk: 2.18

Capability PlotProcess Tolerance

Specifications

StDev: 0.514069

III

III

141210

Normal Prob Plot

141210

Capability Histogram

Process Capability Sixpack for CO2-Long

OPS REVIEW

Final Capability Summary (Control)

Questions to Ask• What variables have you conducted capability studies on?• What is the expected Short Term and Long Term performance of

that variable?• What are the suspected contributing factors to the variation in the

process?• How does the measurement system performance impact the

process capability?• What should the goals for the process capability (Cp / Cpk) be

based on the baseline data?• Over what time frame were the data taken?• How many data points / sub-groups were used to determine the

process capability?• Was the data normally distributed?• How do we handle non-normal data?

OPS REVIEW

Control Plan Summary (Control)

Questions• What control methods are

in place for KPIVs and KPOVs rated ‘high’ in the C&E matrix?

• Is SPC utilized for critical customer requirements?

• Are the measurement systems adequate for critical variables?

• Has training occurred for these control points?

Pa

int

thic

kne

ss

Pa

int

ha

rdn

ess

Ra

te

Co

vera

ge

Su

rfa

ce

qu

ali

ty

Pro

pe

r sc

he

me

Process Step Process InputTotal

1 Paint Nozzle type 9 0 9 9 9 9 3512 Paint Paint viscosity 9 9 9 9 3 3 3243 Paint Air pressure 9 3 9 9 9 3 3184 Prime Primer age 9 9 9 3 9 0 2825 Paint Paint age 9 9 9 3 9 0 2826 Paint Surface contamination 1 9 3 9 9 1 2727 Prime Air pressure 9 1 9 9 9 0 2708 Prime Nozzle type 9 0 3 9 9 0 2379 Prime Surface contamination 1 3 3 9 9 0 20810 Prime Relative humidity 1 9 9 1 9 0 20611 Paint Ambient temp 1 9 9 0 9 0 19612 Paint Surface roughness 3 1 3 3 9 3 17413 Paint Lot number 0 1 0 3 1 9 13714 Prime Surface roughness 0 1 3 3 9 0 12315 Prime Lot number 1 3 0 3 1 3 10216 Prime Ambient temp 1 3 9 0 3 0 94

Total 504

630

384

820

928

310

OPS REVIEW

Process Step Input OutputProcess Spec

(LSL, USL, Target)

Cpk / Date (Sample

Size)

Measurement System

%R&R or P/T

Current Control

Method (from FMEA)

Who Where When Reaction Plan

Current Control Plan