Measurement Systems AnalysisMeasurement Systems Analysis

Six Sigma FoundationsContinuous Improvement TrainingSix Sigma FoundationsContinuous Improvement Training

Six Sigma Simplicity

Key Learning PointsKey Learning Points

s Data Needs to be:s Verifiables Reliables Correct

s Stakeholders Believe Data

s Data Needs to be:s Verifiables Reliables Correct

s Stakeholders Believe Data

AGENDAAGENDA

s What is Measurement Systems Analysis?

s Reproducibility and Repeatability

s Summary

s What is Measurement Systems Analysis?

s Reproducibility and Repeatability

s Summary

What is measurement systems analysisWhat is measurement systems analysis

s Whatever it takes to ensure that measurements are reliable and credible

s Anticipate ‘doubting Thomas’ and share the pre-checks on your measurements

s Whatever it takes to ensure that measurements are reliable and credible

s Anticipate ‘doubting Thomas’ and share the pre-checks on your measurements

Measurement Systems Analysis– TermsMeasurement Systems Analysis– Termss Repeatability

Chance that same person (operator) and process will give consistent measurement

s Reproducibility Chance that different person (operator) will give

consistent measurement s Attribute

Qualitative e.g. Pass/Fail, Hot/Cold, Employee name, work order number

s Variable Quantitative Numbers, % Pass, Cost, Cycle Time

s Repeatability Chance that same person (operator) and process will

give consistent measurement s Reproducibility

Chance that different person (operator) will give consistent measurement

s Attribute Qualitative e.g. Pass/Fail, Hot/Cold, Employee name,

work order numbers Variable

Quantitative Numbers, % Pass, Cost, Cycle Time

MSA - Who should create itMSA - Who should create it

s Process owner/team guided by Black or Green Belt

s Quality groups May be part of Quality Management

System (QMS)

s Process owner/team guided by Black or Green Belt

s Quality groups May be part of Quality Management

System (QMS)

ExerciseExercise

12 months ago you agreed to the installation of a $1Million MRP software package with the aim of improving on time delivery

Now a sister plant is looking at introducing a different package to solve the same problem

The reason that they give for using a different package is because they say that your attempt to improve delivery failed

12 months ago you agreed to the installation of a $1Million MRP software package with the aim of improving on time delivery

Now a sister plant is looking at introducing a different package to solve the same problem

The reason that they give for using a different package is because they say that your attempt to improve delivery failed

They quote the data shown on the next graph.They quote the data shown on the next graph.

MRP system ‘improvement’MRP system ‘improvement’% On Time delivery

80

82

84

86

88

90

92

94

96

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Months

JIT System installed

Better or Worse?

MRP System An AnswerMRP System An Answer

s Comparing the data from before and after there is no relationship between the month of the year and the on time delivery

s Comparing the data from before and after there is no relationship between the month of the year and the on time delivery

2 4 6 8 10 12

86

91

96

Month

On

time

Del

ver

y

Graph to compare bef ore & af ter change

Bef ore

Af ter

s Before the MRP system there was more spread than afterwards

s After the system introduction the average on time delivery fell

s But difference is small 1.2/88 (delta/sigma) and we only had monthly figures!

s Because of our measurement system we would be unwise to assume that the observed sample average (before and after) is an accurate prediction of future on time delivery

s Before the MRP system there was more spread than afterwards

s After the system introduction the average on time delivery fell

s But difference is small 1.2/88 (delta/sigma) and we only had monthly figures!

s Because of our measurement system we would be unwise to assume that the observed sample average (before and after) is an accurate prediction of future on time delivery

AfterBefore

96

91

86

Dotplots of Before and After(means are indicated by lines)

Before After

86

91

96

Boxplots of Before and After(means are indicated by solid circles)

Sample RatesSample Rates

s It is unsafe to say whether the project worked or not because the project to introduce the system was poorly set up!!

s It is unsafe to say whether the project worked or not because the project to introduce the system was poorly set up!!

s In future the Green belt should check s The source of the

datas The speed and

size of sampling s The rate of

changes you introduce elsewhere which may have a domino effect

s The delta sigma of improvement you want to see

s In future the Green belt should check s The source of the

datas The speed and

size of sampling s The rate of

changes you introduce elsewhere which may have a domino effect

s The delta sigma of improvement you want to see

NoiseNoise

s Was on-time delivery the right thing to try and change?

s If ‘it depends’ on many factors such as:

order clauses, customers, capacity, changing volumes of orders, seasonal delivery demands and backlog of orders…this is called noise

s Was on-time delivery the right thing to try and change?

s If ‘it depends’ on many factors such as:

order clauses, customers, capacity, changing volumes of orders, seasonal delivery demands and backlog of orders…this is called noise

s In future, check …s That the metric is as

free of noise as possible

s That there is not a less noisy measure - such as % of orders delivered within 8 weeks, or RTY through a bottleneck (such as test)

s In future, check …s That the metric is as

free of noise as possible

s That there is not a less noisy measure - such as % of orders delivered within 8 weeks, or RTY through a bottleneck (such as test)

LinearityLinearity

Measurement System Linearity

0

20

40

60

80

100

120

0 50 100 150

Actual

Ob

serv

ed Linear

Step

Non Linear

s A measurement is linear if there is a straight line relationship between the observed or measured value and the actual value

s A measurement is linear if there is a straight line relationship between the observed or measured value and the actual value

Impact of LinearityImpact of Linearity

s Is 95% on time delivery only 5% better than 90%?

s If lead time is 50% do sales go up by 50%

s Is it necessary to train 100% of employees or just 50%

s Do all the engineers need a computer?

s Is 95% on time delivery only 5% better than 90%?

s If lead time is 50% do sales go up by 50%

s Is it necessary to train 100% of employees or just 50%

s Do all the engineers need a computer?

s In each case we will be more in control if we fully understand the relationship between

s Y and Xs and how the magnitude of X or Y effects our decisions

s In each case we will be more in control if we fully understand the relationship between

s Y and Xs and how the magnitude of X or Y effects our decisions

Repeatability and ReliabilityRepeatability and Reliability

s To determine the accuracy of our measurement system and the limits on the usefulness of the measurement device

s To determine the accuracy of our measurement system and the limits on the usefulness of the measurement device

The necessity of training farmhands for first-class farms in the fatherly handling of farm livestock is foremost in the eyes of farm owners. Since the forefathers of the farm owners trained the farmhands for first-class farms in the fatherly handling of farm livestock, the farm owners feel they should carry on with the family tradition of training farmhands of first class farmers in the fatherly handling of farm livestock because they believe it is the basis of good fundamental farm management.

s Task: You have 60 seconds to document the number of times the 6th letter of the alphabet appears in the following text.

s Task: You have 60 seconds to document the number of times the 6th letter of the alphabet appears in the following text.

Repeatability ExerciseRepeatability Exercise

Number: ____

s You receive a complaint from a salesman that the on time delivery of your plant is low. The last 20 units shipped to a customer were late.

s The plant quotes 100% on time. Every departmental manager is 100% on time according to their schedule

s How can this be?

s You receive a complaint from a salesman that the on time delivery of your plant is low. The last 20 units shipped to a customer were late.

s The plant quotes 100% on time. Every departmental manager is 100% on time according to their schedule

s How can this be?

Repeatability - DiscussionRepeatability - Discussion

R&R - Why do we need it R&R - Why do we need it

s To identify how much the chosen measuring system limits our ability to improve

s Because measurement systems are often significantly flawed

s To guarantee the truth of improvements made

s To identify how much the chosen measuring system limits our ability to improve

s Because measurement systems are often significantly flawed

s To guarantee the truth of improvements made

How to do R&RHow to do R&R

1. Select a minimum of 30 parts/documents from the process and at least 2 operators

2. Half should have defects some of which are marginally defective, half defect free

3. Each operator examines each piece at least twice in a random order (ensure understanding of type of data being collected - Variable or Attribute)

4. Plot measurement against operator, run, part and standard

5. If R&R are not acceptable adjust process and repeat

1. Select a minimum of 30 parts/documents from the process and at least 2 operators

2. Half should have defects some of which are marginally defective, half defect free

3. Each operator examines each piece at least twice in a random order (ensure understanding of type of data being collected - Variable or Attribute)

4. Plot measurement against operator, run, part and standard

5. If R&R are not acceptable adjust process and repeat

Attribute ExampleAttribute Example

Sample #Standard Run #1 Run #2 Run #1 Run #2 Run #1 Run #21 pass pass pass pass pass fail fail2 pass pass pass pass pass fail fail3 fail fail fail fail pass fail fail4 fail fail fail fail fail fail fail5 fail fail fail pass fail fail fail6 pass pass pass pass pass pass pass7 pass fail fail fail fail fail fail8 pass pass pass pass pass pass pass9 fail pass pass pass pass pass pass10 fail pass pass fail fail fail fail11 pass pass pass pass pass pass pass12 pass pass pass pass pass pass pass13 fail fail fail fail fail fail fail14 fail fail fail pass fail fail fail

Operator #1 Operator #2 Operator #3

Attribute AnalysisAttribute Analysis

s Agrees with himself 14/14, 11/14, 14/14

s Agrees with standard 26/28, 21/28, 22/28

s Overall agree with standard = 26+21+22/(3*28)

s Agrees with himself 14/14, 11/14, 14/14

s Agrees with standard 26/28, 21/28, 22/28

s Overall agree with standard = 26+21+22/(3*28)

Sample #Standard Run #1 Run #2 Run #1 Run #2 Run #1 Run #21 pass pass pass pass pass fail fail2 pass pass pass pass pass fail fail3 fail fail fail fail pass fail fail4 fail fail fail fail fail fail fail5 fail fail fail pass fail fail fail6 pass pass pass pass pass pass pass7 pass fail fail fail fail fail fail8 pass pass pass pass pass pass pass9 fail pass pass pass pass pass pass10 fail pass pass fail fail fail fail11 pass pass pass pass pass pass pass12 pass pass pass pass pass pass pass13 fail fail fail fail fail fail fail14 fail fail fail pass fail fail fail

Operator #1 Operator #2 Operator #3

Presenting ResultsPresenting Results

s Operator 2 was inconsistents Operator 2 was inconsistentNo operator was perfectNo operator was perfect

% Agrees with himself

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Operator 1 Operator 2 Operator 3

Operator agrees with standard

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Operator 1 Operator 2 Operator 3

Presenting ResultsPresenting Results

s Overall accuracy of measurement 82%s What is it about some parts that we can not

measure them correctly?

s Overall accuracy of measurement 82%s What is it about some parts that we can not

measure them correctly?

Correct Measurement by Sample

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Sample

Using computer softwareUsing computer software

Some of the initial conclusions that you can make are limited

If you want more precise information the team should seek advice from a specialist

Some of the initial conclusions that you can make are limited

If you want more precise information the team should seek advice from a specialist

Attribute R&R - Graphical outputAttribute R&R - Graphical output

Buffalo, NY Plant

1.5 mmSix Sigma BB01/01/1998

Gage #020371

Misc:

Tolerance:Reported by:Date of study:

Gage name:

10 9 8 7 6 5 4 3 2 1

1.1

1.0

0.9

0.8

0.7

0.6

0.5

0.4

Part ID

OperatorOperator*Part Interaction

Ave

rage

1

23

Gage R&R (ANOVA) for Measure

321

1110090807060504

Oper ID

By OperatorBu falo, NY Plant15 mmSx Sgma BB0101/1998Gage #020371

Msc:Tole ance:Reportedby:Dae of udy:Gagename:

Gage R & R (ANOVA) for Measure10 9 8 7 6 5 4 3 2 1

1110090807060504

Part ID

By PartBuffalo, NY Plant15 mmSix Sigma BB01/01/1998Gage #020371

Misc:Tolerance:Reported by:Date of study:Gage name:

Gage R & R (ANOVA) for Measure

Variable R&R - Graphical AnalysisVariable R&R - Graphical Analysis

In this example 3 operators measured twice the diameter of 10 turned components. What would the ideal graph look like for each?

In this example 3 operators measured twice the diameter of 10 turned components. What would the ideal graph look like for each?

R&R Class ExerciseR&R Class Exercise

s Now its your turn

s Grab a bag of m&m’s

s Now its your turn

s Grab a bag of m&m’s

Measurement Systems AnalysisMeasurement Systems Analysis

Six Sigma FoundationsContinuous Improvement TrainingSix Sigma FoundationsContinuous Improvement Training