planning for doe, lesson 1
DESCRIPTION
Statistics - Design of ExperimentsTRANSCRIPT
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 1/58
Engineering Statistics Training
Introduction to Design of Experiments
Planning and Executing
Dave Jackson, M.S.
EW Engineering Statistics
Introduction to Design of Experiments, Planning and Execution, July ‘05 1
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 2/58
Contents
Introduction to DOE -------------------------------------------------------------- 4
Prerequisites ------------------------------------------------------------------- 5Training objective ------------------------------------------------------------- 6What is DOE? ------------------------------------------------------------------ 7
Applications of DOE ---------------------------------------------------------- 8DOE overview ----------------------------------------------------------------- 9Why make an effort to learn DOE?---------------------------------------10DOE vocabulary --------------------------------------------------------------13
2 and 3 level designs -------------------------------------------------------14The empirical model ---------------------------------------------------------16The steps for completing a DOE -----------------------------------------18The sequential nature of DOE --------------------------------------------19
Planning for DOE ----------------------------------------------------------------20
• DOE planning prior to selecting a design --------------------------21
• Define the process you want to study -------------------------------22• Define the experiment objectives ------------------------------------23
• Identify responses -------------------------------------------------------24
• The sample mean --------------------------------------------------------25
• The sample standard deviation ---------------------------------------36
• The %CV -------------------------------------------------------------------38
• Pass/fail responses ------------------------------------------------------39
• Sample size ---------------------------------------------------------------30
• Response calculation worksheet (example) -----------------------33
• Measuring the response ------------------------------------------------34
Introduction to Design of Experiments, Planning and Execution, July ‘05 2
• Response worksheet ----------------------------------------------------40
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 3/58
Contents
Planning for DOE ------------------------------------------------------ continued
• Process variables worksheet (example) ----------------------------42
• Process variables (noise) ----------------------------------------------45
• The importance of wide factor level ranges -----------------------46
• Signal-to-noise review --------------------------------------------------48
• Factor that are a nuisance to vary -----------------------------------49
• Additional notes on factors --------------------------------------------50
Executing a DOE ----------------------------------------------------------------51
• Checklist for executing a DOE ----------------------------------------52
Final Thoughts --------------------------------------------------------------------53
Appendix A ------------------------------------------------------------------------54
References ------------------------------------------------------------------------55
Introduction to Design of Experiments, Planning and Execution, July ‘05 3
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 4/58
Introduction to DOE
Introduction to Design of Experiments, Planning and Execution, July ‘05 4
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 5/58
Prerequisites
• An interest in using Design of Experiments (DOE)
• An expectation for using DOE in the near future
• The ability to navigate in a Windows environment
• A working knowledge of basic statistical concepts. The
training will reinforce and expand on these concepts usingpractical examples.
Introduction to Design of Experiments, Planning and Execution, July ‘05 5
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 6/58
Training Objective
The objective of this course is to increase the likelihood of successful
experiment outcomes at Edwards.
The course will cover DOE planning, design selection, proper
execution, and analysis of DOE data using the Statgraphics software
package.
A successful experiment,
• Yields useful information that moves the project forward
• Does not have to be repeated
Introduction to Design of Experiments, Planning and Execution, July ‘05 6
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 7/58
What is DOE?
DOE is a procedure for learning how process variables actingsimultaneously affect, and interact to affect process performance.
Introduction to Design of Experiments, Planning and Execution, July ‘05 7
Estimated Response SurfaceMixer Speed=25.0
TemperatureTime
A f T e m p
e r a t u r e
556 558 560 562 564 240250260270280290300
-11
-10
-9-8
-7
-6
-5
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 8/58
Applications of DOE
• Determine the process variable settings that simultaneouslyoptimize several performance characteristics
• Screen out the important process variables from a large set ofcandidate variables.
• Reduce development time for new processes
• Minimize cost by finding process variable settings that reduce thecost of running a process
• Reduce variation in a process
Introduction to Design of Experiments, Planning and Execution, July ‘05 8
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 9/58
Why Make the Effort to Learn DOE?
DOE is a vast improvement over one-factor-at-a-time experimentation
and/or a series of trial-and-error efforts:
• More powerful – DOE gets the right answer more often
• More efficient – DOE gets the answer in fewer runs, with
fewer test articles and overall shortens
development time
• More actionable – DOE results are easier to interpret
Introduction to Design of Experiments, Planning and Execution, July ‘05 9
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 10/58
Why Make the Effort to Learn DOE? cont.
Statistically Designed Experiments Can Detect and Describe
Interactions Between Input variables
Two variables interact if changing the level of one variable changes
the relationship between a second variable and the response. Time
and Temperature interact in the process of baking cookies.
Introduction to Design of Experiments, Planning and Execution, July ‘05 10
highlow
Flavor
Oven Temperature
Oven Time = short
Oven Time = long
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 11/58
Introduction to Design of Experiments, Planning and Execution, July ‘05 11
Why Make the Effort to Learn DOE? cont.
Statistically Designed Experiments Can Deal With Curvature in the
Response.
Flavor
low
Amount of Salt in Food
medium high
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 12/58
ction to Design of Experiments, Planning and Execution, July ‘05 12
DOE Overview
A statistically designed experiment consist of a series of runs, or trials, in whicchanges are made simultaneously to the input variables of a process, and (2) pobserved.
Introdu
TissueThickness
FlowRate Power Pressure Duration
Ave.LesionWidth
SDLesionWidth
9 1 8 225 6012 6 8 225 60
9 1 8 75 90
12 1 8 225 90
12 1 12 225 60
9 6 8 75 60
12 6 12 225 90
9 6 12 75 90
10.5 3.5 10 150 75
12 1 8 75 6012 6 8 75 90
12 1 12 75 90
9 6 8 225 90
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 13/58
DOE Vocabulary
• There are typically many process variables that affect aresponse. Those process variables whose levels you changeper your experiment design are called factors.
• Responses are the process performance characteristics thatyou are interested in studying.
• An experiment design consists of the entire set of runs in theDOE.
• Runs are different combinations of factor levels. Identical back-to-back runs require a complete reset of the process betweenruns.
• Center points are runs located in the center of the experimentspace.
• The experiment space is the multidimensional envelopeextending to the low and high levels of the factors.
Introduction to Design of Experiments, Planning and Execution, July ‘05 13
Factor B
Factor A
Factor C
Center points
Experiment space
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 14/58
2 and 3-level Designs
Two Level Designs:
• Factors are run at two levels (low and high) in the basedesign.
• Even when center points are added, the design is stillconsidered two-level.
• Used to model responses with, at most, a small amount ofcurvature.
• Screening experiments1
• Range finding
Three Level Designs:
• Factors are run at three or more levels in the base design
• Center points are often an integral part of the base design
• Used to model significant curvature in the response
Introduction to Design of Experiments, Planning and Execution, July ‘05 14
1 A two-level experiment is called a “screening experiment” if its primary purposeis to screening-out the most important factors from a large group of candidatefactors. In this scenario, we are interested in identifying significant main effects,rather than interaction effects or curvature effects; the latter assumed to be anorder of magnitude less important.
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 15/58
Which Response Can You Model With a 2-Level Design?
Response Surface A
X
Y
R e s p o n s e
00.2
0.40.6
0.81
0
80
Y
Response Surface B
XY
R e s p o n s e
-4-2
-100
Response Surface C
X
Y
R e s p o n s e
-2
-30
-12
(X 1000)
Introduction to Design of Experiments, Planning and Execution, July ‘05 15
Answer: A only. B and C require a design with at least threelevels.
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 16/58
The Empirical Model
The software builds a model which is a polynomial approximation forthe true and unknown relationship between each response and the
factors.
Here is the complete model (up to 3rd order) for a three-level
experiment involving three factors, x1, x2, and x3.
0 1 1 2 2 3 3
12 1 2 13 1 3 23 2 3 123 1 2 3
2 2 211 1 22 2 33 3
2 2 2 2 2112 1 2 113 1 3 122 1 2 133 1 3 223 2 3 233 2 3
3 3 3111 1 222 2 333 3
ˆ y b b x b x b x
b x x b x x b x x b x x x
b x b x b x
b x x b x x b x x b x x b x x b x x
b x b x b x ε +
= + + +
+ + + +
+ + +
+ + + + + +
+ + + +
2
Model Term Definition
ˆ y The estimated average response overthe experiment space.
0b Constant.
1 1 2 2 3 3, ,b x b x b x Main effects representing the linearpart of the model.
Introduction to Design of Experiments, Planning and Execution, July ‘05 16
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 17/58
The Empirical Model cont.
Model Term Definition
12 1 2 13 1 3 23 2 3, ,b x x b x x b x x
The two-factor interactionsrepresenting mild curvature in themodel.
1 2 3 1 2 3b x x x
The three-factor interaction. Usuallyconsidered small or not important formany processes.
2 211 1 22 2 33 3
, ,b x b x b x2 Quadratic effects representingsignificant curvature in the model.
2112 1 2b x x , , ,2
113 1 3b x x 2122 1 2b x x
2133 1 3
b x x , ,2
223 2 3b x x 2
233 2 3b x x
Partial cubic effects. Usuallyconsidered small or not important formany processes.
3111 1b x , ,3222 2b x 3333 3b x
Cubic effects. Usually consideredsmall or not important for manyprocesses.
ε
The error term representing commoncause variation not accounted for bythe other terms. Assumed to have anormal distribution with mean zero and
variance, 2σ .
Note that you can select designs to estimate three-factor interactions,partial cubic, and cubic effects, but they are seldom important andtherefore usually not worth the extra expense to estimate.
Introduction to Design of Experiments, Planning and Execution, July ‘05 17
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 18/58
Steps for Completing a DOE
The font size indicates the relative importance for having asuccessful DOE outcome.
1. Planning
2. Select Experiment Design
3. Execute Experiment & Collect Data
4. Analyze Data
5. Perform best-setup confirmation run, or validation
Introduction to Design of Experiments, Planning and Execution, July ‘05 18
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 19/58
The sequential Nature of DOE
DOE is iterative; rarely does one run a large, comprehensive design
in which a final conclusion is made.
Introduction to Design of Experiments, Planning and Execution, July ‘05 19
Factor A
Direction of better performance
First DOE
Second DOE
Factor B
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 20/58
Planning for DOE
Introduction to Design of Experiments, Planning and Execution, July ‘05 20
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 21/58
DOE Planning Prior to Selecting an Experiment Design
Introduction to Design of Experiments, Planning and Execution, July ‘05 21
Define the process you want to study
Define experiment objectives
Identify responses and measurement methods (IQ/OQ)
Identify process variables and measurement methods (IQ/OQ)
Start
Are interactions between factors likely?
Is curvature in the response likely?
Will any factor be a nuisance to vary?
End
Make sure process is stable (IQ/OQ)
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 22/58
Define the Process You Want To Study
• A process is a sequence of tasks required to accomplishsomething, or
• A process is a means of transforming inputs into outputs
IPO process model
Introduction to Design of Experiments, Planning and Execution, July ‘05 22
Product Design Influences
Product Performance
Variation in Product
PerformanceEquipment & Tooling
Procedures & Methods
Operator Influences
Material Properties
Environmental Influences
Process
Controllable process variables
(Inputs)
Responses
(Outputs)
Uncontrollable process variables(Noise)
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 23/58
Define the Experiment Objectives
• Clear statement of what you want to accomplish
• Prioritize objectives
• State constraints
• Get team agreement
Poorly Stated Objective
Maximize seal strength.
Better Objective
Learn the process variable settings that maximize seal strengthsubject to the following constraints:
(1) Sterile barrier pouches must pass the SOP leak test;
(2) The visual appearance of the seals must pass SOPrequirements; and
(3) Throughput must be greater than or equal to 20 pouches perminute.
Introduction to Design of Experiments, Planning and Execution, July ‘05 23
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 24/58
Identify Responses
• Select responses that are good predictors of field performance.
• Use the mean of several observations from a run as the officialresponse value for a run.
• Use the standard deviation of several observations from a runas an additional response representing the variation in theprocess.
• Measure responses as continuous variables instead ofcategorical variables, whenever possible.
• If unsure about two measurement methods, use both.
• Omitting a key response variable can be a big mistake.
• Choosing responses and their measurement methods aretypically the most time-consuming aspects of DOE planning.
Introduction to Design of Experiments, Planning and Execution, July ‘05 24
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 25/58
The Sample Mean
If n observations are denoted by , their mean is:1 2 ,, , n x x xL
1 2
1( )n x x x
n x + + += L .
Or, in more compact notation:
1 i x xn= ∑
Suppose that these response data were taken during a particular run:
545935
4146
The mean response for the run is
1(54 59 35 41 46
5
47
1i x x
n
= + + + +
=
= ∑
Introduction to Design of Experiments, Planning and Execution, July ‘05 25
)
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 26/58
The Sample Standard Deviation
An experiment objective might be to minimize the standard deviation,
or at least make sure that there is no obvious indication that the
standard deviation appears noticeably worse within the
manufacturing window you end up selecting.
If n observations are denoted by , their variance is:1 2
,, , n x x xL
2 2 21 2
1( ) ( ) ( )
1 ns x x x x xn
⎡ ⎤⎣ ⎦
= − + − + + −−
L 2 x .
Or more compactly:
2 21( )
1 ii
s xn
= −− ∑
x
The standard deviation, s, is the square root of the variance
2s s=
Introduction to Design of Experiments, Planning and Execution, July ‘05 26
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 27/58
The Sample Standard Deviation cont.
Suppose these response data were taken during a particular run:
5459354146
The sample standard deviation response is
2
2 2
2 2 2 2
1( )
1
1(54 47) (59 47) (35 47) (41 47) (46 47)
1
93.51
9.67
i
s
s x xn
n
s
⎡ ⎣
= −−
= − + − + − + − + −−
=
=
=
∑
∑ 2
Introduction to Design of Experiments, Planning and Execution, July ‘05 27
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 28/58
The %CV
Like the standard deviation, the coefficient of variation (%CV) may be
used as a response representing variation in performance. An
experiment objective might be to minimize the %CV.
The %CV of n observations1 2
,, , n x x xL is
% sample standard deviation
100 100sample mean
CV s
x= ⋅ ⋅=
For many processes, the variation increases as the mean increases.In this situation, the %CV is a better response than the standarddeviation. See Appendix B for more information.
Introduction to Design of Experiments, Planning and Execution, July ‘05 28
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 29/58
Pass/Fail Responses
Consider as a last resort because the sample size requirements foreach run can reach into the hundreds
The percent defective for DOE run i , is
number defective in runˆ 100
total number tested in runi p = ⋅
Strategies:
• Substitute a continuous response whenever possible
• If a Pass/Fail test must be used, redefine it so more test articlesfail
o Change a 10 psig test to a 30 psig test
• If a Pass/Fail test must be used, rate each test article as to theextent of failure
o 0=no failure, 1=25% failure, 2=50%, 3=75%, 4=100%failure
Introduction to Design of Experiments, Planning and Execution, July ‘05 29
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 30/58
More Than One Sample Per Run Usually Required
The software assumes that the response value you provide for aparticular run is close to the true (long-run) response for that run.
An approach for getting near the true response value for the run isto use the mean of multiple samples processed during the run.
Response (units)
d e n s i t y
0 3 6 9 12 15
0
0.1
0.2
0.3
0.4
0.5
Run #1 Run #2
Long-run Performance
Suppose you make one observation per run. . .
You might get lucky and get these values; then the software canaccurately estimate the difference between the runs.
You might get these values; then the software will over estimatethe difference between the runs.
You might get these values; then the software will underestimatethe difference between the runs.
Introduction to Design of Experiments, Planning and Execution, July ‘05 30
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 31/58
What Sample Size (# Observations per Run) Do I Need?
Use available feasibility data taken under a single set of process
conditions, and plot the data as shown below.
After about 12 test articles (for this example only) the mean remains
roughly unchanged.
n
m e a n s o f t
h e f i r s t n t e s t a r t i c l e s
2 3 4 5 6 7 8 9 1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
-2
-1
0
1
2
3
4
Note: all 20 test articles were from the same run
Introduction to Design of Experiments, Planning and Execution, July ‘05 31
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 32/58
More on Sample Size and Noise
• The greater the variation (noise) in the response, the larger the
sample size needed to get good estimates for the response
values for the runs.
• To reduce noise, consider reporting the official measurement of
a test article as the mean of several measurements made on
that test article
o Use when there is noticeable within-test article variation
o Use when there is noticeable measurement variation
• The process must be stable during the execution of a DOE;
otherwise, process drift or jumps will likely obscure the real
effects of the factors
o Stability means in a state of statistical control. See
references on statistical process control (SPC).
Introduction to Design of Experiments, Planning and Execution, July ‘05 32
• Rule-of-thumb: it usually takes about twice as many samples
(observations) to get as good an estimate of the standard
deviation as it does the mean; however, most experimenters
will go with the sample size required for the mean and glean
whatever information they can about the standard deviation
response. This approach works pretty well unless your primary
experiment objective is to study and reduce process variation.
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 33/58
Response Calculation Worksheet (an example)
In this example, there were 10 samples used in Run #1 and 5measurements per sample. The “Run Mean” and “Run SD” werethe response values for Run #1 that were entered intoStatGraphics
Introduction to Design of Experiments, Planning and Execution, July ‘05 33
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 34/58
Measuring the Response
• Measurement processes include:
o instruments (gages)
o procedures
o fixtures
o software
o people
o environment
o assumptions
• Basic requirements for measurement processes:
o good discrimination
o adequate range
o low bias
o good repeatability
Introduction to Design of Experiments, Planning and Execution, July ‘05 34
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 35/58
Measurement Discrimination
• Discrimination is the smallest change response that the
measurement process can faithfully detect and indicate
• Discrimination should be no larger than about 1/10 of the
smallest difference in the response you require the experiment to
detect
Response (units)
d e n s i t y
0 3 6 9 12 15
0
0.1
0.2
0.3
0.4
0.5
Run #1 Run #2
Difference in Response
Long-run Performance
Introduction to Design of Experiments, Planning and Execution, July ‘05 35
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 36/58
Measurement Discrimination cont.
• Alternatively, the discrimination should divide the process
variation into 10 parts or more (rule of tens). The process
variation is the long run, ±3σ spread in the response observed
over a single set of process conditions. The ±3σ spread is also
called the natural tolerance of the process.
Introduction to Design of Experiments, Planning and Execution, July ‘05 36
response measured in standard deviations
d e n s i t y
-4 -3 -2 -1 0 1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
99.7% of observations
95% of observations
68% of observations
Normal Distribution Model
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 37/58
Measurement Discrimination cont.
The process standard deviation, , may be estimated as follows:
4
ˆ s
cσ = ,
where s is the center line from a standard deviation control chart,
and is a constant found in SPC references.4c
Alternatively,
ˆ sσ = ,
Introduction to Design of Experiments, Planning and Execution, July ‘05 37
where is the sample standard deviation of n observations made over
a single set of process conditions.
s
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 38/58
Measurement Range & Bias
• Range refers to the smallest and largest values that a
measurement process can faithfully detect and indicate. Select
a gage will adequate range.
• Bias2 is the difference between the mean of multiple
measurements on the same reference standard and the
accepted value of the reference standard. Bias is a smaller-
the-better characteristic. Bias is usually handled through
calibration.
Introduction to Design of Experiments, Planning and Execution, July ‘05 38
2 Bias is sometimes referred to as “inaccuracy”
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 39/58
Introduction to Design of Experiments, Planning and Execution, July ‘05 39
Measurement Repeatability
Repeatability is that portion of the observed process variation
attributable to the measurement process alone (assumes the same
operator makes all of the measurements).
Repeatability may be thought of as the sample standard deviation of
n measurements made at the same location on the same test article,
by the same person3.
Guidelines:
• Repeatability must be small compared to the smallestdifference in the response that you want the experiment to beable to detect
• Alternatively, repeatability must be small compared to the ±3σ process variation
• Where appropriate, report the official measurement for a testarticle as the mean of two or more observations made on thattest article
3 This working definition may be used in the early stages of the DOE effort, if theresults of a formal gage capability study are not be available
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 40/58
ction to Design of Experiments, Planning and Execution, July ‘05 40
Response Worksheet
Response
Procedure &
Spec
Estimated
Range ofData
/
Ga
Di
Low:
High:
Low:
High:
Low:
High:
Low:
High:
= Smallest change in the response that is worth investigating= Standard deviation of the experimental error (standard deviation of the means
Introdu
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 41/58
Completing the Response Worksheet
• For each line item, think about how you can reduce the overall
noise in the experiment.
• / expresses the smallest change in the response that is
worth investigating, as a multiple of the background noise.
• You use / in the StatGraphics Power Curve procedure to
calculate the number of extra replicate runs you need to add to
the base design.
• If a transformation is to be performed on a response variable,
then and need to be in transformed units.
Estimate (sigma) as follows:
1. Select one set of run conditions
2. Set up, stabilize and run the process in the same manner as youwill for the DOE
3. Collect n observations, and calculate their mean
4. Completely reset the process
5. Repeat steps 2 through 4 two or more times.
Introduction to Design of Experiments, Planning and Execution, July ‘05 41
6. Calculate the standard deviation of the three or more means.
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 42/58
Process Variables Worksheet (example)
Constant,
Noise, orFactor
Process Variable DOE Value
Operator Influences
C Process operator Single operator, Peng
C Af measurement operator Single operator, Peng
Material PropertiesC BioPhysio valve Model 3100
C BioPhysio 2-D flat pattern 193396-029 Rev E, Lot # 0989MG 3011/248-10
C Nitinol flat sheet MetalTex, p/n 193402, Lot #248-10
C Salt Nitrate Salt, p/n 123456001
CDoes salt go bad over time or over# parts processed? No
C Quench water Tap water. Room temperature (20°C ± 5°C),
Equipment and Tooling
C Shape-setting mandrel, 29 mm p/n 392664
C Salt bath Ajax Hultgren Salt Bath Furnace, model # 3100.
C Mandrel rack, 5 up p/n 392827
C Mixer Accumix variable speed controller model MM23401C. ¼Hp motor. Propeller location and configuration unknown.
C Salt bath temp controller
For gross adjustment only; record setting for each DOErun, but use the chart recorder to setup the DOE runs.
Met ID 116229.
CChart recorder & Temperatureprobe Met ID 117406
Procedures and Methods
C Shape setting SOP 2394 red-lined, attached
C Bath stabilization time >= 90 minutes
CTest article position on mandrelrack
Position the test article on the mandrel location marked.Do not load frames on the remaining 4 mandrels(assumption: the missing frames will not invalidate theDOE results with respect to how the process will behavein production).
C Number of mandrels on rack 5 mandrels
C Orientation of rack in bath Test article at 6:00 with respect to operator.
C
Depth of salt in bath
Start DOE: 21.5 inches
Introduction to Design of Experiments, Planning and Execution, July ‘05 42
Minimum height is 21 – 22 inches from bottom of bath.
C Depth of valve frames in bathFixed by the geometry of the mandrel rack and by thedepth of the salt.
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 43/58
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 44/58
Completing the Process Variables Worksheet
• For each line item, think about how you can reduce the overallnoise in the experiment.
• Serves as good documentation.
• Include all variables, even those believed only remotelyinfluential.
• Keep in mind the six process variables categories.
• Brainstorm as the team watches the process operation or theproduct build.
• Brainstorm as the team reviews applicable SOPs, processsheets, equipment manuals, and the like.
• Consider measurement discrimination, range, and bias as youdo for responses.
• Consider completing an input variables worksheet for eachresponse measurement process.
Introduction to Design of Experiments, Planning and Execution, July ‘05 44
• Failure to include an important factor in your DOE will likelymake the results unusable, especially if the factor is involved inan interaction.
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 45/58
Process Variables of the Noise Variety
• Noise variables are not closely controlled because, (1) they
are thought to have at most a small effect on performance,
and (2) they are considered difficult to closely control in long
term production.
• During the execution of the DOE, hold noise variables
constant if you can, and record their values.
Introduction to Design of Experiments, Planning and Execution, July ‘05 45
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 46/58
The Importance of Wide Factor Level Ranges
• Wide ranges magnify effects, making it easier for the software
to detect which factors are important and the extent of their
importance.
• Wide ranges minimize the need for extrapolation post-DOE.
Introduction to Design of Experiments, Planning and Execution, July ‘05 46
M a g n
i f i e d e
f f e c
t
Response
Smalleffect
Factor A
Possible DOE factor ranges
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 47/58
Wide Factor Ranges, But Not Too Wide
• Typically wider than production ranges.
• Some runs may yield bad product; that’s OK, as long as the
product is recognizable and measurable in terms of the
responses.
• Make sure that all of the runs are doable before executing the
DOE; one missing run can prevent you from drawing useful
conclusions.
• The DOE procedure works only when the response is “smooth”
over the experiment space, so avoid factor ranges that yield
responses of zero or infinity.
Introduction to Design of Experiments, Planning and Execution, July ‘05 47
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 48/58
Signal-to-Noise Review
DOE operates on a signal-to-noise basis; signals are the effects of
the factors; noise includes the variation in the measurement process,
the inherent variation in the process under study, and the variation
within individual test articles.
Increase signal where you can
• Use wide factor ranges
Decrease noise where you can
• Report response values as the means of several test articlesper run.
• Where appropriate, report the official measurement for a testarticle as the mean of two or more observations made on thattest article.
• Make sure the process is stable prior to executing the DOE.
• Thoroughly account for all process variables and control themduring the execution of the DOE.
• During the execution of the DOE, treat each run and each testarticle identically except for the factor level changes indicatedby the experiment design.
• During the execution of the DOE, wait for the process tostabilize after making factor level changes.
Introduction to Design of Experiments, Planning and Execution, July ‘05 48
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 49/58
What about Factors that are a Nuisance to Vary?
Sort the design by the nuisance factor, and perform the runs in the
sorted order; that way you minimize the number of level changes
for the nuisance factor.
• With this approach, lurking variables may influence the
experiment, so make sure that the process conditions are
identical from run to run.
Introduction to Design of Experiments, Planning and Execution, July ‘05 49
• Of course, the best approach for guarding against lurking
variables is to randomize the run order.
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 50/58
Factors, Additional Notes
If factor levels are indicated by a knob position that does not
correspond to a calibrated scale. . .
• Use an independent gage to learn the true value of eachfactor level; it may be that knob position does not correspondto a linear scale. For example, knob position #4 may notcorrespond to twice the factor level of knob position #2. Thissituation can be a big barrier to analyzing your experiment.
Factors, continuous and categorical
• The levels for continuous factors can be set to (almost) anyvalue along some region of the real number line.
• The levels for categorical factors fall into a finite number ofcategories.
• Use continuous factors whenever possible.
Introduction to Design of Experiments, Planning and Execution, July ‘05 50
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 51/58
Introduction to Design of Experiments, Planning and Execution, July ‘05 51
Executing a DOE
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 52/58
Checklist for Executing a DOE
____ Are there enough test articles and materials for the DOE plus extra tocover mishaps and allow for a confirmation experiment?
____ Is the principal investigator available to answer questions?
(It’s a good practice for the principle investigator to be present for thefirst couple of runs to help make sure that there are nomisunderstandings about how the DOE should be executed)
Has team practiced/rehearsed all procedures related to the DOE?
____ Will the process be stable for the duration of the DOE execution?
____ Perform runs in the order specified.
____ Note any unusual occurrences during the DOE; record the time, run#, and test article number.
____ Hold all conditions the same for every run and every test article
except for the specified factor level changes.
____ Allow the process to warm up and stabilize after factor level changes.
____ Save all test articles at least until the DOE report is approved.
____ Always record individual data observations, even if means or otherstatistics will be used as “official” response values.
____ If a run is accidentally skipped, do it at the end of the experiment; the
run order is not as important as completing all of the runs.
____
Introduction to Design of Experiments, Planning and Execution, July ‘05 52
Know the minimum number of decimal places for recording the data.
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 53/58
Final Thoughts
• Include DOE tasks in your project schedule.
• Begin collecting DOE planning information long before you
need to execute your DOE.
• Process knowledge and engineering/scientific expertise
must be applied throughout the DOE process; the key is to
get good data; statistical aspects are of secondary
importance.
• Team effort
Introduction to Design of Experiments, Planning and Execution, July ‘05 53
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 54/58
Appendix A
If your response variables include both the mean and standard
deviation of a performance characteristic, plot the standard deviations
vs their corresponding means (choose Plot… Scatterplots… X-Y
Plot… from the menu bar). If the plot shows the standard deviations
increasing with the means, as shown below, it’s possible that one of
the basic DOE assumptions (homogeneity of variances) has not been
met. Take the natural log of the individual response measurements,generate the scatter plot again, and see if the transformation corrects
the problem; if the transformation solves the problem, analyze the log
response data just as you would without the transformation.
Introduction to Design of Experiments, Planning and Execution, July ‘05 54
Run Means
R u n
S t a n
d a r d
D e v
i a t i o n s
5 1 0
5
1 0
. 5
5 1 1
5
1 1
. 5
5 1 2
5
1 2
. 5
5 1 3
5
1 3
. 5
(X 0.001)
4.2
6.2
8.2
10.2
12.2
14.2
(X 0.0001)
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 55/58
References
Box, G. E. P., Hunter, W. G., and Hunter, J.S. (1978). Statistics for
Experimenters. Wiley, New York
Schmidt, S. R. and Launsby R. G. (1998). Understanding Industrial
Designed Experiments 4th ed. Air Academy Press, Colorado Springs,
Colorado
Taylor, W. T. (1995). Screening Experiments. Baxter Healthcare
Corp. short course
Hahn, G. J. (1977). Some Things Experimenters Should Know About
Experimental Design. J. Quality Technology. Vol. 9, No. 1
Cusimano J. (1996). Understanding & Using Design of Experiments.
Quality, April 1996
Introduction to Design of Experiments, Planning and Execution, July ‘05 55
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 56/58
Introduction to Design of Experiments, Planning and Execution, July ‘05 56
Notes
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 57/58
Introduction to Design of Experiments, Planning and Execution, July ‘05 57
Notes
7/18/2019 Planning for DOE, Lesson 1
http://slidepdf.com/reader/full/planning-for-doe-lesson-1 58/58
Notes