planning for doe, lesson 1
DESCRIPTION
Design of ExperimentsTRANSCRIPT
Engineering Statistics Training
Introduction to Design of Experiments
Planning and Executing
Dave Jackson, M.S. EW Engineering Statistics
Introduction to Design of Experiments, Planning and Execution, July ‘05 1
Contents Introduction to DOE -------------------------------------------------------------- 4
Prerequisites ------------------------------------------------------------------- 5 Training objective ------------------------------------------------------------- 6 What is DOE? ------------------------------------------------------------------ 7 Applications of DOE ---------------------------------------------------------- 8 DOE overview ----------------------------------------------------------------- 9 Why make an effort to learn DOE?---------------------------------------10 DOE vocabulary --------------------------------------------------------------13 2 and 3 level designs -------------------------------------------------------14 The empirical model ---------------------------------------------------------16 The steps for completing a DOE -----------------------------------------18 The sequential nature of DOE --------------------------------------------19
Planning for DOE ----------------------------------------------------------------20
• DOE planning prior to selecting a design --------------------------21 • Define the process you want to study -------------------------------22 • Define the experiment objectives ------------------------------------23 • Identify responses -------------------------------------------------------24 • The sample mean --------------------------------------------------------25 • The sample standard deviation ---------------------------------------36 • The %CV -------------------------------------------------------------------38 • Pass/fail responses ------------------------------------------------------39 • Sample size ---------------------------------------------------------------30 • Response calculation worksheet (example) -----------------------33 • Measuring the response ------------------------------------------------34
Introduction to Design of Experiments, Planning and Execution, July ‘05 2
• Response worksheet ----------------------------------------------------40
Contents Planning for DOE ------------------------------------------------------ continued
• Process variables worksheet (example) ----------------------------42 • Process variables (noise) ----------------------------------------------45 • The importance of wide factor level ranges -----------------------46 • Signal-to-noise review --------------------------------------------------48 • Factor that are a nuisance to vary -----------------------------------49 • Additional notes on factors --------------------------------------------50
Executing a DOE ----------------------------------------------------------------51
• Checklist for executing a DOE ----------------------------------------52
Final Thoughts --------------------------------------------------------------------53 Appendix A ------------------------------------------------------------------------54 References ------------------------------------------------------------------------55
Introduction to Design of Experiments, Planning and Execution, July ‘05 3
Introduction to DOE
Introduction to Design of Experiments, Planning and Execution, July ‘05 4
Prerequisites
• An interest in using Design of Experiments (DOE)
• An expectation for using DOE in the near future
• The ability to navigate in a Windows environment
• A working knowledge of basic statistical concepts. The training will reinforce and expand on these concepts using practical examples.
Introduction to Design of Experiments, Planning and Execution, July ‘05 5
Training Objective The objective of this course is to increase the likelihood of successful
experiment outcomes at Edwards.
The course will cover DOE planning, design selection, proper
execution, and analysis of DOE data using the Statgraphics software
package.
A successful experiment,
• Yields useful information that moves the project forward
• Does not have to be repeated
Introduction to Design of Experiments, Planning and Execution, July ‘05 6
What is DOE?
DOE is a procedure for learning how process variables acting
simultaneously affect, and interact to affect process performance.
Introduction to Design of Experiments, Planning and Execution, July ‘05 7
Estimated Response SurfaceMixer Speed=25.0
TemperatureTimeA
f Tem
pera
ture
556 558 560 562 564 240250260270280290300-11-10-9-8-7-6-5
Applications of DOE • Determine the process variable settings that simultaneously
optimize several performance characteristics • Screen out the important process variables from a large set of
candidate variables. • Reduce development time for new processes • Minimize cost by finding process variable settings that reduce the
cost of running a process • Reduce variation in a process
Introduction to Design of Experiments, Planning and Execution, July ‘05 8
Why Make the Effort to Learn DOE? DOE is a vast improvement over one-factor-at-a-time experimentation
and/or a series of trial-and-error efforts:
• More powerful – DOE gets the right answer more often
• More efficient – DOE gets the answer in fewer runs, with
fewer test articles and overall shortens
development time
• More actionable – DOE results are easier to interpret
Introduction to Design of Experiments, Planning and Execution, July ‘05 9
Why Make the Effort to Learn DOE? cont. Statistically Designed Experiments Can Detect and Describe
Interactions Between Input variables
Two variables interact if changing the level of one variable changes
the relationship between a second variable and the response. Time
and Temperature interact in the process of baking cookies.
Introduction to Design of Experiments, Planning and Execution, July ‘05 10
highlow
Flavor
Oven Temperature
Oven Time = short
Oven Time = long
Introduction to Design of Experiments, Planning and Execution, July ‘05 11
Why Make the Effort to Learn DOE? cont. Statistically Designed Experiments Can Deal With Curvature in the
Response.
Flavor
low
Amount of Salt in Food
medium high
ction to Design of Experiments, Planning and Execution, July ‘05 12
DOE Overview
A statistically designed experiment consist of a series of runs, or trials, in which (1) purposeful changes are made simultaneously to the input variables of a process, and (2) process performance is observed.
Introdu
Tissue Thickness
Flow Rate Power Pressure Duration
Ave. Lesion Width
SD Lesion Width
Ave. Lesion Depth
SD Lesion Depth
9 1 8 225 60 12 6 8 225 60
9 1 8 75 90 12 1 8 225 90 12 1 12 225 60
9 6 8 75 60 12 6 12 225 90
9 6 12 75 90 10.5 3.5 10 150 75 12 1 8 75 60 12 6 8 75 90 12 1 12 75 90
9 6 8 225 90
DOE Vocabulary
• There are typically many process variables that affect a response. Those process variables whose levels you change per your experiment design are called factors.
• Responses are the process performance characteristics that
you are interested in studying.
• An experiment design consists of the entire set of runs in the DOE.
• Runs are different combinations of factor levels. Identical back-
to-back runs require a complete reset of the process between runs.
• Center points are runs located in the center of the experiment
space.
• The experiment space is the multidimensional envelope
extending to the low and high levels of the factors.
Introduction to Design of Experiments, Planning and Execution, July ‘05 13
Factor B
Factor A
Factor C
Center points
Experiment space
2 and 3-level Designs
Two Level Designs:
• Factors are run at two levels (low and high) in the base design.
• Even when center points are added, the design is still
considered two-level.
• Used to model responses with, at most, a small amount of curvature.
• Screening experiments1
• Range finding
Three Level Designs:
• Factors are run at three or more levels in the base design • Center points are often an integral part of the base design
• Used to model significant curvature in the response
Introduction to Design of Experiments, Planning and Execution, July ‘05 14
1 A two-level experiment is called a “screening experiment” if its primary purpose is to screening-out the most important factors from a large group of candidate factors. In this scenario, we are interested in identifying significant main effects, rather than interaction effects or curvature effects; the latter assumed to be an order of magnitude less important.
Which Response Can You Model With a 2-Level Design?
Response Surface A
X
Y
Res
pons
e
00.20.40.60.81 0
80
Y
Response Surface B
XY
Res
pons
e
-4-2
-100
Response Surface C
XY
Res
pons
e
-2-30
-12
(X 1000)
Introduction to Design of Experiments, Planning and Execution, July ‘05 15
Answer: A only. B and C require a design with at least three levels.
The Empirical Model
The software builds a model which is a polynomial approximation for
the true and unknown relationship between each response and the
factors.
Here is the complete model (up to 3rd order) for a three-level
experiment involving three factors, x1, x2, and x3.
0 1 1 2 2 3 3
12 1 2 13 1 3 23 2 3 123 1 2 3
2 2 211 1 22 2 33 3
2 2 2 2 2112 1 2 113 1 3 122 1 2 133 1 3 223 2 3 233 2 3
3 3 3111 1 222 2 333 3
y b b x b x b x
b x x b x x b x x b x x x
b x b x b x
b x x b x x b x x b x x b x x b x x
b x b x b x ε+
= + + +
+ + + +
+ + +
+ + + + + +
+ + + +
2
Model Term
Definition
y
The estimated average response over the experiment space.
0b
Constant.
1 1 2 2 3 3, , b x b x b x
Main effects representing the linear part of the model.
Introduction to Design of Experiments, Planning and Execution, July ‘05 16
The Empirical Model cont.
Model Term
Definition
12 1 2 13 1 3 23 2 3, , b x x b x x b x x
The two-factor interactions representing mild curvature in the model.
123 1 2 3b x x x
The three-factor interaction. Usually considered small or not important for many processes.
2 2
11 1 22 2 33 3, , b x b x b x2
Quadratic effects representing significant curvature in the model.
2
112 1 2b x x , , , 2113 1 3b x x 2
122 1 2b x x2
133 1 3b x x , , 2223 2 3b x x 2
233 2 3b x x
Partial cubic effects. Usually considered small or not important for many processes.
3
111 1b x , , 3222 2b x 3
333 3b x
Cubic effects. Usually considered small or not important for many processes.
ε
The error term representing common cause variation not accounted for by the other terms. Assumed to have a normal distribution with mean zero and variance, 2σ .
Note that you can select designs to estimate three-factor interactions, partial cubic, and cubic effects, but they are seldom important and therefore usually not worth the extra expense to estimate.
Introduction to Design of Experiments, Planning and Execution, July ‘05 17
Steps for Completing a DOE The font size indicates the relative importance for having a successful DOE outcome.
1. Planning
2. Select Experiment Design
3. Execute Experiment & Collect Data
4. Analyze Data
5. Perform best-setup confirmation run, or validation
Introduction to Design of Experiments, Planning and Execution, July ‘05 18
The sequential Nature of DOE
DOE is iterative; rarely does one run a large, comprehensive design
in which a final conclusion is made.
Introduction to Design of Experiments, Planning and Execution, July ‘05 19
Factor A
Direction of better performance
First DOE
Second DOE
Factor B
Planning for DOE
Introduction to Design of Experiments, Planning and Execution, July ‘05 20
DOE Planning Prior to Selecting an Experiment Design
Introduction to Design of Experiments, Planning and Execution, July ‘05 21
Define the process you want to study
Define experiment objectives
Identify responses and measurement methods (IQ/OQ)
Identify process variables and measurement methods (IQ/OQ)
Start
Are interactions between factors likely?
Is curvature in the response likely?
Will any factor be a nuisance to vary?
End
Make sure process is stable (IQ/OQ)
Define the Process You Want To Study
• A process is a sequence of tasks required to accomplish something, or
• A process is a means of transforming inputs into outputs
IPO process model
Introduction to Design of Experiments, Planning and Execution, July ‘05 22
Product Design Influences
Product Performance
Variation in Product Performance
Equipment & Tooling
Procedures & Methods
Operator Influences
Material Properties
Environmental Influences
Process
Controllable process variables (Inputs)
Responses (Outputs)
Uncontrollable process variables (Noise)
Define the Experiment Objectives
• Clear statement of what you want to accomplish • Prioritize objectives
• State constraints
• Get team agreement
Poorly Stated Objective
Maximize seal strength.
Better Objective
Learn the process variable settings that maximize seal strength subject to the following constraints: (1) Sterile barrier pouches must pass the SOP leak test; (2) The visual appearance of the seals must pass SOP
requirements; and (3) Throughput must be greater than or equal to 20 pouches per
minute.
Introduction to Design of Experiments, Planning and Execution, July ‘05 23
Identify Responses • Select responses that are good predictors of field performance.
• Use the mean of several observations from a run as the official response value for a run.
• Use the standard deviation of several observations from a run as an additional response representing the variation in the process.
• Measure responses as continuous variables instead of categorical variables, whenever possible.
• If unsure about two measurement methods, use both.
• Omitting a key response variable can be a big mistake.
• Choosing responses and their measurement methods are typically the most time-consuming aspects of DOE planning.
Introduction to Design of Experiments, Planning and Execution, July ‘05 24
The Sample Mean
If n observations are denoted by , their mean is: 1 2,, , nx x xL
1 21 ( )nx x xnx + + += L .
Or, in more compact notation:
1ix xn= ∑
Suppose that these response data were taken during a particular run:
54 59 35 41 46
The mean response for the run is
1 (54 59 35 41 465
47
1ix xn
= + + + +
=
= ∑
Introduction to Design of Experiments, Planning and Execution, July ‘05 25
)
The Sample Standard Deviation
An experiment objective might be to minimize the standard deviation,
or at least make sure that there is no obvious indication that the
standard deviation appears noticeably worse within the
manufacturing window you end up selecting.
If n observations are denoted by , their variance is: 1 2,, , nx x xL
2 2 21 2
1 ( ) ( ) ( )1 ns x x x x xn⎡ ⎤⎣ ⎦= − + − + + −
−L 2x .
Or more compactly:
2 21 ( )1 ii
s xn= −− ∑ x
The standard deviation, s, is the square root of the variance
2s s=
Introduction to Design of Experiments, Planning and Execution, July ‘05 26
The Sample Standard Deviation cont.
Suppose these response data were taken during a particular run:
54 59 35 41 46
The sample standard deviation response is
2
2 2
2 2 2 2
1 ( )1
1 (54 47) (59 47) (35 47) (41 47) (46 47)1
93.51
9.67
i
s
s x xn
n
s
⎡ ⎤⎣ ⎦
= −−
= − + − + − + − + −−
=
=
=
∑
∑ 2
Introduction to Design of Experiments, Planning and Execution, July ‘05 27
The %CV Like the standard deviation, the coefficient of variation (%CV) may be
used as a response representing variation in performance. An
experiment objective might be to minimize the %CV.
The %CV of n observations 1 2,, , nx x xL is
% sample standard deviation 100 100sample meanCV sx= ⋅ ⋅=
For many processes, the variation increases as the mean increases. In this situation, the %CV is a better response than the standard deviation. See Appendix B for more information.
Introduction to Design of Experiments, Planning and Execution, July ‘05 28
Pass/Fail Responses
Consider as a last resort because the sample size requirements for
each run can reach into the hundreds
The percent defective for DOE run i, is
number defective in runˆ 100total number tested in runip = ⋅
Strategies:
• Substitute a continuous response whenever possible • If a Pass/Fail test must be used, redefine it so more test articles
fail
o Change a 10 psig test to a 30 psig test
• If a Pass/Fail test must be used, rate each test article as to the extent of failure
o 0=no failure, 1=25% failure, 2=50%, 3=75%, 4=100%
failure
Introduction to Design of Experiments, Planning and Execution, July ‘05 29
More Than One Sample Per Run Usually Required
The software assumes that the response value you provide for a particular run is close to the true (long-run) response for that run. An approach for getting near the true response value for the run is to use the mean of multiple samples processed during the run.
Response (units)
dens
ity
0 3 6 9 12 150
0.1
0.2
0.3
0.4
0.5
Run #1 Run #2
Long-run Performance
Suppose you make one observation per run. . .
You might get lucky and get these values; then the software can
accurately estimate the difference between the runs.
You might get these values; then the software will over estimate the difference between the runs.
You might get these values; then the software will underestimate the difference between the runs.
Introduction to Design of Experiments, Planning and Execution, July ‘05 30
What Sample Size (# Observations per Run) Do I Need? Use available feasibility data taken under a single set of process
conditions, and plot the data as shown below.
After about 12 test articles (for this example only) the mean remains
roughly unchanged.
n
mea
ns o
f the
firs
t n te
st a
rticl
es
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
-2
-1
0
1
2
3
4
Note: all 20 test articles were from the same run
Introduction to Design of Experiments, Planning and Execution, July ‘05 31
More on Sample Size and Noise
• The greater the variation (noise) in the response, the larger the
sample size needed to get good estimates for the response
values for the runs.
• To reduce noise, consider reporting the official measurement of
a test article as the mean of several measurements made on
that test article
o Use when there is noticeable within-test article variation
o Use when there is noticeable measurement variation
• The process must be stable during the execution of a DOE;
otherwise, process drift or jumps will likely obscure the real
effects of the factors
o Stability means in a state of statistical control. See
references on statistical process control (SPC).
Introduction to Design of Experiments, Planning and Execution, July ‘05 32
• Rule-of-thumb: it usually takes about twice as many samples
(observations) to get as good an estimate of the standard
deviation as it does the mean; however, most experimenters
will go with the sample size required for the mean and glean
whatever information they can about the standard deviation
response. This approach works pretty well unless your primary
experiment objective is to study and reduce process variation.
Response Calculation Worksheet (an example)
In this example, there were 10 samples used in Run #1 and 5 measurements per sample. The “Run Mean” and “Run SD” were the response values for Run #1 that were entered into StatGraphics
Introduction to Design of Experiments, Planning and Execution, July ‘05 33
Measuring the Response
• Measurement processes include:
o instruments (gages)
o procedures
o fixtures
o software
o people
o environment
o assumptions
• Basic requirements for measurement processes:
o good discrimination
o adequate range
o low bias
o good repeatability
Introduction to Design of Experiments, Planning and Execution, July ‘05 34
Measurement Discrimination • Discrimination is the smallest change response that the
measurement process can faithfully detect and indicate
• Discrimination should be no larger than about 1/10 of the
smallest difference in the response you require the experiment to
detect
Response (units)
dens
ity
0 3 6 9 12 150
0.1
0.2
0.3
0.4
0.5
Run #1 Run #2
Difference in Response
Long-run Performance
Introduction to Design of Experiments, Planning and Execution, July ‘05 35
Measurement Discrimination cont. • Alternatively, the discrimination should divide the process
variation into 10 parts or more (rule of tens). The process
variation is the long run, ±3σ spread in the response observed
over a single set of process conditions. The ±3σ spread is also
called the natural tolerance of the process.
Introduction to Design of Experiments, Planning and Execution, July ‘05 36
response measured in standard deviations
dens
ity
-4 -3 -2 -1 0 1 2 3 40
0.1
0.2
0.3
0.4
0.5
99.7% of observations
95% of observations
68% of observations
Normal Distribution Model
Measurement Discrimination cont.
The process standard deviation, �, may be estimated as follows:
4ˆ s
cσ = ,
where s is the center line from a standard deviation control chart,
and is a constant found in SPC references. 4c
Alternatively,
ˆ sσ = ,
Introduction to Design of Experiments, Planning and Execution, July ‘05 37
where is the sample standard deviation of n observations made over
a single set of process conditions.
s
Measurement Range & Bias
• Range refers to the smallest and largest values that a
measurement process can faithfully detect and indicate. Select
a gage will adequate range.
• Bias2 is the difference between the mean of multiple
measurements on the same reference standard and the
accepted value of the reference standard. Bias is a smaller-
the-better characteristic. Bias is usually handled through
calibration.
Introduction to Design of Experiments, Planning and Execution, July ‘05 38
2 Bias is sometimes referred to as “inaccuracy”
Introduction to Design of Experiments, Planning and Execution, July ‘05 39
Measurement Repeatability Repeatability is that portion of the observed process variation
attributable to the measurement process alone (assumes the same
operator makes all of the measurements).
Repeatability may be thought of as the sample standard deviation of
n measurements made at the same location on the same test article,
by the same person3.
Guidelines:
• Repeatability must be small compared to the smallest difference in the response that you want the experiment to be able to detect
• Alternatively, repeatability must be small compared to the ±3σ
process variation
• Where appropriate, report the official measurement for a test article as the mean of two or more observations made on that test article
3 This working definition may be used in the early stages of the DOE effort, if the results of a formal gage capability study are not be available
ction to Design of Experiments, Planning and Execution, July ‘05 40
Response Worksheet
Response Procedure & Spec
Estimated Range of
Data � � � / � Gage
Disc. Gage Range
Calib. ID
Low: High:
Low: High:
Low: High:
Low: High:
Low: High:
Low: High:
Low: High:
Low: High:
� = Smallest change in the response that is worth investigating � = Standard deviation of the experimental error (standard deviation of the means of identical runs)
Introdu
Completing the Response Worksheet
• For each line item, think about how you can reduce the overall
noise in the experiment.
• � / � expresses the smallest change in the response that is
worth investigating, as a multiple of the background noise.
• You use � / � in the StatGraphics Power Curve procedure to
calculate the number of extra replicate runs you need to add to
the base design.
• If a transformation is to be performed on a response variable,
then � and � need to be in transformed units.
Estimate � (sigma) as follows: 1. Select one set of run conditions 2. Set up, stabilize and run the process in the same manner as you
will for the DOE 3. Collect n observations, and calculate their mean 4. Completely reset the process 5. Repeat steps 2 through 4 two or more times.
Introduction to Design of Experiments, Planning and Execution, July ‘05 41
6. Calculate the standard deviation of the three or more means.
Process Variables Worksheet (example)
Constant, Noise, or
Factor Process Variable
DOE Value
Operator Influences C Process operator Single operator, Peng C Af measurement operator Single operator, Peng
Material Properties
C BioPhysio valve Model 3100 C BioPhysio 2-D flat pattern 193396-029 Rev E, Lot # 0989MG 3011/248-10 C Nitinol flat sheet MetalTex, p/n 193402, Lot #248-10 C Salt Nitrate Salt, p/n 123456001
C Does salt go bad over time or over # parts processed? No
C Quench water Tap water. Room temperature (20°C ± 5°C),
Equipment and Tooling C Shape-setting mandrel, 29 mm p/n 392664 C Salt bath Ajax Hultgren Salt Bath Furnace, model # 3100. C Mandrel rack, 5 up p/n 392827
C Mixer Accumix variable speed controller model MM23401C. ¼ Hp motor. Propeller location and configuration unknown.
C Salt bath temp controller
For gross adjustment only; record setting for each DOE run, but use the chart recorder to setup the DOE runs. Met ID 116229.
C Chart recorder & Temperature probe Met ID 117406
Procedures and Methods C Shape setting SOP 2394 red-lined, attached C Bath stabilization time >= 90 minutes
C Test article position on mandrel rack
Position the test article on the mandrel location marked. Do not load frames on the remaining 4 mandrels (assumption: the missing frames will not invalidate the DOE results with respect to how the process will behave in production).
C Number of mandrels on rack 5 mandrels C Orientation of rack in bath Test article at 6:00 with respect to operator.
C
Depth of salt in bath Start DOE: 21.5 inches
Introduction to Design of Experiments, Planning and Execution, July ‘05 42
Minimum height is 21 – 22 inches from bottom of bath.
C Depth of valve frames in bath Fixed by the geometry of the mandrel rack and by the depth of the salt.
Process Variables Worksheet (example) cont.
Constant, Noise, or
Factor Process Variable
DOE Value
Procedures and Methods continued
F
Temperature This is the temperature read from the chart recorder attached to the temperature probe in the vicinity of the shape setting mandrel. The temperature probe is located at approximately the same depth as the parts being shape set. Bath stabilization time after a DOE temperature adjustment, >= 15 minutes 556 – 564 oC
F Time 240 – 300 sec
F
Mixer RPM 20 knob setting = 368 spindle RPM 25 knob setting = 460 spindle RPM 30 knob setting = 552 spindle RPM No stabilization time required after DOE mixer adjustment. 2-unit knob increments. 20 - 30
C Time between lifting the product out of bath and quench. < 5 sec
C Quench time >= 60 sec
Environmental Influences C Salt bath location COE lab, first floor T&D building
Product Design Influences N/A
Introduction to Design of Experiments, Planning and Execution, July ‘05 43
N/A N/A
Completing the Process Variables Worksheet
• For each line item, think about how you can reduce the overall noise in the experiment.
• Serves as good documentation.
• Include all variables, even those believed only remotely influential.
• Keep in mind the six process variables categories.
• Brainstorm as the team watches the process operation or the
product build.
• Brainstorm as the team reviews applicable SOPs, process sheets, equipment manuals, and the like.
• Consider measurement discrimination, range, and bias as you
do for responses.
• Consider completing an input variables worksheet for each response measurement process.
Introduction to Design of Experiments, Planning and Execution, July ‘05 44
• Failure to include an important factor in your DOE will likely make the results unusable, especially if the factor is involved in an interaction.
Process Variables of the Noise Variety
• Noise variables are not closely controlled because, (1) they
are thought to have at most a small effect on performance,
and (2) they are considered difficult to closely control in long
term production.
• During the execution of the DOE, hold noise variables
constant if you can, and record their values.
Introduction to Design of Experiments, Planning and Execution, July ‘05 45
The Importance of Wide Factor Level Ranges
• Wide ranges magnify effects, making it easier for the software
to detect which factors are important and the extent of their
importance.
• Wide ranges minimize the need for extrapolation post-DOE.
Introduction to Design of Experiments, Planning and Execution, July ‘05 46
Mag
nifie
d ef
fect
Response
Small effect
Factor A
Possible DOE factor ranges
Wide Factor Ranges, But Not Too Wide
• Typically wider than production ranges.
• Some runs may yield bad product; that’s OK, as long as the
product is recognizable and measurable in terms of the
responses.
• Make sure that all of the runs are doable before executing the
DOE; one missing run can prevent you from drawing useful
conclusions.
• The DOE procedure works only when the response is “smooth”
over the experiment space, so avoid factor ranges that yield
responses of zero or infinity.
Introduction to Design of Experiments, Planning and Execution, July ‘05 47
Signal-to-Noise Review DOE operates on a signal-to-noise basis; signals are the effects of
the factors; noise includes the variation in the measurement process,
the inherent variation in the process under study, and the variation
within individual test articles.
Increase signal where you can
• Use wide factor ranges Decrease noise where you can
• Report response values as the means of several test articles per run.
• Where appropriate, report the official measurement for a test
article as the mean of two or more observations made on that test article.
• Make sure the process is stable prior to executing the DOE.
• Thoroughly account for all process variables and control them
during the execution of the DOE.
• During the execution of the DOE, treat each run and each test article identically except for the factor level changes indicated by the experiment design.
• During the execution of the DOE, wait for the process to
stabilize after making factor level changes.
Introduction to Design of Experiments, Planning and Execution, July ‘05 48
What about Factors that are a Nuisance to Vary?
Sort the design by the nuisance factor, and perform the runs in the
sorted order; that way you minimize the number of level changes
for the nuisance factor.
• With this approach, lurking variables may influence the
experiment, so make sure that the process conditions are
identical from run to run.
Introduction to Design of Experiments, Planning and Execution, July ‘05 49
• Of course, the best approach for guarding against lurking
variables is to randomize the run order.
Factors, Additional Notes If factor levels are indicated by a knob position that does not
correspond to a calibrated scale. . .
• Use an independent gage to learn the true value of each factor level; it may be that knob position does not correspond to a linear scale. For example, knob position #4 may not correspond to twice the factor level of knob position #2. This situation can be a big barrier to analyzing your experiment.
Factors, continuous and categorical
• The levels for continuous factors can be set to (almost) any value along some region of the real number line.
• The levels for categorical factors fall into a finite number of
categories.
• Use continuous factors whenever possible.
Introduction to Design of Experiments, Planning and Execution, July ‘05 50
Introduction to Design of Experiments, Planning and Execution, July ‘05 51
Executing a DOE
Checklist for Executing a DOE
____
Are there enough test articles and materials for the DOE plus extra to cover mishaps and allow for a confirmation experiment?
____
Is the principal investigator available to answer questions? (It’s a good practice for the principle investigator to be present for the first couple of runs to help make sure that there are no misunderstandings about how the DOE should be executed)
Has team practiced/rehearsed all procedures related to the DOE? ____
Will the process be stable for the duration of the DOE execution?
____ Perform runs in the order specified.
____ Note any unusual occurrences during the DOE; record the time, run
#, and test article number. ____
Hold all conditions the same for every run and every test article except for the specified factor level changes.
____
Allow the process to warm up and stabilize after factor level changes.
____
Save all test articles at least until the DOE report is approved.
____
Always record individual data observations, even if means or other statistics will be used as “official” response values.
____
If a run is accidentally skipped, do it at the end of the experiment; the run order is not as important as completing all of the runs.
____
Introduction to Design of Experiments, Planning and Execution, July ‘05 52
Know the minimum number of decimal places for recording the data.
Final Thoughts
• Include DOE tasks in your project schedule.
• Begin collecting DOE planning information long before you
need to execute your DOE.
• Process knowledge and engineering/scientific expertise
must be applied throughout the DOE process; the key is to
get good data; statistical aspects are of secondary
importance.
• Team effort
Introduction to Design of Experiments, Planning and Execution, July ‘05 53
Appendix A
If your response variables include both the mean and standard
deviation of a performance characteristic, plot the standard deviations
vs their corresponding means (choose Plot… Scatterplots… X-Y
Plot… from the menu bar). If the plot shows the standard deviations
increasing with the means, as shown below, it’s possible that one of
the basic DOE assumptions (homogeneity of variances) has not been
met. Take the natural log of the individual response measurements,
generate the scatter plot again, and see if the transformation corrects
the problem; if the transformation solves the problem, analyze the log
response data just as you would without the transformation.
Introduction to Design of Experiments, Planning and Execution, July ‘05 54
Run Means
Run
Sta
ndar
d D
evia
tions
510
510.
5
511
511.
5
512
512.
5
513
513.
5
(X 0.001)
4.2
6.2
8.2
10.2
12.2
14.2(X 0.0001)
References Box, G. E. P., Hunter, W. G., and Hunter, J.S. (1978). Statistics for
Experimenters. Wiley, New York
Schmidt, S. R. and Launsby R. G. (1998). Understanding Industrial
Designed Experiments 4th ed. Air Academy Press, Colorado Springs,
Colorado
Taylor, W. T. (1995). Screening Experiments. Baxter Healthcare
Corp. short course
Hahn, G. J. (1977). Some Things Experimenters Should Know About
Experimental Design. J. Quality Technology. Vol. 9, No. 1
Cusimano J. (1996). Understanding & Using Design of Experiments.
Quality, April 1996
Introduction to Design of Experiments, Planning and Execution, July ‘05 55
Introduction to Design of Experiments, Planning and Execution, July ‘05 56
Notes
Introduction to Design of Experiments, Planning and Execution, July ‘05 57
Notes
Introduction to Design of Experiments, Planning and Execution, July ‘05 58
Notes