better industrial and scientific experiments: the overview and new directions

Better Industrial and Scientific Experiments: The Overview and New Directions

byJames M. Lucas and Derek F.

Webb2002 Fall Technical Conference

Valley Forge,PAOctober 17-18, 2002

J. M. Lucas and Associates 2

Contact InformationJames M. LucasJ. M. Lucas and

Associates5120 New Kent Road Wilmington, DE

19808(302) [email protected].

net

Derek F. WebbBemidji State Univ.HS-341, Box 231500 Birchmont Dr.

NEBemidji, MN 56601(218) [email protected]

PRELIMINARIES

How do you run Experiments?


QUESTIONS How many of you are involved with

running experiments? How many of you “randomize” to guard

against trends or other unexpected events?

If the same level of a factor such as temperature is required on successive runs, how many of you set that factor to a neutral level and reset it?


ADDITIONAL QUESTIONS

How many of you have conducted experiments on the same process on which you have implemented a Quality Control Procedure?

What did you find?


COMPARING THE RESIDUAL STANDARD DEVIATION FROM AN EXPERIMENT WITH THE RESIDUAL STANDARD DEVIATION FROM AN IN-CONTROL PROCESS

MY OBSERVATIONS

EXPERIMENTAL STANDARD DEVIATION IS LARGER. 1.5X TO 3X IS COMMON.


Research Team Huey Ju Jeetu Ganju Frank Anbari

Malcolm Hazel Derek Webb John Borkowski


IMPLICATIONS

HOW SHOULD EXPERIMENTS BE CONDUCTED?

•“COMPLETE RANDOMIZATION” (and the completely randomized design)

•RANDOM RUN ORDER (Often Achieved When Complete Randomization is Assumed)

•SPLIT PLOT BLOCKING (Especially When There are Hard-to-Change Factors)


Results for Experiments with Hard-to-Change and Easy-to-Change Factors

One H-T-C or E-T-C Factor: use split-plot blocking

Two H-T-C Factors: may split-plot Three or more H-T-C Factors:

consider RRO or Low Cost Options Consider “Diccon’s Rule”: Design

for the H-T-C Factor


Not Resetting Factors Common practice Not addressed by the classical

definition Gives a split-plot blocking structure with

the blocks determined at random May be cost effective Causes biased hypothesis tests over all

randomizations (Ganju and Lucas 1997)


RRO EXPERIMENTS (Random Run Order Without Resetting Factors)

OFTEN USED BY EXPERIMENTERS NEVER EXPLICITLY RECOMMENDED

ADVANTAGES•Often achieves successful results•Can be cost-effectiveDISADVANTAGES•Often can not be detected after experiment is conducted (Ganju and Lucas 99)•Biased tests of hypothesis (Ganju and Lucas 97, 02)•Can often be improved upon•Can miss significant control factors


AN ESSENTIAL INGREDIENT OFRANDOM RUN ORDER (RRO)EXPERIMENTS (DuPont QM&TC)


ADVANTAGES OF COMPLETE ADVANTAGES OF COMPLETE RANDOMIZATIONRANDOMIZATION

INDEPENDENT ERRORS GUARDS AGAINST TRENDS AND CYCLES VALIDATES RANDOMIZATION TESTS SIMPLE ANALYSIS NOTE: IN ADDITION TO A RANDOM RUN

ORDER, THE PHYSICAL ACT OF RESETTING IS NEEDED TO ACHIEVE “COMPLETE” RANDOMIZATION


DISADVANTAGES OF COMPLETE DISADVANTAGES OF COMPLETE RANDOMIZATIONRANDOMIZATION

MORE TIME REQUIRED MORE EXPENSIVE LESS EFFICIENT

For easily changed factors


Defining the Properties of Random Run Order Experiments


A Fundamental TheoremTheorem 1. The expected covariance matrix, V, for an

experiment, which uses a random run order, is:

V p I p UUs w w ( ( ) ) ( ) 2 2 21

With one hard-to-change factor, the value of p is

• 1, for a completely restricted experiment;

• 0, for a completely randomized experiment;

• , for a random run order. 2)1(

21 LLk

Ju 1992, Ju and Lucas 2002, extended by Webb 1999, Webb, Lucas and Borkowski 2002

Some Examples of Super-Efficient Experiments

Optimum Blocking with one or two Hard-to-Change Factors


24 with one Hard-to-Change Factor Obs. A B C D Blk 1 - - - - 1 2 - - + + 1 3 - + + - 1 4 - + - + 1 5 + - - - 2 6 + - + + 2 7 + + - + 2 8 + + + - 2

Obs. A B C D Blk 9 - - - + 3 10 - - + - 3 11 - + + + 3 12 - + - - 3 13 + - - + 4 14 + - + - 4 15 + + - - 4 16 + + + + 4


24 with two Hard-to-Change Factors Obs. A B C D Blk. 1 - - - - 1 2 - - + + 1 3 - + + - 2 4 - + - + 2 5 + - - - 3 6 + - + + 3 7 + + - + 4 8 + + + - 4

Obs. A B C D Blk.

9 - - - + 5 10 - - + - 5 11 - + + + 6 12 - - - - 6 13 + - - + 7 14 + - + - 7 15 + + - - 8 16 + + + + 8


24 Split-Plot is Super Efficient Main Effects plus interaction Model

11 Terms = (1 + 4 + 6) For I and A the variance is

02/16 + 1

2/4 For other terms it is 0

2/16 All terms of a CRD have (0

2 + 12 )/16

G-efficiency= 11(0

2 + 12)/(11 0

2 + 8 12)


24 with two Hard-to-Change Factors

Nest Factor B within each A block giving a split-split-plot with 8 Blocks I=A1=BCD1=ABCD1=B2=AB2=CD2=ACD2

I and A have variance 02/16 + 1

2/4 +22 /8

B, AB and CD have 02/16 + 2

2 /8 Other terms have variance 0

2/16 G-efficiency =

11(02+1

2+22)/(110

2+812+102

2 ) >1.0


2 BlocksBlock Size = 8

4 Blocks Block Size = 4

Lambdar = $Hard/$Easy is the Ratio of the Costs of Changing the Hard-to-Change Factor and the Easy-to-Change Factors.

LAMBDA is the Ratio of the Hard-to-Change Factor's Variance Component and the Other Variance Component.

IMPLICATIONSOptimum Block Size (Considering Costs)

24 DesignMain Effectsand 2 FactorInteractionsModel

r


26-1 with one or two Hard-to-Change Factors

Main Effects plus interaction Model 22 Terms = (1 + 6 + 15)

Use Resolution V, not VI with I=ABCDEUse four blocks I=A=BCF=ABCF=BCDE=ADEF=DEF

Nest Factor B within each A block giving a split-split-plot with 8 Blocks =B2=AB2=CF2=ACF2=CDE2=ABDEF2=BDEF2

I and A have variance 02/32 + 1

2/4 +22 /8

B, AB and CF have 02/32 + 2

2 /8 Other terms have variance 0

2/32 G-efficiency =

22(02+1

2+22)/(220

2+1612+202

2 ) >1.0 Drop 2

2 terms for one h-t-c factor results

Super Efficient Experiments are not always Optimal

Example :24 Main effects model


24 Main Effects Model Super-Efficient 8 blocks design with

I=A=BC=ABC=CD=ACD=BD=ABD V(b0) = V(b1) = A

2/8 + 2/16

V(b2) = V( b3)= V( b4) = 2/16

max variance = (4A2 + 52)/16

Design can be improved upon when A

2/2 >2.5 by a 12-block design


12-Block 24 Design I = CD in four blocks; I = -CD in eight blocks

Randomized four block weightsV(b0) = V(b1) (A

2 + 2)/8 A2/4 + 2/8 2/3, 1/3

V(bi,i>1) (A2 + 2)/8 2/8 0, 1

The maximum variance is: max variance = A

2/6 + 372/72

Super-Efficient max variance = (4A2 + 52)/16

A2=3, 2=1 gives 73/72 vs 17/16


IMPLICATIONS SUPER EFFICIENT EXPERIMENTS (With One or Two Hard-to-Change Factor) SPLIT PLOT BLOCKING GIVES HIGHER PRECISION AND LOWER COSTS THAN COMPLETELY RANDOMIZED EXPERIMENTS

Random Run Order Experiments

See Webb, Lucas and Borkowski handout

Useful for RSM and for more than two H-t-C Factors

Response Surface Example

Good Experimental Practiceand Split-Plot Analysis


One Factor at a Time(OFAT)Experiments


2k Changing O-F-A-TThe lowest cost 2k experiment

FACTORS (Ranked in Increasing H-T-C Order)

Factor 1 2 3 4 … First Consecutive Same Sign 1 2 4 8 … Then Switch Signs Every 2 4 8 16 … Number of Changes 2k-1 2k-2 2k-3 2k-4 … - - - - … + - - - … + + - - … - + - - … - + + - … + + + - … + - + - … - - + - … - - + + …


Other O-F-A-T 2k Designs More Balanced 2-2-3 Changes instead of 1-2-4 Least Expensive Way to Run Require large S/N Ratio


Conclusions Developed properties of RRO

experiments Given Implications Exciting research area Much more to do

better industrial and scientific experiments: the overview and new directions

Documents

c factors

scientific experiments

running experiments

splitplot blockingtwo

change factorsone ht

c factorj

complete randomizationj

change factorsj