Chapter 13 ANOVA

The Design and Analysis of Single

Factor Experiments - DOE

Chapter 13A

A nova (pl. novae) is a cataclysmic nuclear explosion caused by the accretion of hydrogen onto the surface of a white dwarf star

What is a DOE?

Doe is the name designating the females of certain animals, such as rabbits, squirrels and some species of antelope and deer

Doe may also refer to: Doe (ethnic group), a people of coastal Tanzania Ernest Doe & Sons, tractor manufacturers and agricultural merchants John Doe or Jane Doe a name used as a placeholder for an unknown

or anonymous person, especially in a legal context Doe is used in a song (Do-Re-Mi) in "The Sound of Music" DOE or DoE can be an acronym for:

the United States Department of Energy the Duke of Edinburgh Distributed Objects Everywhere in distributed computing Diffractive Optical Element in optics Date of Effect, the date an event comes into effect, used in contracts. Date Of Execution (When sentenced to the death penalty) Design of Experiments (statistics)

The Chapter in OutlineToday only

Design of Experiments (DOE) or (DOX)

The orderly collection and analysis of data with the objective of obtaining the most information with the least cost

DOE consists of set of treatments (factors)

• maybe qualitative or quantitative set of experimental units rules for assigning treatments to experimental units – e.g.

randomization taking measurements – response variable

• always quantitative analyzing the results - ANOVA

A treatment consists of a factor level or combination of factor levels

13-1 Designing Engineering Experiments

Every experiment involves a sequence of activities:

1. Conjecture – the original hypothesis that motivates the experiment.

2. Experiment – the test performed to investigate the conjecture.

3. Analysis – the statistical analysis of the data from the experiment.

4. Conclusion – what has been learned about the original conjecture from the experiment. Often the experiment will lead to a revised conjecture, and a new experiment, and so forth.

Why, this is nothing more than the

scientific method.

3 Facets of the experimental design Replication

makes possible the assessment of MSE controls the precision of the estimates (i.e. power

of the hyp. tests) Randomization

eliminates bias averages out systemic effects (apparent or hidden)

Control (a designed experiment) the statistical design used reduces experimental errors affects precision and power eliminate uncontrolled variables - blocking effect dictates analysis procedures

Completely Randomized (single factor) Design (CRD)

treatments assigned to experimental units completely at random each unit has an equal probability to receive any

treatment used when experimental units are homogeneous characteristics

flexible can vary sample sizes within treatments statistical analysis is straight-forward more df for MSE than other designs can have missing observations requires few assumptions

CRD - Example The Makeit Company uses three different machines (A, B, and C)

for processing a certain job in the manufacturer of its products. Of interest is the processing times among the 3 machines Each machine will process n jobs with the individual times

recorded 3n jobs are randomly assigned to each machine (experimental

unit) The question: Is there a significant difference in processing time

among the three machines?

TreatmentsMachine:A J1, J2, J3, … , Jn

B Jn+1, Jn+2, Jn+3, … , J2n

C J2n+1, J2n+2, J2n+3, … , J3n

sources of uncontrollablevariation may include jobdifferences and operatordifferences.

More to do with DOE Differs from regression

qualitative factors no assumptions necessary concerning

relationship with the response variable Nuisance variables

undesired sources of variation which may bias, if uncontrolled, the results

control by• holding constant• randomly assigning subjects• include as a factor• remove statistically (analysis of covariance)

Analysis of Variance -- ANOVA

Single factor of interest, two levels – Use t-test from Chapter 10.

Single factor of interest, several levels – ANOVA Example in text: A civil engineer wants to compare

several methods for curing concrete. The single factor is the curing method. There are more than two ways of curing concrete. Methods do not necessarily differ in just a quantitative

way, they may be based on using totally different procedures.

13-2 The Completely Randomized Single-Factor Experiment

13-2.1 An Example

13-2.1 An Example

An experimental unit gone awry

More 13-2.1 An Example

• The levels of the factor are sometimes called treatments.

• Each treatment has six observations or replicates.

• The treatments are run in random order.

13-2.2 The Analysis of Variance

Suppose there are “a” different levels of a single factor that we wish to compare. The levels are sometimes called treatments.

Fixed Effects – the “a” treatment levels are the ones we are interested in specifically.

Random Effects – the “a” treatments were selected randomly so that we can reach a general conclusion about the population of all treatments

Data LayoutTreatment Observations Totals Averages

....

...

.........

...2

...1

..21

.2.222221

.1.111211

yy

yyyyya

yyyyy

yyyyy

aaanaa

n

n

In the ‘dot’ notation, a dot (.) implies summation over an index.

The Model

We may describe the observations in previous table by the linear statistical model:

The model could be written as

Fixed-effects Model The treatment effects are usually defined as deviations from the overall mean so that:

a

i

n

jij

ii

n

jiji

a

ii

anNNyyyy

ainyyyy

1....

1..

..1

.

1

/

...,,2,1/

0

The grand mean our best estimate of .

The Hypotheses

Under the null hypothesis every observation yij = + ij

The analysis of variance partitions the total variability into two parts.

Partitioning Total variability

22..

1.

2

1 1

2.

2..

1 1 1.

2..

)()()1(

)()()(

p

a

ii

a

i

n

jiij

a

i

n

j

a

iiij

SaNyynSN

yyyynyy

SSSSTotalTotal = SS = SSTT = Total Sum = Total Sum of Squaresof Squares

• • a measure of a measure of the total variation the total variation

in the datain the data

SSSSTreatmentsTreatments = Treatment = Treatment

Sum of SquaresSum of Squares

• • a measure of a measure of the variation the variation

between between treatmentstreatments

SSSSErrorError = SS = SSEE = = ErrorError Sum of Sum of

SquaresSquares

• • a measure of the a measure of the variation within the variation within the individual samplesindividual samples

Df = an – 1 a – 1 a(n-1)

Chapter Seven Stuff – Expected Values of Sums of Squares

2

1

22

)1()(

)1()(

naSSE

naSSE

E

a

iiTreatments

Degrees of Freedom

SST: N = a*n observations -> use one to estimate the overall mean

a*n – 1 degrees of freedom.

SSTreatments: a treatment levels -> but constraint

requires means to sum to zero, so (a – 1) degrees of freedom.

SSError: n observations at each treatment -> use one to estimate

the treatment mean so a * (n – 1) degrees of freedom.

Can’t You See the F-Test Coming?

As always, we think about how things are distributed under the null hypothesis so we can come up with a test statistic.

Under the null hypothesis (all treatment effects are zero),

MSTreatments = SSTreatments / (a – 1)

is an unbiased estimator of variance (2).

Whether or not the null hypothesis is true,

MSError = SSError / [a (n-1)]

is also an unbiased estimator of variance (2).

)1(,1,0 ~)]1(/[

)1/(

naaE

Treatments

E

Treatments FMS

MS

naSS

aSSF

2

1

22

)1()(

)1()(

naSSE

naSSE

E

a

iiTreatments

The F-Test

If the treatment effects really are significant, then MSTreatments will be larger than MSE. So we will reject the null hypothesis if

f0 > f,a-1,a(n-1)

where f0 is the actual value of F0 for the sample we are looking at.

Note that this is a one-sided, upper tail test. There are no lower tail, or two-sided versions that would even make sense.

Computational Versions of Formulas

TreatmentsTE

a

i

a

iiTreatments

a

i

n

j

a

i

n

jijT

SSSSSS

N

y

n

yyynSS

N

yyyySS

i

ij

1

2..

2

1

2...

1 1

2..2

1 1

2..

.)(

)(

The ANOVA Table

Example 13-1

The Sums of All Squares

The ANOVA

MSE – our estimate of variance. MSE =

2.551^2

Reject H0 – Now What??

Individual 95% CIs For Mean

Based on Pooled StDev

Level N Mean StDev -----+---------+---------+---------+-

5 6 10.000 2.828 (---*---)

10 6 15.667 2.805 (---*----)

15 6 17.000 1.789 (---*---)

20 6 21.167 2.639 (---*----)

-----+---------+---------+---------+-

Pooled StDev = 2.551 10.0 15.0 20.0 25.0

Which of the treatments are significant?

n

MStyonIC E

naii )1(,2/. /:.. MSE – our estimate of variance. MSE =

2.551^2

Confidence Intervals of the treatment levels

For 20% hardwood, the resulting confidence interval on the mean is

Difference in Two Treatment Means

2 2 2. .

. . /2, ( 1)

( ) / / 2 /

2. . ( )

i j

i j i j a n

V Y Y n n n

MSEC I on is y y t

n

Notice there is no covariance term between sample mean i and j – they are independent

here.

Confidence level is 100(1 – ). What happens if we want to construct several of these confidence intervals?

Are all the C.I.’s independent? Product form of joint probability of all being correct?

CI on Differences

For the hardwood concentration example,

Look, no difference.

Multiple Comparisons

Controlling the error rate

For 4 treatment levels, there are pairwise comparisons

If each comparison is at the 5 percent level (i.e. 95 percent confidence intervals), then assuming independence among the CI’s, the probability of at least Type I error is 1 – (.95)6 = .2649.

4 4!6

2 2!2!

Multiple Comparisons – Which Treatments are Different?

The more comparisons you try to make, the greater the chance of error in a conclusion. As a corollary, your confidence intervals must become wider to hold error rate down. But that weakens the force of your conclusions.

Fisher’s LSD Method – Liberal. Probability of making one type I error over several comparisons is much greater than level you specified for a single test.

Graphical Method –Eyeball Test. Look back several slides. Not much to say here.

Reject H0 – Now What??

Individual 95% CIs For Mean

Based on Pooled StDev

Level N Mean StDev -----+---------+---------+---------+-

5 6 10.000 2.828 (---*---)

10 6 15.667 2.805 (---*----)

15 6 17.000 1.789 (---*---)

20 6 21.167 2.639 (---*----)

-----+---------+---------+---------+-

Pooled StDev = 2.551 10.0 15.0 20.0 25.0

Which of the treatments are significant?

n

MStyonIC E

naii )1(,2/. /:.. MSE – our estimate of variance. MSE =

2.551^2

Multiple Comparisons Following the ANOVA

The least significant difference (LSD) is

H0: i = i for all i,j H1: i = i

. .0

2i j

E

y yt

MSn

. .i jy y LSD significant if:

Example 13-2

Multiple Comparisons

Fisher’s LSD Method

. .

/2, ( 1)

2

i j

Ea n

y y LSD

MSLSD t

n

Intervals for (column level mean) - (row level mean)

5 10 15

10 -8.739

-2.594

15 -10.072 -4.406

-3.928 1.739

20 -14.239 -8.572 -7.239

-8.094 -2.428 -1.094

Fisher's pairwise comparisons

Family error rate = 0.192

Individual error rate = 0.050

Critical value = 2.086

Pooled StDev = 2.551 = sqrt(MSE)

T statistic with 20 d.f. at

alpha = .05

Probability of no Type I errors among the comparisons is 1 - .192 = .808

Interval half-widths2.086*2.551*sqrt(2/6) =

3.07

The Excel Way

32 individuals were randomly assigned to one of 4 groups. Each group of 8 subjects was trained to perform a complex

task by a different instructor Each individual then performed the assigned task with the

task time being the measured response.

experimental units 3 of the 4 instructors

The Data

Instructor A B C D sum mean1 3 4 7 7 21 5.252 6 5 8 8 27 6.753 3 4 7 9 23 5.754 3 3 6 8 20 55 1 2 5 10 18 4.56 2 3 6 10 21 5.257 2 4 5 9 20 58 2 3 6 11 22 5.5

total 22 28 50 72 172mean 2.75 3.5 6.25 9 5.375

Task times in hours

Excel ANOVA

Anova: Single Factor

SUMMARYGroups Count Sum Average Variance

A 8 22 2.75 2.21429B 8 28 3.5 0.85714C 8 50 6.25 1.07143D 8 72 9 1.71429

ANOVASource of Variation SS df MS F P-value F critBetween Groups 194.5 3 64.8333 44.2764 9.32E-11 2.946685Within Groups 41 28 1.46429

Total 235.5 31

5% level

Minitab

One-way Analysis of Variance

Analysis of VarianceSource DF SS MS F PFactor 3 194.50 64.83 44.28 0.000Error 28 41.00 1.46Total 31 235.50 Individual 95% CIs For Mean Based on Pooled StDevLevel N Mean StDev ---+---------+---------+---------+---A 8 2.750 1.488 (---*---) B 8 3.500 0.926 (---*---) C 8 6.250 1.035 (---*---) D 8 9.000 1.309 (---*---) ---+---------+---------+---------+---Pooled StDev =1.210 2.5 5.0 7.5 10.0

Fisher's pairwise comparisonsmean differences (COL - ROW)

A B CB -0.75C -3.5 -2.75D -6.25 -5.5 -2.75

Fisher's pairwise comparisonsFamily error rate = 0.0875Individual error rate = 0.0200

Critical value = t.02,28= 2.467

/2, ( 1)

2

2

2 1.212.467 1.493

8

Ea n

MSLSD t

n

Family error rate = 0.195Individual error rate = 0.0500

Critical value = t.05,28 = 2.048

22 1.21

2.048 1.2398

LSD

More Minitab

98 PERCENTIntervals for (column level mean) - (row level mean)

A B C

B -2.243 0.743

C -4.993 -4.243 -2.007 -1.257

D -7.743 -6.993 -4.243 -4.757 -4.007 -1.257

This completes today’s class

Listen closely next time for a discussion on the Random Effects Model and Randomized Complete Block Designs

And we learned so much today.