1

Experimental StatisticsExperimental Statistics - week 7 - week 7Experimental StatisticsExperimental Statistics - week 7 - week 7

Chapter 15:

Factorial Models (15.5)

Chapter 17:

Random Effects Models

2

Testing Procedure RevistedTesting Procedure Revisted2 factor CRD Design

Step 1. Test for interaction.

Step 2.(a) IF there IS NOT a significant interaction - test the main effects

(b) IF there IS a significant interaction - compare a x b cell means (by hand)

Main Idea:

We are trying to determine whether the factors effect the response either individually or collectively.

3

Statistics 5372: Experimental StatisticsAssignment Report Form

  Name: Data Set or Problem Description  Key Results of the Analysis

Conclusions in the Language of the Problem    Appendices:  

A. Tables and Figures Cited in the Report  B. SAS Log from the Final SAS Run    Notes: 

1. All assignments should be typed using a word processor according to the format above. 2. SAS output should consist only of tables and figures cited in the report. The report should refer to these tables and figures using numbers you assign, i.e. Table 1, etc. 3. The data should be listed somewhere in the report. (within SAS code is ok)

4

.204 .257

.170 .279

.181 .269

.167 .283

.182 .235

.187 .260

.202 .256

.198 .281

.236 .258

Auditory Visual

5 sec

10 sec

15 sec

WarningTime

1..y .192 2..y .264

.1.y .227

.2.y .219

.3.y .239

.. .228y

5

Note: For balanced designs,

..y

average of all data values

average of marginal row means

average of marginal column means

i.e. for STIMULUS data

.228 = (.227+.219+.239)/3

= (.192+.264)/2

6

.204 .257

.170 .279

.181 .269

.167 .283

.182 .235

.187 .260

.202 .256

.198 .281

.236 .258

Auditory Visual

5 sec

10 sec

15 sec

WarningTime

1..y .190 2..y .264

.1.y .231

.2.y .219

.3.y .239

.. ???y

Now Consider:

In this case, average of marginal row means

average of marginal col. means

7

• Every Combination of the Factor Levels has an Equal Number of Repeats

• Sums of Squares– Uniquely Calculated

» Usual Textbook Formulas

• Not Every Combination of the Factor Levels has an Equal Number of Repeats

• Sums of Squares– Not Uniquely Calculated

» Usual Textbook Formulas Are Not Valid

Balanced Experimental DesignsBalanced Experimental Designs

Unbalanced Experimental DesignsUnbalanced Experimental Designs

8

- they typically use “Textbook Formulas”

Many Software Programs Cannot Properly Calculate Sums of Squares for Unbalanced Designs

SAS:

- use Type III sums of squares

-- analysis is closest to that for “Balanced Experiments”

Unbalanced Experimental DesignsUnbalanced Experimental Designs

- must Use Proc GLM, not Proc ANOVA

- Type I and Type III sums-of-squares results will not generally agree

9

The GLM Procedure Dependent Variable: response Sum of Source DF Squares Mean Square F Value Pr > F Model 5 0.02547774 0.00509555 19.13 <.0001 Error 11 0.00293050 0.00026641 Corrected Total 16 0.02840824 R-Square Coeff Var Root MSE response Mean 0.896843 7.112913 0.016322 0.229471 Source DF Type I SS Mean Square F Value Pr > F type 1 0.02309680 0.02309680 86.70 <.0001 time 2 0.00122742 0.00061371 2.30 0.1460 type*time 2 0.00115351 0.00057676 2.16 0.1611 Source DF Type III SS Mean Square F Value Pr > F type 1 0.02367796 0.02367796 88.88 <.0001 time 2 0.00130085 0.00065042 2.44 0.1326 type*time 2 0.00115351 0.00057676 2.16 0.1611

Unbalanced Data -- GLM Output

10

Model for 3-factor Factorial Design

ijkm i j k

ij ik jk

ijk

ijkm

y

1 1 1

0a b c

i j ki j k

where

and also, the sum over any subscript of a 2 or 3 factor interaction is zero

11

2....

1 1 1 1

( )a b c n

ijkmi j k k

y y

= SSA + SSB + SSC

+ SSAB + SSAC + SSBC + SSABC + SSE

Sum-of-Squares Breakdown

(3-factor ANOVA)

12

3-Factor ANOVA Table(3-Factor Completely Randomized Design)

 

 

Source SS df MS F 

Main Effects

A SSA a 1 B SSB b 1C SSC c1Interactions

AB SSAB (a 1)(b1)AC SSAC (a 1)(c1)BC SSBC (b 1)(c1)ABC SSABC (a 1)(b1)(c1)

Error SSE abc(n 1)   Total TSS abcn  

/( 1)MSB SSB b

/ ( 1)MSE SSE ab n

/MSA MSE

See page 908

/( 1)( 1)MSAB SSAB a b

/MSB MSE/( 1)MSA SSA a

/MSAB MSE

/( 1)MSC SSC c /MSC MSE

/( 1)( 1)MSBC SSBC b c /( 1)( 1)MSAC SSAC a c

/( 1)( 1)( 1)MSABC SSAB a b c

/MSAC MSE/MSBC MSE/MSABC MSE

13

Popcorn Data

Factors(A) Brand (3 brands)

(B) Power of Microwave (500, 600 watts)

(C) 4, 4.5 minutes

n = 2 replications per cell

Response variable -- % of kernels that popped

14

Popcorn Data

1 500 4.5 70.31 500 4.5 91.01 500 4 72.71 500 4 81.91 600 4.5 78.71 600 4.5 88.71 600 4 74.11 600 4 72.12 500 4.5 93.42 500 4.5 76.32 500 4 45.32 500 4 47.62 600 4.5 92.22 600 4.5 84.72 600 4 66.32 600 4 45.73 500 4.5 50.13 500 4.5 81.53 500 4 51.43 500 4 67.73 600 4.5 71.53 600 4.5 80.03 600 4 64.03 600 4 77.0

15

PROC GLM;class brand power time;MODEL percent=brand power time brand*power brand*time power*time brand*power*time;Title 'Popcorn Example -- 3-Factor ANOVA';MEANS brand power time/LSD; RUN;

SAS GLM Code – 3 Factor Model

MODEL percent=brand power time brand*power brand*time power*time brand*power*time

The Statement

can be written as

MODEL percent=brand | power | time;

16

The GLM ProcedureDependent Variable: percent Sum of Source DF Squares Mean Square F Value Pr > F Model 11 3589.988333 326.362576 2.71 0.0503 Error 12 1444.170000 120.347500 Corrected Total 23 5034.158333

R-Square Coeff Var Root MSE percent Mean 0.713126 15.27011 10.97030 71.84167

Source DF Type I SS Mean Square F Value Pr > F brand 2 566.690833 283.345417 2.35 0.1372 power 1 180.401667 180.401667 1.50 0.2443 time 1 1545.615000 1545.615000 12.84 0.0038 brand*power 2 125.125833 62.562917 0.52 0.6074 brand*time 2 1127.672500 563.836250 4.69 0.0314 power*time 1 0.015000 0.015000 0.00 0.9913 brand*power*time 2 44.467500 22.233750 0.18 0.8336

17

Testing ProcedureTesting Procedure 3 factor CRD Design

Step 1. Test for 3rd order interaction.

IF there IS a significant 3rd order interaction - compare cell means

IF there IS NOT a significant 3rd order interaction - test 2nd order interactions

IF there IS NOT a sig. 2nd order interaction - test the main effects

IF there IS a significant 2rd order interaction - compare associated cell means

In general -- test main effects only for variables not involved in a significant 2nd or 3rd order interaction

18

Examine brand x time cell meansExamine Power main effect

The GLM ProcedureDependent Variable: percent Sum of Source DF Squares Mean Square F Value Pr > F Model 11 3589.988333 326.362576 2.71 0.0503 Error 12 1444.170000 120.347500 Corrected Total 23 5034.158333

R-Square Coeff Var Root MSE percent Mean 0.713126 15.27011 10.97030 71.84167

Source DF Type I SS Mean Square F Value Pr > F brand 2 566.690833 283.345417 2.35 0.1372 power 1 180.401667 180.401667 1.50 0.2443 time 1 1545.615000 1545.615000 12.84 0.0038 brand*power 2 125.125833 62.562917 0.52 0.6074 brand*time 2 1127.672500 563.836250 4.69 0.0314 power*time 1 0.015000 0.015000 0.00 0.9913 brand*power*time 2 44.467500 22.233750 0.18 0.8336

19

20

To complete the analysis:1. The F-test for Power was not significant (.2443)

2. Compare the 6 cell means plotted in interaction plot using procedure analogous to the one used for pilot plant data.

PROC SORT data=one;BY brand time; PROC MEANS mean std data=one;BY brand time; OUTPUT OUT=cells MEAN=percent; Title 'Brand x Time Cell Means for Popcorn Data'; RUN;

Obs brand time _TYPE_ _FREQ_ percent 1 1 4 0 4 75.200 2 1 4.5 0 4 82.175 3 2 4 0 4 51.225 4 2 4.5 0 4 86.650 5 3 4 0 4 65.025 6 3 4.5 0 4 70.775

LSD = 22

( ) α/MSE

tN

21

Popcorn Data

1 500 4.5 70.31 500 4.5 91.01 500 4 72.71 500 4 81.91 600 4.5 78.71 600 4.5 88.71 600 4 74.11 600 4 72.12 500 4.5 93.42 500 4.5 76.32 500 4 45.32 500 4 47.62 600 4.5 92.22 600 4.5 84.72 600 4 66.32 600 4 45.73 500 4.5 50.13 500 4.5 81.53 500 4 51.43 500 4 67.73 600 4.5 71.53 600 4.5 80.03 600 4 64.03 600 4 77.0

70.3+91.0+78.7+88.74

= 82.175

= cell mean for Brand 1 and Time 4.5

22

Models with Random Effects

Fixed-Effects Models -- the models we’ve studied to this point -- factor levels have been specifically selected - investigator is interested in testing effects of these specific levels on the response variable

Examples: -- CAR data - interested in performance of these 5 gasolines

-- Pilot Plant data - interested in the specific temperatures (160o and 180o) and catalysts (C1 and C2)

23

Random-Effect Factor -- the factor has a large number of possible levels

-- the levels used in the analysis are a random sample from the population of all possible levels

- investigator wants to draw conclusions about the population from which these levels were chosen

(not the specific levels themselves)

24

Fixed Effects vs Random Effects

This determination affects

- the model

- the hypothesis tested

- the conclusions drawn

- the F-tests involved (sometimes)

25

1-Factor Random Effects Model

ij i ijy

Assumptions:

'i s3. are independent.

4. 'ij s 2 normally distributed with mean 0 and variance

ii2. is th random observation on factor A

2-- normally distributed with mean 0 and variance

'ij s5. are independent.

i ij 6. The r.v.'s and are independent.

1. is overall mean

26

Hypotheses:

Ho:

Ha:

Ho says (considering the variability of the yij’s) :

- the component of the variance due to “Factor” has zero variance

-- i.e. no factor “level-to-level” variation

- all of the variability observed is just unexplained subject-to-subject variation

-- at least none is explained by variation due to the factor

27

DATA one;INPUT operator output;DATALINES;1 175.41 171.71 173.01 170.52 168.52 162.72 165.02 164.13 170.13 173.43 175.73 170.74 175.24 175.74 180.14 183.7;PROC GLM; CLASS operator; MODEL output=operator; RANDOM operator; TITLE ‘Operator Data: One Factor Random Effects Model';RUN;

These are data from an experiment studying the effect of four operators (chosen randomly) on the output of a particular machine.

t =

n =

28

The GLM Procedure Dependent Variable: output Sum of Source DF Squares Mean Square F Value Pr > F Model 3 371.8718750 123.9572917 14.91 0.0002 Error 12 99.7925000 8.3160417 Corrected Total 15 471.6643750

R-Square Coeff Var Root MSE output Mean 0.788425 1.674472 2.883755 172.2188

Source DF Type I SS Mean Square F Value Pr > F operator 3 371.8718750 123.9572917 14.91 0.0002 The GLM Procedure Source Type III Expected Mean Square operator Var(Error) + 4 Var(operator)

One Factor Random effects Model

29

We reject Ho : (p = .0002)

and we conclude that there isvariability due to operator

Conclusion:

Note:Multiple comparisons are not used in random effects analyses

-- we are interested in whether there is variability due to operator

- not interested in which operators performed better, etc. (they were randomly chosen)

30

2

0 2( 1, )We reject at significance level if B

TW

sH F F t n t

s

Rationale for F-test and critical region:

RECALL: 1-Factor RECALL: 1-Factor (Fixed-Effects(Fixed-Effects) ANOVA Table) ANOVA Table(page 389)(page 389)

2Ws estimates 2

2 Bs estimates 2

+ constant × 2- i - if no factor effects, we expect F ≈ 1;

- if factor effects, we expect F > 1

31

Expected Mean Squares for 1-Factor ANOVA’s (p.979)

  EMS

Source SS df MS Fixed Effects Random Effects  

Treatments SST t 1 MST  

Error SSE t(n 1) MSE  Total TSS tn  

2 2

11

t

ii

nt

2

2 2n

2

2

0 2( 1, )Reject at significance level if B

W

sH F F t tn t

s

Rationale for Test Statistic and Critical Region is the Same: Fixed or Random

32

DATA one;INPUT operator output;DATALINES;1 175.41 171.71 173.01 170.52 168.52 162.72 165.02 164.13 170.13 173.43 175.73 170.74 175.24 175.74 180.14 183.7;PROC GLM; CLASS operator; MODEL output=operator; RANDOM operator; TITLE ‘Operator Data: One Factor Random Effects Model';RUN;

These are data from an experiment studying the effect of four operators (chosen randomly) on the output of a particular machine.

33

The GLM Procedure Dependent Variable: output Sum of Source DF Squares Mean Square F Value Pr > F Model 3 371.8718750 123.9572917 14.91 0.0002 Error 12 99.7925000 8.3160417 Corrected Total 15 471.6643750

R-Square Coeff Var Root MSE output Mean 0.788425 1.674472 2.883755 172.2188

Source DF Type I SS Mean Square F Value Pr > F operator 3 371.8718750 123.9572917 14.91 0.0002 The GLM Procedure Source Type III Expected Mean Square operator Var(Error) + 4 Var(operator)

One Factor Random effects Model

34

Estimating Variance Components2 2( )E MST n 2( )E MSE

Solving for we get:

2 ( ) ( )E MST E MSEn

so, we estimate by

2ˆMST MSE

n

Also, 2ˆ MSE

35

For OPERATOR Data,

2ˆ

36

RECALL: 2-Factor Fixed-Effects Model

ijk i j ij ijky

1 1 1 1

0a b a b

i j ij iji j i j

where

2

's are independent and

normally distributed with mean 0 and variance

ijk

37

Expected Mean Squares for

2-Factor ANOVA with Fixed EffectsFixed Effects:

A

B

AB

Error

2 2

11

a

ii

nba

2 2

11

b

jj

nab

2 2

1( 1)( 1)

b

ijj

na b

2

Expected MS F-test

MSA/MSE

MSB/MSE

MSAB/MSE

38

2-Factor Random Effects Model

ijk i j ij ijky

Assumptions:

2(0,2. ) i N :2-- normally distributed with mean 0 and variance

6. 's, 's, 's and are independent rv'si j ij ij

1. is overall mean

2(0,3. ) j N :2(0,4. ) ij N :

2(0,5. ) ijk N :

Sum-of-Squares obtained as in Fixed-Effects case

39

Expected Mean Squares for

2-Factor ANOVA with Random EffectsRandom Effects:

A

B

AB

Error

2 2 2n bn

2

Expected MS

2 2 2n an

2 2n

40

To Test:2

0

2

: 0

: 0a

H

H

we use F =

20

2

: 0

: 0a

H

H

we use F =

we use F =

20

2

: 0

: 0a

H

H

Note: Test each of these 3 hypotheses (no matter

whether Ho: is rejected)

41

2-Factor Random Effects ANOVA Table

 

 

Source SS df MS F 

Main Effects

A SSA a 1  

B SSB b1

Interaction

AB SSAB (a 1)(b1)

Error SSE ab(n 1)   Total TSS abn  

/( 1)MSB SSB b

/ ( 1)MSE SSE ab n

/MSA MSAB

/( 1)( 1)MSAB SSAB a b

/MSB MSAB

/( 1)MSA SSA a

/MSAB MSE

42

Estimating Variance Components2-Factor Random Effects Model

2ˆMSAB MSE

n

2ˆ MSE

2ˆMSA MSAB

bn

2ˆMSB MSAB

an

(note error on page 986)

43

DATA one;INPUT operator filter loss;DATALINES;1 1 16.21 1 16.81 1 17.11 2 16.61 2 16.91 2 16.8 . . .4 1 14.94 2 15.44 2 14.64 2 15.94 3 16.14 3 15.44 3 15.6;PROC GLM; CLASS operator filter; MODEL loss=operator filter operator*filter; TITLE ‘2-Factor Random Effects Model'; RANDOM operator filter operator*filter/test;RUN;

1 2 3 4 16.2 15.9 15.6 14.91 16.8 15.1 15.9 15.2 17.1 14.5 16.1 14.9

16.6 16.0 16.1 15.42 16.9 16.3 16.0 14.6 16.8 16.5 17.2 15.9

16.7 16.5 16.4 16.13 16.9 16.9 17.4 15.4 17.1 16.8 16.9 15.6

Operator

Filter

Filtration Process:Response - % material lost through filtrationA – Operator (randomly selected) (a = )B – Filter (randomly selected) (b = )

n =

44

2-Factor Random Effects Model General Linear Models ProcedureDependent Variable: LOSS Sum of MeanSource DF Squares Square F Value Pr > FModel 11 16.60888889 1.50989899 8.16 0.0001Error 24 4.44000000 0.18500000Corrected Total 35 21.04888889

R-Square C.V. Root MSE LOSS Mean 0.789062 2.664175 0.4301163 16.144444

Source DF Type III SS Mean Square F Value Pr > FOPERATOR 3 10.31777778 3.43925926 18.59 0.0001FILTER 2 4.63388889 2.31694444 12.52 0.0002OPERATOR*FILTER 6 1.65722222 0.27620370 1.49 0.2229 Source Type III Expected Mean SquareOPERATOR Var(Error) + 3 Var(OPERATOR*FILTER) + 9 Var(OPERATOR)FILTER Var(Error) + 3 Var(OPERATOR*FILTER) + 12 Var(FILTER)OPERATOR*FILTER Var(Error) + 3 Var(OPERATOR*FILTER)

SAS Random-Effects Output(Filtration Data)

45

Tests of Hypotheses for Random Model Analysis of VarianceDependent Variable: LOSS Source: OPERATORError: MS(OPERATOR*FILTER) Denominator Denominator DF Type III MS DF MS F Value Pr > F 3 3.4392592593 6 0.2762037037 12.4519 0.0055Source: FILTERError: MS(OPERATOR*FILTER) Denominator Denominator DF Type III MS DF MS F Value Pr > F 2 2.3169444444 6 0.2762037037 8.3885 0.0183Source: OPERATOR*FILTERError: MS(Error) Denominator Denominator DF Type III MS DF MS F Value Pr > F 6 0.2762037037 24 0.185 1.4930 0.2229

SAS Random-Effects Output – continued

“../test” option