Outline of Stratification Lectures

• Definitions, examples and rationale (credibility)

• Implementation– Fixed allocation (permuted blocks)– Adaptive (minimization)

• Rationale - variance reduction

Stratification

• A procedure in which factors known to be associated with the response (prognostic factors) are taken into account in the design (e.g., randomization)

• Pre-stratification refers to a stratified design; post-stratification refers to the analysis

Pre- versus Post Stratification and Precision (Variance Reduction)

• As a general rule, the precision gained with pre- versus post-stratification is less than one might expect

• The gain in precision is greatest in small studies (where you need it the most) because the risk of chance imbalance is greater.

• Covariate adjustment for prognostic factors is usually carried out with regression (e.g., linear, logistic, or proportional hazards regression.

Stratification Can Increase Precision

Simple versus stratified random sampling. Snedecor and Cochran note (p. 520): “If we

form strata so that a heterogeneous population is divided into parts each of which is fairly homogeneous, we may expect a gain in precision over simple random sampling”.

Ref. Snedecor and Cochran, Statistical Methods

Stratification Can Increase Precision

Randomized block versus completely random design.

Snedecor and Cochran note (p. 299): “Knowledge (about predictors or response) can be used to increase the accuracy of experiments. If there are a treatments to be compared,…first arrange the experimental units in groups of a, often called replications. The rule is that units assigned to the same replication should be as similar in responsiveness as possible. Each treatment is then allocated by randomization to one unit in each replication…Replications are therefore usually compact areas of land…This experimental plan is called randomized blocks.”

Pre-stratification Does Not Matter.

Peto et al note: “As long as good statistical methods,…,are used to analyze data from clinical trials, there is no need for randomization to be stratified by prognostic features.”

• Keep it simple so investigators are not discouraged from participating.

• Post-stratified analysis is needed with pre-stratification anyway.

• Improvement in sensitivity (precision) with pre-stratification compared to letting stratum sizes be determined by chance is small.

Peto R et al., Br. J Cancer, pp. 585-612,1976

Stratified Design for Comparing Treatments

Stratum A B1

2

3

4

m1

m2

m3

m4

na nb

Treatment

m1A

m2A

m3A

m4A

m1B

m2B

m3B

m4B

• Typical situation:

m1 ≠ m2 m3 m4≠ ≠• Study is designed/powered based on na and nb

• Goal: miA = miB for all i.

How much of a price does one pay with respect to precision by trusting randomization to achieve reasonable balance?

Consider the relative efficiency of a stratified design to an unstratified design:

Var (treatment contrast with stratification)

Var (treatment contrast with no stratification in design, but post-stratified analyses)

RE =

Pooling Estimates

Estimates: E , E

Var (E ) = Var (E ) =

w E + w EPooled Estimate:

Best Pooled Est: w = , w =

w + w Variance Pooled Est:

w + w

1 1

1 2

2

121

22

1 1

1

2 2

21

21

22

(w + w )1 1

21

21

22

2

2

22

2

Continuous response, equal variance - effect of chance imbalance

nA = total number randomly assigned to AnB = total number randomly assigned to Bg = fraction of those given A with prognostic factorh = fraction of those given B with prognostic factor

Treatment

Stratum

A BS1

S2

An g

Bn h

nA nB

(1-g)nA (1-h)nB

RE =A

n gB

n h(1-g) (1-h)An g

Bn h+

+ An g

Bn h+

An

Bn +1 -

-1

RE obtained by noting:1) Var( y - y ) =

1A 1B 1 +n g A

1n h

+

( ) 2

2) Var(y - y ) = 2A 2B n

An

B( ) 2(1-g) (1-h)

3) Pooled variance is:

VarPooled(y - y ) = A B

wi2 Var (y - y )

iA iB2

wi 2

w1 =A

n gB

n h+An g

Bn h

An

Bn (1-g) (1-h)+w2 = A

n B

n (1-g) (1-h)

B1 1

For Stratified Design, g = h

w =1 n

An B

+n

An

Bg

w =2 n

An

B+n An

B(1 - g)

Assume A

n B

n =

RE = g(1-g) h(1-h)g + h

+g + h

21 - e.g., block randomization used

Consider the case of g = 2h:

0.10, 0.05 0.990.25, 0.1250.970.50, 0.25 0.930.75, 0.3750.86

g, h RE

-1

Bernouli Response

Loss of efficiency =h2

n21 -

h = lack of balance2n = number in each stratum

Ref: Meier (Controlled Clinical Trials, 1981)

This can be seen by noting:

1) Stratified design, for stratum 1

Var(p - p ) = A B

since n = n = nA1 B1

1 1

1nA 1

+ 1nB 1

p1q1( )

2n= p1 q1( () )

Note: q1 = 1- p1

The ratio of these variances is proportional to:

2) No stratification in design; post-stratification in analysisVar(p - p ) =

A B11

1 n+h + 1

n-h p1q1( )

1/n+h + 1/n-h2/n = h2

n21 -

n = 10

1 (11, 9) 0.99

2 (12, 8) 0.96

4 (14, 6) 0.86

5 (15, 5) 0.75

h RE(n , n )1A 1B

Example:Brown et al. Clinical Trial of Tetanus Anti-toxin in Treatment of Tetanus. Lancet, 227-30;1960 (see also Meier, Cont Clin Trials, 1981; a slightly different approach is taken here).

Anti-Toxin (A)

Alive

Dead

21 9

20 29

No Anti-Toxin (B)

41 38

30

49

79

p = overall death rate == 0.620

4979

^

= 20/41 = 0.488p̂A

= 29/38 = 0.763p̂B

p̂A

p̂B

- = -0.275

Var p̂A

p̂B

-( ) = +A

1n

B

1n ][ p̂ 1 p̂-( )

= 1 41

1 38

4979

49791 -[ ( ( ))]+

= 0.01195

SE 0.109

p̂A

p̂B

-( ) =

Time from first symptoms to admission turned out to be an important prognostic factor; therefore, post-stratification was carried out.

A

Alive

Dead

10 4

18 26

B

28 30

A

Alive

Dead

11 5

2 3

B

13 8

< 72 Hours ≥ 72 Hours

Stratum 1: < 72 hoursStratum 2: ≥ 72 hours

p̂1A

p̂1B

- = - 0.223

p̂1A= 0.643 p̂

1B= 0.866

p̂1= 0.759

SE( ) = 0.112

p̂1A

p̂1B

-

p̂2A

p̂2B

- = - 0.221

p̂2= 0.238

SE( ) = 0.191p̂2A

p̂2B

-

Weighted diff. (ˆ p A - ̂ p B)wˆ

Let G = fraction of patients in Stratum 1

= 58/79 = 0.734(̂ p A - ̂ p B)w = ̂ G (̂ p 1A - ̂ p 1B) + (1 - G)(̂ p 2A - ̂ p 2B) = - 0.223

compared to - 0.275 unweighted

VAR(p - p ) = G VAR(p - p )ˆ A ˆ B w

ˆ 1A

ˆ 1B ̂

SE(p - pˆ A - ̂ B)w = .097+ (1-G ) VAR(̂ p 2A - ̂ p 2B) = 0.009382

2

ˆ

ˆ

Gain in precision achieved with post-stratification

Var(post-stratification)Var(no stratification)

RE =

= 0.009380.01195

= 0.78 22% reduction

How much gain in precision would be achieved if stratification was used in the

design?

Force balance within stratum

Assume ̂ p ‘s don’t change ij

ˆ 1A ˆ 1B

ˆ 2A ˆ

2B

A

Alive

Dead

B

29 29

A

Alive

Dead

B

11 10

< 72 Hours ≥ 72 Hours

SE(p - p ) = 0.109instead of 0.112

SE(p - p ) = 0.186instead of 0.191

Var(stratified design)Var(no stratification)

RE1

= (0.096)(0.109)

= 0.77 23% reduction

=

2

2

Var(stratified design)

Var(post-stratification)

RE2

= (0.096)(0.097)

= 0.98 2% reduction

=

2

2

SE stratified design = 0.096 (same weights are used)

0.50 0.60 0.30 0.910.20 0.60 0.30 0.940.10 0.60 0.30 0.96

0.50 0.60 0.20 0.830.20 0.60 0.20 0.870.10 0.60 0.20 0.92

0.50 0.10 0.05 0.9910.20 0.10 0.05 0.9920.10 0.10 0.05 0.996

G REP1. P2.

Gp1.(1-p1.) + (1-G)p2.(1-p2.)[ Gp1. + (1-G)p2. ] [1 – Gp1. – (1-G)p2.]

RE =

If p1. = p2. Then RE = 1

1) the distribution of the prognostic factor in the population;

2) the relative strength of the prognostic factor; and

3) the expected endpoint rate in the group studied.

The reduction in variance achieved with post-stratification depends on:

Scott’s Survey of Trials Published in Lancet and N Eng J Med in 2001

StratificationPermuted block 43/150Minimization 6/150Other adaptive 3/150Other 19/150Unspecified 79/150

Scott et al. Cont Clin Trials 2002; 23:662-674

Kahan’s Survey of 258 Trials Published in Four Major Medical

Journals in 2010Method of RandomizationSimple: 4Permuted blocks, no stratification: 40Permuted blocks, stratification: 85Minimization: 29Other: 4Unclear: 96

Kahan BC et al. BMJ 2012; 345:e5840

Conclusions

1. Usually there is little loss of efficiency with post-stratification as compared to a stratified design.

2. Loss of efficiency results from large chance imbalances for important prognostic factors, which are more likely in small studies.

3. Stratified designs should be considered in small studies (n < 50) with important prognostic factors.

4. Strictly speaking, analysis should account for pre-stratification.

Recommendation for Multi-Center Trials:Always Consider Stratification on Center

1. Clinic populations differ.

2. Treatment differs from clinic to clinic.

3. Each center represents a replicate of overall trial – can investigate treatment x clinic interactions.

4. In some trials (surgery), it may be better to stratify on surgeon within clinic.

5. If there are a very large number of clinical sites, small block size may have to be used and site combined into a priori defined larger strata (e.g., region or country) for analysis

General Recommendations

• Large trials– Block randomization with stratification by center– Stratification on other factors not necessary (I am a lumper)– If needed, usually okay to carry out block

randomization within each stratum

• Small trials– Block randomization with stratification by center– If stratification on other factors is considered,

may have to use an adaptive approachThese are consistent with Freidman, Furberg and DeMets (see page 111)