1 randomization in practice. unit of randomization randomizing at the individual level randomizing...

1

Randomization in Practice

Unit of randomization

• Randomizing at the individual level

• Randomizing at the group level– School– Community / village– Health center– District

• How do you tell which level to randomize at?

Unit of randomization

• Individual randomization gives you a bigger sample size at lower cost

• Politically may be difficult to have unequal treatment within a community

• Program can only be implemented at a certain level

• Spillovers

How to randomize?

• The lottery• Phase-in design• Rotation design• Encouragement design

NB: These are not mutually exclusive

4

How to randomize?

• Simple case: the lottery– There is oversubscription: you have to choose

who will benefit from the program.– Instead of choosing the beneficiaries on criteria

(proximity, need), you choose them randomly

• Is it fair to allocate the program randomly?– Partial lottery– Pilot before full-scale implementation

5

How to randomize?

• Phase-in design: takes advantage of expansion– You may serve everybody, but not all at once– Choose the first people served randomly– The others will be the comparison, until they

get served

6

Adapted from How to Randomize, Rachel Glennerster, J-PAL.

Phase-in Design

7

Adapted from How to Randomize, Rachel Glennerster, J-PAL.

How to randomize?

• Rotation design– You may serve half of the people, but don’t

want to leave people out– Group A gets the program on Year 1, Group B

on Year 2– Group B is control in Year 1, Group B in Year 2

8

Rotation design

9

Adapted from How to Randomize, Rachel Glennerster, J-PAL.

How to randomize?

10

• Encouragement design: What to do when you can’t randomize access- Sometimes practically or ethically impossible to

randomize program access- But most programs have less than 100% take

up- Randomize encouragement to receive

treatment

Adapted from How to Randomize, Rachel Glennerster, J-PAL.

Encouragement Design

11

Adapted from How to Randomize, Rachel Glennerster, J-PAL.

Key steps in conducting an experiment

1. Understand the program in depth (Theory of Change)

2. Design the study carefully

3. Randomly assign people to treatment or control

4. Collect baseline data (→ panel)

5. Verify that assignment looks random (← small sample)

6. Monitor process so that integrity of experiment is not compromised

12

Key steps in conducting an experiment (cont.)

7. Collect follow-up data for both the treatment and control groups in identical ways.

8. Estimate program impacts by comparing mean outcomes of treatment group vs. mean outcomes of control group.

9. Assess whether program impacts are statistically significant and practically significant.

13