SJS SDI_1 1

Design of Statistical Investigations

Stephen Senn

1 General Introduction

SJS SDI_1 2

Course Outline

• General Introduction

• Experiments

• Observational studies

• Sample surveys (and other sampling schemes)

NB Each of these fields is huge and all that is attempted is a brief introduction

SJS SDI_1 3

General Warning

• Your lecturer is not equally experienced in these fields

• I know more about experimental design than the other two

• Examples from my personal experience tend to be drawn from pharmaceutical research and development or other medical applications

SJS SDI_1 4

Example Exp_1A Simple Experiment

• Four experimental p38 kinase inhibitors

• Vehicle and marketed product as controls

• Thrombaxane B2 (TXB2) is used as a marker of COX-1 activity (low values bad)

• Six rats per group were treated for a total of 36 rats

• At the end of the study rats are sacrificed and TXB2 is measured.

SJS SDI_1 5

a vehicle b marketed c d

Treat

0

100

200

300

txb2

Thrombaxane B2 (TXB2) inhibitionFour experimental p38 kinase inhibitors

SJS SDI_1 6

Specific Features of this Design• Several experimental treatments

• Two controls– active– neutral

• Six replicates per treatment

• Several tests compounds– no ordering

• No blocks– rats considered exchangeable

The meaning and relevance of these terms will be explained during the course

SJS SDI_1 7

Example Obs_1An Observational Study

• Case-control study (Fine et al Lancet, 1986, Quoted in Clayton and Hills)

• Does BCG protect against leprosy?

• BCG scar status in a population survey were available

• Data from 260 leprosy cases were obtained

SJS SDI_1 8

Fine et al

BCG Scar LeprosyCases

PopulationSurvey

TOTAL

Present 101 46028 46129

Absent 159 34594 34753

TOTAL 260 80622 80882

SJS SDI_1 9

Case-Control Study• Note that this is sampled by outcome

• The number of these is fixed

• Exposure is measured

• In a clinical trial, patients are assigned the exposure (the treatment)

• The outcome is measured

• An experiment involves manipulation

• Case-control does not

SJS SDI_1 10

Example Surv_1A Sample Survey

• Population of pharmaceutical record forms in Pembury Hospital, Tunbridge Wells

• Thousands of such forms available• A sample of 108 forms was chosen from patients

discharged between 1 July and 31 December 1976– Records chosen at fixed intervals

• Number of prescriptions recorded on each was noted

SJS SDI_1 11

0

10

20

30

0 2 4 6 8 10

number of prescriptions

fre

qu

en

cy

Distribution of Number of Prescriptions per Form

SJS SDI_1 12

Purposes of Statistical Investigations

Past/Present Future

Causal InferenceExperimentsObservational

Decision- making

DescriptionSurveys

Prediction

SJS SDI_1 13

Problem to Bear in Mind

• We can only study past/present

• We can construct formal theories of inference only about the past/present

• We often wish to make inference about the future

• This requires an ‘extra-statistical’ element– Most naively an assumption that the future is

like the past

SJS SDI_1 14

Example

• The effect of streptomycin on TB

• Trial carried out by Austin Bradford Hill and colleagues 1947

• Treatment highly effective

• Is it still as effective?

SJS SDI_1 15

Experimentation v Sampling

Experiments• Causal purpose• Convenient material• Allocation of

treatments crucial– Randomisation

Sampling• Descriptive purpose• Representative

material• Choice of sample

members crucial– Random sampling

SJS SDI_1 16

Caution

• These two are sometimes confused

• The growth of modelling approaches tends to increase the confusion

• Experiments rarely use representative material

• Surveys (and other samples) usually do.

SJS SDI_1 17

Basic Design CycleObjective

Tentative Design

Potential Data

Possible Analysis

Possible Conclusions

Relevant factors

SJS SDI_1 18

Questions 1Exp_1 Rat TXB2

• How do you decide which rat gets which treatment?

• How would you analyse these data?

• What use will be made of these data?

SJS SDI_1 19

Questions 2Obs_1 Fine et al

• What difference would it make to the precision of the conclusions if the population survey had been smaller?

• What difference would it make if there had been fewer leprosy cases?

• How would you test for an association between BCG and leprosy?

• What interpretations are there for an association?

SJS SDI_1 20

Questions 3Surv_1 Pharmaceutical Record Forms

• What is a simple random sample?

• In this specific case how would one choose such a sample?*

• Suppose that the sample of 108 forms was chosen from 5,000. What should the size of the sample have to be if there were 10,000 to choose from?

* The sample chosen in this example was not a simple random sample

SJS SDI_1 21

Suggested ReadingExperimental Design: Mead, R. The Design of Experiments, Cambridge University Press, Cambridge, 1988

Clarke, G.M and Kempson, R.E.Introduction to the Design and Analysis of Experiments, Arnold, London, 1997.

Case-control Studies, Breslow and Day, 1980, Statistical Method in Cancer Research, vol 1

SamplingHague and Harris, Sampling and Statistics, Kogan Page (This is a very elementary book.)

S-PLUSKrause, A and Olson, M The Basics of S and S-PLUS (2nd edition) , Springer, 2000