Introduction to Biostatistics for Clinical and Translational

Researchers

KUMC Departments of Biostatistics & Internal MedicineUniversity of Kansas Cancer Center

FRONTIERS: The Heartland Institute of Clinical and Translational Research

Course InformationJo A. Wick, PhD

Office Location: 5028 RobinsonEmail: jwick@kumc.edu

Lectures are recorded and posted at http://biostatistics.kumc.edu under ‘Events & Lectures’

ObjectivesUnderstand the role of statistics in the scientific

process and how it is a core component of evidence-based medicine

Understand features, strengths and limitations of descriptive, observational and experimental studies

Distinguish between association and causationUnderstand roles of chance, bias and

confounding in the evaluation of research

Course CalendarJuly 5: Introduction to Statistics: Core ConceptsJuly 12: Quality of Evidence: Considerations for

Design of Experiments and Evaluation of LiteratureJuly 19: Hypothesis Testing & Application of

Concepts to Common Clinical Research QuestionsJuly 26: (Cont.) Hypothesis Testing & Application

of Concepts to Common Clinical Research Questions

Why is there conflicting evidence?Answer: There is no perfect research study.

Every study has limitations.Every study has context.

Medicine (and research!) is an art as well as a science.

Unfortunately, the literature is full of poorly designed, poorly executed and improperly interpreted studies—it is up to you, the consumer, to critically evaluate its merit.

Critical Evaluation

Validity and RelevanceIs the article from a peer-reviewed journal?How does the location of the study reflect the

larger context of the population?Does the sample reflect the targeted population?Is the study sponsored by an organization that

may influence the study design or results?Is the intervention feasible? available?

Critical EvaluationIntentTherapy: testing the efficacy of drug treatments, surgical

procedures, alternative methods of delivery, etc. (RCT)Diagnosis: demonstrating whether a new diagnostic test

is valid (Cross-sectional survey)Screening: demonstrating the value of tests which can

be applied to large populations and which pick up disease at a presymptomatic stage (Cross-sectional survey)

Prognosis: determining what is likely to happen to someone whose disease is picked up at an early stage (Longitudinal cohort)

Critical EvaluationCausation: determining whether a harmful agent

is related to development of illness (Cohort or case-control)

Critical EvaluationValidity based on intentWhat is the study design? Is it appropriate and optimal

for the intent?Are all participants who entered the trial accounted for

in the conclusion?What protections against bias were put into place?

Blinding? Controls? Randomization?If there were treatment groups, were the groups similar

at the start of the trial?Were the groups treated equally (aside from the actual

intervention)?

Critical EvaluationIf statistically significant, are the results clinically

meaningful?If negative, was the study powered prior to

execution?Were there other factors not accounted for that

could have affected the outcome?

Miser, WF Primary Care 2006.

Experimental DesignStatistical analysis, no matter how intricate, cannot

rescue a poorly designed study.No matter how efficient, statistical analysis cannot

be done overnight.A researcher should plan and state what they are

going to do, do it, and then report those results.Be transparent!

Types of SamplesRandom sample: each

person has equal chance of being selected.

Convenience sample: persons are selected

The principal way to guarantee that the sample

population sample

because they are convenient or readily available.Systematic sample: persons selected based

on a pattern.Stratified sample: persons selected from

within subgroup.

Random SamplingFor studies, it is optimal (but not always possible) for

the sample providing the data to be representative of the population under study.

Simple random sampling provides a representative sample (theoretically) and protections against selection bias.A sampling scheme in which every possible sub-sample of

size n from a population is equally likely to be selectedAssuming the sample is representative, the summary

statistics (e.g., mean) should be ‘good’ estimates of the true quantities in the population.• The larger n is, the better estimates will be.

Random SamplesThe Fundamental Rule of Using Data for

Inference requires the use of random sampling or random assignment.

Random sampling or random assignment ensures control over “nuisance” variables.

We can randomly select individuals to ensure that the population is well-represented.Equal sampling of males and femalesEqual sampling from a range of agesEqual sampling from a range of BMI, weight, etc.

Random SamplesRandomly assigning subjects to treatment levels to

ensure that the levels differ only by the treatment administered.weightsagesrisk factors

Nuisance VariationNuisance variation is any undesired sources of

variation that affect the outcome.Can systematically distort results in a particular direction

—referred to as bias.Can increase the variability of the outcome being

measured—results in a less powerful test because of too much ‘noise’ in the data.

Example: Albino RatsIt is hypothesized that exposing albino rats to

microwave radiation will decrease their food consumption.

Intervention: exposure to radiationLevels exposure or non-exposureLevels 0, 20000, 40000, 60000 uW

Measurable outcome: amount of food consumedPossible nuisance variables: sex, weight,

temperature, previous feeding experiences

Experimental DesignTypes of data collected in a clinical trial:

Treatment – the patient’s assigned treatment and actual treatment received

Response – measures of the patient’s response to treatment including side-effects

Prognostic factors (covariates) – details of the patient’s initial condition and previous history upon entry into the trial

Experimental DesignThree basic types of outcome data:

Qualitative – nominal or ordinal, success/failure, CR, PR, Stable disease, Progression of disease

Quantitative – interval or ratio, raw score, difference, ratio, %

Time to event – survival or disease-free time, etc.

Experimental DesignFormulate statistical hypotheses that are germane

to the scientific hypothesis.Determine:

experimental conditions to be used (independent variable(s))

measurements to be recordedextraneous conditions to be controlled (nuisance

variables)

Experimental DesignSpecify the number of subjects required and the

population from which they will be sampled.Power, Type I & II errors

Specify the procedure for assigning subjects to the experimental conditions.

Determine the statistical analysis that will be performed.

Experimental DesignConsiderations:

Does the design permit the calculation of a valid estimate of treatment effect?

Does the data-collection procedure produce reliable results?

Does the design possess sufficient power to permit and adequate test of the hypotheses?

Experimental DesignConsiderations:

Does the design provide maximum efficiency within the constraints imposed by the experimental situation?

Does the experimental procedure conform to accepted practices and procedures used in the research area?• Facilitates comparison of findings with the results of other

investigations

Types of StudiesPurpose of research

1) To explore2) To describe or classify3) To establish relationships4) To establish causality

Strategies for accomplishing these purposes:1) Naturalistic observation2) Case study3) Survey4) Quasi-experiment5) Experiment

Ambiguity C

ontrol

Generating Evidence

Studies

Descriptive Studies

Populations Individuals

Case Reports

Case Series

Cross Sectiona

l

Analytic Studies

Observational

Case Control Cohort

Experimental

RCT

Complexity and Confidence

Observation versus ExperimentA designed experiment involves the investigator

assigning (preferably randomly) some or all conditions to subjects.

An observational study includes conditions that are observed, not assigned.

Example: Heart StudyQuestion: How does serum total cholesterol vary

by age, gender, education, and use of blood pressure medication? Does smoking affect any of the associations?

Recruit n = 3000 subjects over two yearsTake blood samples and have subjects answer a

CVD risk factor surveyOutcome: Serum total cholesterolFactors: BP meds (observed, not assigned)Confounders?

Example: DiabetesQuestion: Will a new treatment help overweight

people with diabetes lose weight?N = 40 obese adults with Type II (non-insulin

dependent) diabetes (20 female/20 male)Randomized, double-blind, placebo-controlled

study of treatment versus placeboOutcome: Weight lossFactor: Treatment versus placebo

Cross-Sectional StudiesDesigned to assess the association between an

independent variable (exposure?) and a dependent variable (disease?)

Selection of study subjects is based on both their exposure and outcome status, thus there is no direction of inquiry

Cross-Sectional StudiesD

efin

ed P

opul

atio

n

Gather data on Exposure & Disease

ExposedDiseased

ExposedNo Disease

Not ExposedDiseased

Not ExposedNo Disease

Cross-Sectional StudiesCannot determine causal relationships between

exposure and outcomeCannot determine temporal relationship between

exposure and outcome

“Exposure is associated with Disease”

“Exposure causes Disease”

“Disease follows Exposure”

Analysis of Cross-Sectional Data

Disease+ -

Exposure

+ a b

- c d

Prevalence of disease compared in exposed versus non-exposed groups:

( ) a|p D Ea b

+ + = +

( )| cp D Ec d

+ - = +

Prevalence of exposure compared in diseased versus non-diseased groups:

( ) a| |p E Da c

+ + = +

( )| bp E Db d

+ - = +

Case-Control StudiesDesigned to assess the association between

disease and past exposuresSelection of study subjects is based on their

disease statusDirection of inquiry is backward

Case-Control Studies

Def

ined

P

opul

atio

nGather data on

Disease

Diseased

Unexposed

Exposed

No Disease

Unexposed

Exposed

Time

Direction of Inquiry

Analysis of Case-Control DataDisease

+ - TotalExposure

+ a b a+b

- c d c+d

Total a+c b+d

Odds ratio: odds of case exposure . odds of control exposure

aadcOR b bc

d

= =

Cohort StudiesDesigned to assess the association between

exposures and disease occurrenceSelection of study subjects is based on their

exposure statusDirection of inquiry is forward

Cohort StudiesD

efin

ed P

opul

atio

n

Gather data on Exposure

ExposedDisease

No Disease

Not ExposedDisease

No Disease

Time

Direction of Inquiry

Cohort StudiesAttrition or loss to follow-upTime and money!Inefficient for very rare outcomesBias

Outcome ascertainmentInformation biasNon-response bias

Analysis of Cohort Data

Disease+ - Total

Exposur

e

+ a b a+b

- c d c+d

Total a+c b+d

Relative Risk: risk of disease in exposed . risk of disease in unexposed

aa bRR c

c d

+=+

Randomized Controlled TrialsDesigned to test the association between

exposures and diseaseSelection of study subjects is based on their

assigned exposure statusDirection of inquiry is forward

Randomized Controlled TrialsD

efin

ed P

opul

atio

n

Randomize to Exposure

Exposed (Treated)

Disease

No Disease

Not Exposed (Control)

Disease

No Disease

Time

Direction of Inquiry

Why do we randomize?Suppose we wish to compare surgery for CAD to

a drug used to treat CAD. We know that such major heart surgery is invasive and complex—some people die during surgery. We may assign the patients with less severe CAD (on purpose or not) to the surgery group.If we see a difference in patient survival, is it due to

surgery versus drugs or to less severe disease versus more severe disease?

Such a study would be inconclusive and a waste of time, money and patients.

How could we fix it?Randomize!

Randomization is critical because there is no way for a researcher to be aware of all possible confounders.

Observational studies have little to no formal control for any confounders—thus we cannot conclude cause and effect based on their results.

Randomization forms the basis of inference.

Other Protections Against BiasBlinding

Single (patient only), double (patient and evaluator), and triple (patient, evaluator, statistician) blinding is possible

Eliminates biases that can arise from knowledge of treatment

ControlNull (no treatment), placebo (no active treatment), active

(current standard of care) controls are usedEliminates biases that can arise from the natural

progression of disease (null control) or simply from the act of being treated (placebo)

Analysis of RCT Data What kind of outcome do you have?

Continuous? Categorical?How many samples (groups) do you have?

Are they related or independent?

Types of TestsParametric methods: make assumptions about the

distribution of the data (e.g., normally distributed) and are suited for sample sizes large enough to assess whether the distributional assumption is met

Nonparametric methods: make no assumptions about the distribution of the data and are suitable for small sample sizes or large samples where parametric assumptions are violatedUse ranks of the data values rather than actual data values

themselvesLoss of power when parametric test is appropriate

Two independent percentages? Fisher’s Exact test, chi-square test, logistic regression

Two independent means? Mann-Whitney, Two-sample t-test, analysis of variance, linear regression

Two independent time-to-event outcomes? Log-rank test, Wilcoxon test, Cox regression

Any adjustments for other prognostic factors can be accomplished with the appropriate regression models (e.g., logistic for yes/no outcomes, linear for continuous, Cox for time-to)

Analysis of RCT Data

Threats to Valid InferenceStatistical Conclusion Validity

• Low statistical power - failing to reject a false hypothesis because of inadequate sample size, irrelevant sources of variation that are not controlled, or the use of inefficient test statistics.

• Violated assumptions - test statistics have been derived conditioned on the truth of certain assumptions. If their tenability is questionable, incorrect inferences may result.

Many methods are based on approximations to a normal distribution or another probability distribution that becomes more accurate as sample size increases—using these methods for small sample sizes may produce unreliable results.

Threats to Valid InferenceStatistical Conclusion Validity

Reliability of measures and treatment implementation.Random variation in the experimental setting and/or

subjects.• Inflation of variability may result in not rejecting a false hypothesis

(loss of power).

Threats to Valid InferenceInternal Validity

Uncontrolled events - events other than the administration of treatment that occur between the time the treatment is assigned and the time the outcome is measured.

The passing of time - processes not related to treatment that occur simply as a function of the passage of time that may affect the outcome.

Threats to Valid InferenceInternal Validity

Instrumentation - changes in the calibration of a measuring instrument, the use of more than one instrument, shifts in subjective criteria used by observers, etc.• The “John Henry” effect - compensatory rivalry by subjects

receiving less desirable treatments.• The “placebo” effect - a subject behaves in a manner consistent

with his or her expectations.

Threats to Valid InferenceExternal Validity—Generalizability

Reactive arrangements - subjects who are aware that they are being observed may behave differently that subjects who are not aware.

Interaction of testing and treatment - pretests may sensitize subjects to a topic and enhance the effectiveness of a treatment.

Threats to Valid InferenceExternal Validity—Generalizability

Self-selection - the results may only generalize to volunteer populations.

Interaction of setting and treatment - results obtained in a clinical setting may not generalize to the outside world.

Clinical Trials—PurposePrevention trials look for more effective/safer

ways to prevent a disease in individuals who have never had it, or to prevent a disease from recurring in individuals who have.

Screening trials attempt to identify the best methods for detecting diseases or health conditions.

Diagnostic trials are conducted to distinguish better tests or procedures for diagnosing a particular disease or condition.

Clinical Trials—PurposeTreatment trials assess experimental treatments,

new combinations of drugs, or new approaches to surgery or radiation therapy for efficacy and safety.

Quality of life (supportive care) trials explore means to improve comfort and quality of life for individuals with chronic illness.

Classification according to the U.S. National Institutes of Health

Clinical Trials—PhasesPre-clinical studies involve in vivo and in vitro

testing of promising compounds to obtain preliminary efficacy, toxicity, and pharmacokinetic information to assist in making decisions about future studies in humans.

Clinical Trials—PhasesPhase 0 studies are exploratory, first-in-human

trials, that are designed to establish very early on whether the drug behaves in human subjects as was anticipated from preclinical studies.Typically utilizes N = 10 to 15 subjects to assess

pharmacokinetics and pharmacodynamics.Allows the go/no-go decision usually made from animal

studies to be based on preliminary human data.

Clinical Trials—PhasesPhase I studies assess the safety, tolerability,

pharmacokinetics, and pharmacodynamics of a drug in healthy volunteers (industry standard) or patients (academic/research standard).Involves dose-escalation studies which attempt to identify

an appropriate therapeutic dose.Utilizes small samples, typically N = 20 to 80 subjects.

Clinical Trials—PhasesPhase II studies assess the efficacy of the drug

and continue the safety assessments from phase I.Larger groups are usually used, N = 20 to 300.Their purpose is to confirm efficacy (i.e., estimation of

effect), not necessarily to compare experimental drug to placebo or active comparator.

Clinical Trials—PhasesPhase III studies are the definitive assessment of

a drug’s effectiveness and safety in comparison with the current gold standard treatment.Much larger sample sizes are utilized, N = 300 to 3,000,

and multiple sites can be used to recruit patients.Because they are quite an investment, they are usually

randomized, controlled studies.

Clinical Trials—Phases Phase IV studies are also known as post-

marketing surveillance trials and involve the ongoing or long-term assessment of safety in drugs that have been approved for human use.Detect any rare or long-term adverse effects in a much

broader patient population

The Size of a Clinical TrialLasagna’s Law

Once a clinical trial has started, the number of suitable patients dwindles to a tenth of what was calculated before the trial began.

The Size of a Clinical Trial“How many patients do we need?”Statistical methods can be used to determine the

required number of patients to meet the trial’s principal scientific objectives.

Other considerations that must be accounted for include availability of patients and resources and the ethical need to prevent any patient from receiving inferior treatment.We want the minimum number of patients required to

achieve our principal scientific objective.

The Size of a Clinical TrialEstimation trials involve the use of point and

interval estimates to describe an outcome of interest.

Hypothesis testing is typically used to detect a difference between competing treatments.

The Size of a Clinical TrialType I error rate (α): the risk of concluding a

significant difference exists between treatments when the treatments are actually equally effective.

Type II error rate (β): the risk of concluding no significant difference exists between treatments when the treatments are actually different.

The Size of a Clinical TrialPower (1 – β): the probability of correctly detecting

a difference between treatments—more commonly referred to as the power of the test.

Truth

Conclusion H1 H0

H1 1 – β αH0 β 1 – α

The Size of a Clinical TrialSetting three determines the fourth:

For the chosen level of significance (α), a clinically meaningful difference (δ) can be detected with a minimally acceptable power (1 – β) with n subjects.

Depending on the nature of the outcome, the same applies: For the chosen level of significance (α), an outcome can be estimated within a specified margin of error (ME) with n subjects.

Example: Detecting a DifferenceThe Anturane Reinfarction Trial Research Group

(1978) describe the design of a randomized double-blind trial comparing anturan and placebo in patients after a myocardial infarction.What is the main purpose of the trial?What is the principal measure of patient outcome?How will the data be analyzed to detect a treatment

difference?What type of results does one anticipate with standard

treatment?How small a treatment difference is it important to detect and

with what degree of uncertainty?

Example: Detecting a DifferencePrimary objective: To see if anturan is of value in

preventing mortality after a myocardial infarction.Primary outcome: Treatment failure is indicated by

death within one year of first treatment (0/1).Data analysis: Comparison of percentages of

patients dying within first year on anturan (π1) versus placebo (π2) using a χ2 test at the α = 0.05 level of significance.

Example: Detecting a DifferenceExpected results under placebo: One would

expect about 10% of patients to die within a year (i.e., π2 = .1).

Difference to detect (δ): It is clinically interesting to be able to determine if anturan can halve the mortality—i.e., 5% of patients die within a year—and we would like to be 90% sure that we detect this difference as statistically significant.

Example: Detecting a DifferenceWe have:

H0: π1 = π2 versus H1: π1 π2 (two-sided test) α = 0.051 – β = 0.90δ = π2 – π1 = 0.05

The estimate of power for this test is a function of sample size:

2 2

1 1 1 2 2 2 1 1 1 2 2 2

1z SE z SE

P z P zp q n p q n p q n p q n

- - - - +

+ +

n = 583 patients per group is required

Example: Detecting a Difference

-zα/2 zα/2

Fail to reject H0

Conclude no difference

Reject H0

Conclude differenceReject H0

Conclude difference

α/2 α/21 - α

β 1 - β

Power and Sample Sizen is roughly inversely proportional to δ2; for fixed α and

β, halving the difference in rates requiring detection results in a fourfold increase in sample size.

n depends on the choice of β such that an increase in power from 0.5 to 0.95 requires around 3 times the number of patients.

Reducing α from 0.05 to 0.01 results in an increase in sample size of around 40% when β is around 10%.

Using a one-sided test reduces the required sample size.

Example: Detecting a DifferencePrimary objective: To see if treatment A increases

outcome W.Primary outcome: The primary outcome, W, is

continuous.Data analysis: Comparison of mean response of

patients on treatment A (μ1) versus placebo (μ2) using a two-sided t-test at the α = 0.05 level of significance.

Example: Detecting a DifferenceExpected results under placebo: One would

expect a mean response of 10 (i.e., μ2 = 10).Difference to detect (δ): It is clinically interesting to

be able to determine if treatment A can increase response by 10%—i.e., we would like to see a mean response of 11 (10 + 1) in patients getting treatment A and we would like to be 80% sure that we detect this difference as statistically significant.

Example: Detecting a DifferenceWe have:

H0: μ1 = μ2 versus H1: μ1 μ2 (two-sided test) α = 0.051 – β = 0.80δ = 1

For continuous outcomes we need to determine what difference would be clinically meaningful, but specified in the form of an effect size which takes into account the variability of the data.

Example: Detecting a DifferenceEffect size is the difference in the means divided

by the standard deviation, usually of the control or comparison group, or the pooled standard deviation of the two groups

where

1 2d -

2 21 2

1 2n n

+

Example: Detecting a Difference

-zα/2 zα/2

Fail to reject H0

Conclude no difference

Reject H0

Conclude differenceReject H0

Conclude difference

α/2 α/21 - α

β 1 - β

Example: Detecting a DifferencePower Calculations an interesting interactive web-

based tool to show the relationship between power and the sample size, variability, and difference to detect.

A decrease in the variability of the data results in an increase in power for a given sample size.

An increase in the effect size results in a decrease in the required sample size to achieve a given power.

Increasing α results in an increase in the required sample size to achieve a given power.

Prognostic FactorsIt is reasonable and sometimes essential to collect

information of personal characteristics and past history at baseline when enrolling patient’s onto a clinical trial.

These variables allow us to determine how generalizable the results are.

Prognostic FactorsPrognostic factors known to be related to the

desired outcome of the clinical trial must be collected and in some cases randomization should be stratified upon these variables.

Many baseline characteristics may not be known to be related to outcome, but may be associated with outcome for a given trial.

Comparable Treatment GroupsAll baseline prognostic and descriptive factors of

interest should be summarized between the treatment groups to insure that they are comparable between treatments. It is generally recommended that these be descriptive comparisons only, not inferential

Note: Just because a factor is balanced does not mean it will not affect outcome and vice versa.

Subgroup AnalysisDoes response differ for differing types of patients?

This is a natural question to ask. To answer this question one should test to see if the

factor that determines type of patient interacts with treatment.

Separate significance tests for different subgroups do not provide direct evidence of whether a prognostic factor affects the treatment difference: a test for interaction is much more valid.

Tests for interactions may also be designed a priori.

Multiplicity of DataMultiple Treatments – the number of possible

treatment comparisons increases rapidly with the number of treatments. (Newman-Keuls, Tukey’s HSD or other adjustment should be designed)

Multiple end-points – there may be multiple ways to evaluate how a patient responds. (Bonferroni adjustment, Multivariate test, combined score, or reduce number of primary end-points)

Multiplicity of DataRepeated Measurements – patient’s progress may

be recorded at several fixed time points after the start of treatment. One should aim for a single summary measure for each patient outcome so that only one significance test is necessary.

Subgroup Analyses – patients may be grouped into subgroups and each subgroup may be analyzed separately.

Interim Analyses – repeated interim analyses may be performed after accumulating data while the trial is in progress.

SummaryStatistics plays a key role in pre-clinical and clinical

researchStatistics helps us determine how ‘confident’ we

should be in the results of a studyConfidence in a study is based on (1) the size of

the study, (2) its safeguards against biases (complexity), (3) how it was actually undertaken

Statistical support is available and should be sought out as early as possible in the process of designing a study

Next Time . . . Basic Descriptive and Inferential MethodsHypothesis Testing

P-valuesConfidence IntervalsInterpretation

Examples