Download - Clinical Trial Statstics 2016
![Page 1: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/1.jpg)
EBM: Clinical Trial Statistics
Stefan Tigges MD MSCRDepartment of Radiology, Emory
1
![Page 2: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/2.jpg)
2
External Industry External Industry
Relationships Relationships **Company Name(s) Company Name(s) Role Role
Equity, stock, or options in Equity, stock, or options in biomedical industry companies biomedical industry companies or publishersor publishers****
General Electric and Microsoft Stockholder
Board of Directors or officerBoard of Directors or officer None
Royalties from Emory or from Royalties from Emory or from external entityexternal entity
None
Industry funds to Emory for my Industry funds to Emory for my research research
None
OtherOther None
*Consulting, scientific advisory board, industry-sponsored CME, expert witness for company, FDA representative for company, publishing contract, etc.
**Does not include stock in publicly-traded companies in retirement funds and other pooled investment accounts managed by others.
Stefan Tigges, Personal/Professional Financial Relationships with Industry within the past year
![Page 3: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/3.jpg)
Lecture/Reading Goals and Objectives• Define: probability, distribution, variability and
central tendency.• Explain how a sample may be biased and the
difference between bias and random sampling error.• Explain how and why hypothesis testing and
statistical inference are used in clinical trial analysis. • Define: confidence intervals, statistical significance,
type I error, type II error, power, p-value, alpha and beta.
• Describe the effect of increasing sample size on type I and type II error.
• Describe the effect of sample size, effect variability, level of alpha, and effect size on power.
3
![Page 4: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/4.jpg)
Learning Approach
1) Readings, 2 Comix2) Lecture3) Homework
1) Optional2) Based on student ?s3) Hard
4) E-mail me
4
![Page 5: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/5.jpg)
Big Question: Approach to Claims
5
![Page 6: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/6.jpg)
Three Explanations• Truth
• Dumb luck
• Fishy
6
![Page 7: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/7.jpg)
Antihypertensive Trial: Result is Positive
7No Effect−10−20−30 +10 +20 +30
Explanations:1)Real Effect: HA true2)FP: Random Error (α)3)FP: Bias
NewDrug
OldDrug
![Page 8: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/8.jpg)
FP Alpha (random) error
8
![Page 9: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/9.jpg)
Antihypertensive Trial: Result is Negative
9No Effect−10−20−30 +10 +20 +30
Explanations:1)Real Effect: H0 true2)FN: Random Error (β)3)FN: Bias
NewDrug
OldDrug
![Page 10: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/10.jpg)
FN Beta error, effect exists, not detected
10
![Page 11: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/11.jpg)
Diagnostic Tests, 2x2 Table
11
TestFinding
Disease Positive
Disease Negativ
ePositive TP FP → PPVNegativ
eFN TN → NPV
↓Sensitiv
ity
↓Specific
ity
Total
![Page 12: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/12.jpg)
Clinical Research, 2x2 Table
12
TrialResult
HA True
HA
FalsePositive TP FP (α)
Type I→ PPV
Negative
FN (β) Type II
TN → NPV
↓Power
↓*p value
Total
![Page 13: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/13.jpg)
Randomized Clinical Trial Steps
13
Populationof interest
Sample
DrugA
DrugB Time
Drug A
∆ BPDrug
B∆ BP
Time Compare,publish,acceptNobelprize
H0: A=BHA: A≠B
Bias vs.random
error
![Page 14: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/14.jpg)
Bias: Systematic
errors in data collection &
interpretation
14
![Page 15: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/15.jpg)
15
![Page 16: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/16.jpg)
16
Voters
Sample
![Page 17: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/17.jpg)
17
$ $
$
$
$
$
$
$
$
$
![Page 18: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/18.jpg)
Types of Statistics• Descriptive– Summarize/display data– Mean, median, mode, σ etc.
• Inferential– Use sample to make
conclusions about population – Example: Hypertension• Test population: all w/ ↑ BP
– Definitive, descriptive stats only
• Test sample– Hypothesis testing– P(Observed results given H0)
13%
17%
57%
13%
1st Qtr2nd Qtr3rd Qtr4th Qtr
18
Population:all w/↑ BP
Sample
![Page 19: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/19.jpg)
Hypothesis Testing: Is H0 plausible? EUSM vs. NBA Mean Height
19
H 0: Expecte
d
Trial: Observed
When you stare into the abyss [of statistics], the abyss stares back into you.
Statistics: P(O given H0)
p<.05reject H0
p≥.05, cannotreject H0190
H0:μEUSM=μNBA(190)HA:μEUSM≠μNBA (190)
170
![Page 20: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/20.jpg)
Determining p value: Normal Distribution
20
0 1 2−2 −1
Central Tendency:Mean, median and mode
Dispersion:Standard deviation
√Σ(x-μ)2/N68%
95%
![Page 21: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/21.jpg)
Normal Distribution: EUSM M1 Height
21
170
σEUSM=10 cm
![Page 22: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/22.jpg)
Population: EUSM M1 Heights (cm)
22
170 180150 160 190
EUSMEUSMClass of ‘18Class of ‘18
![Page 23: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/23.jpg)
Number of σs from mean is probability
23
3σ from mean, p=.0027
140 cm
![Page 24: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/24.jpg)
Example: Heights
24
170 cm 190 cm160 cm
μ= 160, 170,190σ=10, α=.05
ie, 2σ
![Page 25: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/25.jpg)
Example 1: EUSM M-1s vs. NBA Heights• Is mean height of EUSM M-1s different than mean
height of NBA players? • H0:μEUSM=μNBA (190 cm) with σ=10 cm • HA:μEUSM≠μNBA(190 cm) with σ=10 cm• 25 M-1 heights, mean=170 cm, ∆=20 cm• SEM= σ/√n=10/√ 25=2• 20/2= 10 σ, p<.0001• Reject H0 at α of .05• α predetermined for H0 rejection
25
Observe: 170
Expec
t: 190
![Page 26: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/26.jpg)
M-1 vs. NBA Heights: H0 is False (TP)
26
160
190
170
190
170
190
150
160
170
150
180
170
170
180
160
160
180
180
160
150
150
170
160
170
180
180
150
160
180
190
170
190
180
150
190
180
150
170
<150
170
160
170
170
160
170
160
160
170
150
180
150
180
150
>190
190
170
170
170
170
170
<150
170
180
170
180
180
170
170
150
170
180
160
170
160
170
190
160
170
190
160
160
190
180
180
160
170
170
160
150
>190
190
170
>190
190
150
180
180
160
190
150
> Mean170 cmMean < Mean
170↓20
0 1 2−2 −1
![Page 27: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/27.jpg)
Example 2: Heights
27
170 cm 190 cm160 cm
![Page 28: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/28.jpg)
Example 2: EUSM M-1s vs. Brand X M-1s• Is mean height of EUSM M-1s different than mean
height of M-1s at Brand X medical school? • H0:μEUSM=μBrand X (170 cm) with σ=10 cm • HA:μEUSM≠μBrand X(170 cm) with σ=10 cm• 25 M-1 heights, mean=170 cm, ∆=0 cm• SEM= σ/√n=10/√ 25=2• 0/2= 0 σ, p=1• Don’t reject H0
28
Observe: 170
Expec
t: 170
![Page 29: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/29.jpg)
M-1 vs. Brand X Heights: H0 is True (TN)
29
160
190
170
190
170
190
150
160
170
150
180
170
170
180
160
160
180
180
160
150
150
170
160
170
180
180
150
160
180
190
170
190
180
150
190
180
150
170
<150
170
160
170
170
160
170
160
160
170
150
180
150
180
150
>190
190
170
170
170
170
170
<150
170
180
170
180
180
170
170
150
170
180
160
170
160
170
190
160
170
190
160
160
190
180
180
160
170
170
160
150
>190
190
170
>190
190
150
180
180
160
190
150
> Mean170 cmMean < Mean
170
0 1 2−2 −1
![Page 30: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/30.jpg)
M-1 vs. Brand X Heights: Type I error (FP)
30
160
190
170
190
170
190
150
160
170
150
180
170
170
180
160
160
180
180
160
150
150
170
160
170
180
180
150
160
180
190
170
190
180
150
190
180
150
170
<150
170
160
170
170
160
170
160
160
170
150
180
150
180
150
>190
190
170
170
170
170
170
<150
170
180
170
180
180
170
170
150
170
180
160
170
160
170
190
160
170
190
160
160
190
180
180
160
170
170
160
150
>190
190
170
>190
190
150
180
180
160
190
150
> Mean170 cmMean < Mean
180↑10
0 1 2−2 −1
![Page 31: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/31.jpg)
Example 2: Heights
31
Bran
d X
![Page 32: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/32.jpg)
Example 3: Heights
32
170 cm 190 cm160 cm
![Page 33: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/33.jpg)
Example 3: EUSM M-1s vs. Jockeys• Is mean height of EUSM M-1s different than mean
height of Jockeys? • H0:μEUSM=μJockey (160 cm) with σ=10 cm • HA:μEUSM≠μJockey(160 cm) with σ=10 cm• 25 M-1 heights, mean=170 cm, ∆=10 cm• SEM= σ/√n=10/√ 25=2• 10/2= 5 σ, p=.0062• Reject H0 at α of .05
33
Obs
erve
: 170Exp
ect:
160
![Page 34: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/34.jpg)
M-1 vs. Jockey Heights: HA is True (TP)
34
160
190
170
190
170
190
150
160
170
150
180
170
170
180
160
160
180
180
160
150
150
170
160
170
180
180
150
160
180
190
170
190
180
150
190
180
150
170
<150
170
160
170
170
160
170
160
160
170
150
180
150
180
150
>190
190
170
170
170
170
170
<150
170
180
170
180
180
170
170
150
170
180
160
170
160
170
190
160
170
190
160
160
190
180
180
160
170
170
160
150
>190
190
170
>190
190
150
180
180
160
190
150
> Mean170 cmMean < Mean
170↑10
0 1 2−2 −1
![Page 35: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/35.jpg)
M-1 vs. Jockey Heights: Type II Error (FN)
35
160
190
170
190
170
190
150
160
170
150
180
170
170
180
160
160
180
180
160
150
150
170
160
170
180
180
150
160
180
190
170
190
180
150
190
180
150
170
<150
170
160
170
170
160
170
160
160
170
150
180
150
180
150
>190
190
170
170
170
170
170
<150
170
180
170
180
180
170
170
150
170
180
160
170
160
170
190
160
170
190
160
160
190
180
180
160
170
170
160
150
>190
190
170
>190
190
150
180
180
160
190
150
> Mean170 cmMean < Mean
160
0 1 2−2 −1
![Page 36: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/36.jpg)
Putting random errors and p-
values in context36
![Page 37: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/37.jpg)
Meaning of P Value• P value tells us about plausibility of H0, (A=B)
– Assumes H0 is true, what is probability of observed given expected
– Example: Hypertension trial, Drug A>Drug B, p=.031, reject H0
– Example: Coin toss, 5 heads in a row chance
37
.500.500 .250.250 .125.125 .063.063 .031.031
![Page 38: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/38.jpg)
Multiple p values
3899.4%10092.3%5072.3%2540.1%1022.6%518.5%414.3%39.8%25%1
P(≤1 Test Sig) Test #
![Page 39: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/39.jpg)
Statistical Significance≠ Clinical Significance
39
Drug A ↓ BP 11 mm HgDrug A ↓ BP 11 mm HgDrug B ↓ BP 10 mm HgDrug B ↓ BP 10 mm Hg
∆∆=1 mm Hg, p=.01, n=100k=1 mm Hg, p=.01, n=100k
![Page 40: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/40.jpg)
P value: Effect Size & SNR (variability)• Example: Weight loss pills vs. placebo:
• Precise pill: 2 lb loss w/ sem of .9 lbs, p value < .05, reject H0 • Noisy pill: 10 lb loss w/ sem of 6 lbs, p value > .05, don’t
reject H0
• Which pill is more effective?
40
0 lb
s
2 lbs 10 lbs
![Page 41: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/41.jpg)
Confidence limits vs. p values• P value says nothing about effect size or variability• 95% confidence limits: sample mean±2(sem)• Estimate of effect size and precision (variability)• 95%CI≠95% chance μ is w/in CI, more complex• CI does not include bias• Can be used for significance testing
4112 lbs10 lbs8 lbs0 lbs
95% CI
![Page 42: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/42.jpg)
What effects β Error/Power?• Power is P(Detecting real effect) Sensitivity• β is P(Missing a real effect) FN, random effects• Power=1-β• Power effects:– Level of α– Effect size– Sample variability– Sample size
42
![Page 43: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/43.jpg)
Clinical Research, 2x2 Table
43
TrialResult
HA True
HA
FalsePositive TP FP (α)
Type I→ PPV
Negative
FN (β) Type II
TN → NPV
↓Power
↓*p value
Total
![Page 44: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/44.jpg)
44
α=.20, .05, .01
![Page 45: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/45.jpg)
45
TP
FP
α=.20, Big Hole to reject H0
![Page 46: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/46.jpg)
46
TP
FP
α=.05, Just Right Hole to reject H0
![Page 47: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/47.jpg)
47
TP
FP
α=.01, Small Hole to reject H0
![Page 48: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/48.jpg)
What effects Power?
48
Use SNR AnalogyWaldo=effect (signal),
Others=variability/σ (noise) Waldo
![Page 49: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/49.jpg)
Power and sample size: Rachel’s coin
49
![Page 50: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/50.jpg)
Clinical Research, 2x2 Table
50
TrialResult
HA True
HA
FalsePositive TP FP (α)
Type I→ PPV
Negative
FN (β) Type II
TN → NPV
↓Power
↓*p value
Total
![Page 51: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/51.jpg)
Prior Probability and Trial PPV/NPV
51
Eye of newt
Rest of newt
![Page 52: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/52.jpg)
Placebo vs. Emesis for Plague
52
![Page 53: Clinical Trial Statstics 2016](https://reader035.vdocuments.us/reader035/viewer/2022062821/58880e8e1a28ab083c8b491f/html5/thumbnails/53.jpg)
Summary: Clinical Trial Results
53
True? Random? Bias?