introduction to biomedical statistics. signal detection theory what do we actually “detect” when...

Introduction to Biomedical Statistics

Signal Detection Theory

• What do we actually “detect” when we say we’ve detected something?


• What do we actually “detect” when we say we’ve detected something?

• We say we’ve “detected” when a criterion value exceeds a threshold


• examples:– the onset of a light or sound

– the presence of an abnormality on x-ray


• There are 4 possible situations

Target is:

You

Res

pond

:

Present Absent

Pre

sent

Abs

ent



Hit

Target is:

You

Res

pond

:

Present Absent

Pre

sent

Abs

ent



Hit

Miss

Target is:

You

Res

pond

:

Present Absent

Pre

sent

Abs

ent



Hit False

Alarm

Miss

Target is:

You

Res

pond

:

Present Absent

Pre

sent

Abs

ent



Hit False

Alarm

Miss Correct

Rejection

Target is:

You

Res

pond

:

Present Absent

Pre

sent

Abs

ent



Hit False

Alarm

Miss Correct

Rejection

Target is:

You

Res

pond

:

Present Absent

Pre

sent

Abs

ent

This is the total # of Target Present Trials


• Hit Rate (H) is the proportion of target present trials on which you respond “present”

€

H =# Hits

# Hits+# Misses


• Notice that H is a proportion, so 1 - H gives you the “miss” rate or…

€

MissRate =# Misses

#Hits+# Misses


• False-Alarm Rate (FA) is the proportion of target absent trials on which you respond “present”

€

FA =# FalseAlarms

# FalseAlarms+# CorrectRejections


• Notice that FA is a proportion. 1 minus FA gives you the correct rejections or …

€

CorrectRejectionRate =#CorrectRejections

#FalseAlarms+# CorrectRejections


• Signal Detection can be modeled as signal + noise with some detection threshold


Stimulus Intensity

Fre

quen

cy

Noise is normally distributed - Target Absent trials still contain some stimulus

Target Absent


Stimulus Intensity

Fre

quen

cy


Target Present trials contain a little bit extra intensity contributed by the signal

Target Absent

Target Present


Stimulus Intensity

Fre

quen

cy


Target Present trials contain a little bit extra intensity contributed by the signal

Target Absent

This is the signal’s contribution

Target Present


• We can imagine a static criterion above which we’ll respond “target is present”

Stimulus Intensity

Fre

quen

cy

Target Absent

Target Present

Criterion


• Notice that H, FA, etc thus have graphical meanings

Stimulus Intensity

Fre

quen

cy

Proportion Hits

Criterion



Stimulus Intensity

Fre

quen

cy

Proportion Misses

Criterion



Stimulus Intensity

Fre

quen

cy

Proportion False Alarms

Criterion



Stimulus Intensity

Fre

quen

cy

Proportion Correct Rejections

Criterion


• Notice that as H increases, FA also increases

Stimulus Intensity

Fre

quen

cy

H

Criterion


• Notice that as H increases, FA also increases

Stimulus Intensity

Fre

quen

cy

FA

Criterion


• d’ (pronounced d prime) is a measure of sensitivity to detect a signal from noise and does not depend on criterion - it is the distance between the peaks of the signal present and signal absent curves

Stimulus Intensity

Fre

quen

cy

• d’ is computed by converting from H and FA proportions into their corresponding Z scores and subtracting Zinv(FA) from Zinv(H)

Some Common Biomedical Statistics

• Sensitivity• Specificity• Positive Predictive Value• Negative Predictive Value• Likelihood Ratio• Relative Risk• Absolute Risk• Number needed to treat/harm

Sensitivity and Specificity

• Consider a test for a condition– e.g. Pregnancy test– e.g. Prostate-specific Antigen (Prostate

Cancer)– e.g. Ultrasound (Breast Cancer)

• These are all signal detection problems


• Four possible situations:

Condition is:

Tes

t R

esul

t:

Present Absent

Pre

sent

Abs

ent



True Positive

Condition is:

Tes

t R

esul

t:

Present Absent

Pre

sent

Abs

ent



True Positive

False Negative

Condition is:

Tes

t R

esul

t:

Present Absent

Pre

sent

Abs

ent



True Positive

False Positive

False Negative

Condition is:

Tes

t R

esul

t:

Present Absent

Pre

sent

Abs

ent



True Positive

False Positive

False Negative

True Negative

Condition is:

Tes

t R

esul

t:

Present Absent

Pre

sent

Abs

ent



True Positive

False Positive

False Negative

True Negative

Condition is:

Tes

t R

esul

t:

Present Absent

Pre

sent

Abs

ent

This is Total # of Condition Present Cases



True Positive

False Positive

False Negative

True Negative

Condition is:

Tes

t R

esul

t:

Present Absent

Pre

sent

Abs

ent

This is Total # of Condition Absent Cases



True Positive

False Positive

False Negative

True Negative

Condition is:

Tes

t R

esul

t:

Present Absent

Pre

sent

Abs

ent

This is Total # of “positive” tests



True Positive

False Positive

False Negative

True Negative

Condition is:

Tes

t R

esul

t:

Present Absent

Pre

sent

Abs

ent

This is Total # of “negative” tests


• Sensitivity is the proportion of condition present cases on which the test returned “positive”

• Analogous to the hit rate (H) in Signal Detection Theory

€

Sensivity =# True Positives

# True Postives + # False Negatives


• Specificity is the proportion of condition absent cases on which the test returned “negative”

• Analogous to the Correct Rejection rate in Signal Detection Theory

€

Specificity =# True Negative

# True Negative + #False Positive


• Notice that 1 minus the Sensitivity is analogous to the FA of Signal Detection Theory


• Notice that 1 minus the Sensitivity is analagous to the FA of Signal Detection Theory

• Recall that in Signal Detection Theory, as criterion were relaxed both H and FA increased and as criterion were more stringent, H and FA decreased


• Notice that 1 minus the Sensitivity is analagous to the FA of Signal Detection Theory

• Recall that in Signal Detection Theory, as criterion were relaxed both H and FA increased and as criterion were more stringent, H and FA decreased

• Sensitivity and Specificity have a similar relationship: as a cut-off value for a test becomes more stringent the sensitivity goes down and the specificity goes up…and vice versa


• “For detecting any prostate cancer, PSA cutoff values of 1.1, 2.1, 3.1, and 4.1 ng/mL yielded sensitivities of 83.4%, 52.6%, 32.2%, and 20.5%, and specificities of 38.9%, 72.5%, 86.7%, and 93.8%, respectively.”

JAMA. 2005 Jul 6;294(1):66-70.


• Likelihood Ratio is the ratio of True Positive rate to False Positive rate

• Loosely corresponds to d’ in that Likelihood ratio is insensitive to changes in criterion €

Likelihood Ratio =Sensitivity

1 - Specificity


• If a test is positive, how likely is it that the condition is present?

• Positive Predictive Value is the proportion of “positive” test results that are correct

€

PPV =# True Positives

# True Postives + #False Positives


• Negative Predictive Value is the proportion of “negative” test results that are correct

€

NPV =# True Negatives

# True Negatives + # False Negatives


• Consider the influence of exposure to some substance or treatment on the presence or absence of a condition

• e.g. smoking and cancer• e.g. aspirin and heart disease


• A similar logic can be applied

A B

C D

Disease:

Exp

osur

e:

Yes No

No

Yes



A B

C D

Disease:

Exp

osur

e:

Yes No

This is total # exposure

No

Yes



A B

C D

Disease:

Exp

osur

e:

Yes No

This is total non-exposure

No

Yes


• We can think in terms of “Event Rates”

€

Control Event Rate =C

C + D

A B

C D

Disease:

Exp

osu

re:

Yes No

Ye

sN

o

€

Exposure Event Rate =A

A + B

e.g. the proportion of non-smokers who get lung cancer

e.g. the proportion of smokers who get lung cancer


• Relative Risk is the ratio of Exposure Events to Non-Exposure Events

• Often encountered in regard to rate of adverse reactions to drugs

€

Control Event Rate =C

C + D

A B

C D

Disease:

Exp

osu

re:

Yes No

Ye

sN

o

€

Exposure Event Rate =A

A + B

€

Relative Risk =Exposure Event Rate

Control Event Rate=A /(A + B)

C /(C + D)


• Often we are interested in whether the chance of an event changes with exposure

• Relative Risk Reduction is the difference between event rates in the exposure and non-exposure groups, expressed as a fraction of the non-exposure event rate

€

Relative Risk Reduction =Exposure Event Rate - Control Event Rate

Control Event Rate


• Notice that Relative Risk Reduction can be positive or negative: that is, exposure could reduce the risk of some event (e.g. exposure to wine reduces risk of heart disease) or increase the risk (e.g. exposure to cigarette smoke increases risk of heart disease)

€

Relative Risk Reduction =Exposure Event Rate - Control Event Rate

Control Event Rate


• Notice also that these figures do not take into account the absolute numbers

• e.g. control event rate = .264 and exposure event rate = .198• e.g. control event rate = .000000264 and exposure event rate =

.000000198

• Both yield the same relative risk reduction of -25%


• Notice also that these figures do not take into account the absolute numbers

• e.g. control event rate = .264 and exposure event rate = .198• e.g. control event rate = .000000264 and exposure event rate =

.000000198

• Both yield the same relative risk reduction of -25%

• Doesn’t discriminate between large and small effects


• The absolute risk reduction conveys effect size

€

Absolute Risk Reduction = Exposure Rate - Control Rate


• The absolute risk reduction conveys effect size

• An intuitive version is to consider the reciprocal - the “number needed to treat or harm”

• Indicates the number of individuals that would have to be exposed to the treatment in order to cause one to have the outcome of interest

€

Absolute Risk Reduction = Exposure Rate - Control Rate

€

Number Needed to Treat or Harm = 1

Absolute Risk Reduction

introduction to biomedical statistics. signal detection theory what do we actually “detect” when...

Documents

signal detection theory

stimulus target absent

detection threshold

stimulus target present

signal noise

possible situations

xray slide

biomedical statistics