psycho physics 3

Psychophysics 3

Research MethodsFall 2010

Tamás Bőhm

Signal detection theory• Aka. sensory decision theory (SDT)• A model & a data analysis method for decision

problems with uncertainty (noise)• Originates from World

War II: aircraft detection on radar signals

• Today: widely used in psychophysics, medicine, radiology and machine learning

Signal detection theory• Experiment setup:

– In some trials a stimulus (signal) is presented, in others there is no stimulus;

– Observer reports if she/he saw a signal or not– Calculate how many times the observer detected a

signal when she/he was presented one (hit rate)• Is the hit rate all we want to know?

Two observers achieved the same hit rate. Are they certainly behaving the same way?

• NO, we also need to know how many times the observer said “I see” when there was no signal (false alarm rate)

Signal detection theory• Confusion matrix: contains all the information about the

observer’s performance

Signal detection theory• Confusion matrix: contains all the information about the

observer’s performance• As columns add up

to 100%, it is enough to know one item from each column

40 trials

20 20

18

2

6

14

= 100% = 100%

= 90%

= 10% = 70%

= 30%

Signal detection theory

• Perfect detection:

100%

100%0%

0%


• No detection at all (1st example): always reporting “Seen”

100%

0%0%

100%


• No detection (2nd example): always reporting “Not seen”

0%

100%100%

0%


• No detection (3rd example): flipping a coin

50%

50%50%

50%


• No detection (4th example): reporting “Seen” in 30% of the trials (no matter what is presented)

30%

70%70%

30%=

=

Rows equal no detection


• Receiver operating characteristic (ROC):

false alarm rate

hit r

ate

100%

100%



false alarm rate

hit r

ate

100%

100%

90% 30%

10% 70%



false alarm rate

hit r

ate

100%

100%

100% 0%

0% 100%

Perfect detection



false alarm rate

hit r

ate

100%

100%

100% 100%

0% 0%

No detection: always “yes”



false alarm rate

hit r

ate

100%

100%

0% 0%

100% 100%

No detection: always “no”



false alarm rate

hit r

ate

100%

100%

50% 50%

50% 50%

No detection: reporting “yes” in 50% of the trials (flipping a coin)



false alarm rate

hit r

ate

100%

100%

40% 40%

60% 60%

No detection: reporting “yes” in 40% of the trials



false alarm rate

hit r

ate

100%

100%

30% 30%

70% 70%




false alarm rate

hit r

ate

100%

100%

60% 60%

40% 40%




false alarm rate

hit r

ate

100%

100%Diagonal: no detection

Signal detection theory• SDT model:

• No way to remove the noise• But sensation can be separated from decision by

using ROCs

Sensation

Noise

DecisionSignal

present/absent

Sensation level (SL)

SL ≥ β

Criterion (β)

SL < β

YES

NO


Sensation

(Noise)

DecisionSignal

present/absent


SL ≥ β

Criterion (β)

SL < β

YES

NO

sensation level

prob

abili

ty

Without noise: perfect detection is possible

criterion signal present

signal absent


Sensation

(Noise)

DecisionSignal

present/absent


SL ≥ β

Criterion (β)

SL < β

YES

NO

sensation level

prob

abili

ty

criterion signal present

signal absent

100% 0%

0% 100%


Sensation

Noise

DecisionSignal

present/absent


SL ≥ β

Criterion (β)

SL < β

YES

NO

sensation level

prob

abili

ty

Noise: smears the distributions perfect detection is impossible (if the two distributions overlap)

signal absent(noise only)

signal present(signal+noise)criterion

online demo

http://psych.hanover.edu/JavaTest/Media/Chapter2/MedFig.SignalDetectionTheory.html


Sensation level

Sensation level

http://www-psych.stanford.edu/~lera/psych115s/notes/signal/


Sensation level

Sensation level

false alarm ratehi

t rat

e


false alarm ratehi

t rat

e

ROC curve

β = 8

β = 6

β = 10

β = 6β = 8

β = 10


false alarm ratehi

t rat

e

β

sensation level

prob

abili

ty

• Criterion (β): specifies where we are on the ROC curve

• The ROC curve is specified by sensory capacities only(discriminability)

Signal detection theory• Discriminability:

how well the observer can separate the presence of signal from its absence~ overlap between the two distributions~ bowing out of the ROC curve

• Measured by d’ (discriminability index,also called sensitivity)

http://www-psych.stanford.edu/~lera/psych115s/notes/signal/

Signal detection theoryd’: selects the ROC curveβ: specifies a point on the

selected ROC curvesame information as hit rate

& false alarm rate, but:hit rate, false alarm rate:

both reflect sensation & decision characteristics;cannot separate the two

d’: depends only on sensationβ: depends only on decision

β

The two processes are separated

http://psych.hanover.edu/JavaTest/Media/Chapter2/MedFig.ROC.html








Signal detection theoryFechner’s methods:

Is a stimulus detectable? Yes or no?

• Clear-cut threshold value (with some variability) that can be measured– Stimulus intensity >

threshold detectable– Stimulus intensity <

threshold not detectable• Dichotic outcome,

categorical model

Signal detection theory:How well is it detectable? How sensitive the observer is to the stimulus?

• Measured by d’– The higher d’ is, the more

the stimulus is detectable– d’ = 0

not detectable at all• Scalar outcome,

dimensional model


Sensation

(Noise)

Stimulus


Different task

Correct Incorrect

Forced-choice: eliminates the criterion

SDT: separates the criterion

Decision

SL ≥ β

Criterion (β)

SL < β

YES

NO

• Problem with Fechner’s methods: criterion

Signal detection theoryPsychophysical measurements with SDT:

1. Create a stimulus set with a range of intensities (like in the method of constant stimuli)

2. Test each stimulus many times with each observer3. On each trial, either present a randomly selected

stimulus or do not present anything4. Ask the observer if he/she detected the stimulus5. Calculate the hit rate and false alarm rate for each

observer, for each stimulus intensity6. Use the formula/table to calculate d’ for each case7. Examine how d’ changes with intensity: the higher

d’ is for a stimulus intensity, the greater the observer’s ability to detect this intensity

http://psych.hanover.edu/JavaTest/Media/Chapter2/MedFig.SignalDetection.html









• Main results: changes in d’ values

Caudek–Rubin Vision Res. 2001

Signal detection theory• There is also a β value for each d’ value• It can be informative about the decision

behavior:– Balanced: false alarm and

miss rates are equal– Liberal: the observer

says “yes” whenever there may be a signal

– Conservative: decision is yes only when it is almost certain that there is a signal

sensation level

prob

abili

ty

balanced

conservativeliberal

psycho physics 3

Documents