ON THE RELATIONSHIP BETWEEN MOTOR AND PERCEPTUAL BEHAVIOR – A SDT FRAMEWORK
Andrei Gorea
with
Pedro Cardoso-LeiteFlorian Waszak
Pascal Mamassian
Laboratoire de Psychologie de la PerceptionCNRS & René Descartes University
71 Ave Edouard Vaillant, 92774 Boulogne-Billancourt, France
1. Background elements
2. RT & Perceptual States with masked and not-masked stimuli
2.1 A one-path – two-decisions model
3. RT & Temporal Order Judgments
3.1 One-path – two-decisions model again
4. General conclusions
SYNOPSYS
PART I
SOME BACKGROUND
Sensory Input
A. Layman’s view
Action
with perceptual awareness
PerceptualDecision
Verbal reportaware / not aware
Priming & Metacontrast + RTPriming & Metacontrast + RT
Fehrer, E. & Raab, D. (1962) Harrison, K. & Fox, R. (1966) Schiller, P. H. & Smith, M. C. (1966) Neumann, O. (1982) Neumann, O., Esselmann, U. & Klotz, W.
(1993) Klotz, W. & Neumann O. (1999) Taylor, J.L. & McCloskey, D.I. (1990) Steglich, C. & Neumann, O. (2000) Schmidt, T. (2002) Ogmen, H., Breitmeyer, B. & Melvin, R. (2003) Scharlau, I. & Ansorge, U. (2003) Scharlau, I. & Neumann, O. (2003) Vorberg, D., Mattler, U., Heinecke, A.,
Schmidt, T. & Schwarzbach, J. (2003) Rossetti, Y. & Pisella, L. (2002) Breitmayer, B., Ro, T. & Singhal, N. S. (2004)
Perception & Gaze pursuitPerception & Gaze pursuit
Beutter, B.B. & Stone, L.S. (1998) Beutter, B.B. & Stone, L.S. (2000). Gegenfurtner, K.R., Xing, D., Scott, B.H. &
Hawken, M.J. (2003) Osborne, L.C., Lisberger, S.G. & Bialek, B.
(2005) Morrone, M.C., Raffele, S., Ma-Wyatt, A. &
Ross, J. (2005)
TOJ & RTTOJ & RT
Roufs, J.A.J. (1974) Jaskowski, P. (1991, 1992, 1993, 1996) Tappe, T., Niepel, M. & Neumann, O (1993) Jaskowski, P. & Verleger, R. (2000) Spence C., Baddeley R., Zampini M., James
R. & Shore D.I. (2003) Adams, W.J & Mamassian, P. (2004)
Most frequently used experimental paradigms
Decision rulenot specified
Decision rule(implicit)
Sensory Input
Action
with or withoutperceptual awareness
Verbal report
aware / not aware
B. The 2 pathways view
?
Lateral Interactions(implicit)
ventral
dorsal
Most frequently used experimental paradigms
PARADIGM Perceptual task Motor Task Comments
Congruent/Incongruent Priming with/without Metacontrast
Detection &/or Discrimination (typically bearing on the “prime”)Appearance (of the "prime“)
Mostly choice- (but also a few simple-) RT (manual, saccades)
Despite consensus, dissociation was not formally proved: methodological problems of many sorts
Conceptually, choice-RT cannot be dissociated from perceptual decisions
Temporal Order Judgments
TOJ Simple-RT, typically to one single stimulus (rather than the 2 used in the perceptual task)
Inconsistent results Methodological problems
Perception & gaze pursuit
Discrimination (typically of speed)
Pointing, gaze pursuit
Contradictory results (presumably due to technical problems)
None of the experimental paradigms used to explore the
sensorimotor dissociation allowed a trial-by-trial analysis of the
relationship between the motor behavior & the state of the
perceptual system (Hits, FA, Misses, CR).
PART II
RESPONSE TIME&
PERCEPTUAL STATE
Simple RT
Correlation S1-RT
Correlation S2-RT
T I M E
T0 : Start Clock
400-1000
0
d’ modulation
criterion modulation
Stimuli & Paradigm(one trial)
SOA: 50-250
S2 : Mask/Primed (p = 1)
20
Masked(metacontrast)
S2
S1: Yes/No?
HitsFAMissesCR
100-200
S1 : Target/Prime (p = .2, .5, .8)
20
S1
7°
SOA: 50-250
S2 : Mask/Primed (p = 1)
20
NOT Masked
S2
22.5°
Waszak & Gorea (2004).
Of the 4 perceptual response categories, Hits & Misses are of particular interest:
They tell us about the motor behavior when the Obs. says he senses and does not sense the test stimulus (S1), hence establishing the relationship between perceptual and motor behavior*.
* RT for FA are not reliable indices as they have an unknown temporal origin; RT for CR are simply used as reference for the relevant RT.
The motor system appears to react if and only if
the stimulus is present and
the observer is “aware” of it (i.e. only for Hits).
The difference between RTs for Hits & Misses points against a full sensori-motor dissociation.
S1
Masked(metacontrast)
S2
SPACE
4 SjsS1 13 ms
SOA variableS2 36 ms
300 trials / d’ / Obs
TIME
t20 ms 20 ms
SOA
variable
350
400
450
500
550
600
650
0 1 2 3 4 5
RT
(m
s)
AGp[S1] = .5
'ed [S1]
FA
MissesCR
Hits
Waszak & Gorea (2004).
350
400
450
500
550
600
650
0 1 2 3 4 5
TR
(ms)
350
400
450
500
550
600
650
0 1 2 3 4 5
TR
(ms)
350
400
450
500
550
600
650
0 1 2 3 4 5
TR
(ms)
350
400
450
500
550
600
650
0 1 2 3 4 5
TR
(ms)
350
400
450
500
550
600
650
0 1 2 3 4 5
TR
(ms)
350
400
450
500
550
600
650
0 1 2 3 4 5
TR
(ms)
'ed
Hits Misses FA CR
AG SD
.2
.5
p[S1]
.8
350
400
450
500
550
600
650
0 1 2 3 4 5
RT
(m
s)
350
400
450
500
550
600
650
0 1 2 3 4 5
RT
(m
s)
350
400
450
500
550
600
650
0 1 2 3 4 5
RT
(m
s)FW
Waszak & Gorea (2004).
Rank Correlations between RT and S1-, S2-onsets as a fct. of d’
c. Correct Rejections
-0.1
0.3
0.7
0 1 2 3 4 5
r tR
-tS
2
a. Hits
-0.1
0.3
0.7
0 1 2 3 4 5
r tR
-tS
1
&
r tR
-tS
2
'ed
Waszak & Gorea (2004).
TIME
Correlation S1-RT
Correlation S2-RT
S1
Speeded RT
S2
SPACE
S1 S2
SD
AG
FW
Sj
b. Misses
-0.1
0.3
0.7
0 1 2 3 4 5r
tR-t
S1
&
r
tR-t
S2
d. False Alarms
-0.1
0.3
0.7
0 1 2 3 4 5
r tR
-tS
2
Decision rulenot specified
Decision rule(implicit)
Sensory Input
Action
with or withoutperceptual awareness
Verbal report
aware / not aware
?
Lateral Interactions(implicit)
ventral
dorsal
B. The 2 pathways view
We’ve thus replaced the standard 2-pathways view…
Actio
n
with p
erce
ptual
awar
enes
s
A’. Layman’s view modified
…with a slightly modified layman’s view
Where does the discrepancy come from?(Aside from potential methodological problems in previous studies)
Common denominator: Most of the previous (whether simple or choice RT)
studies used 100% contrast targets whose ‘invisibility’ was ensured by strong
backward masking.
In contrast, our targets (S1) yielded maximum contrasts of about 20%. To
allow for higher target contrasts while keeping sensitivity constant, shorter
SOA-s (48 ms instead of an average of 162 ms) were used in a second series
of experiments; these entailed S1 contrasts around 30%.
Waszak & Gorea (2004).
350
400
450
0 1 2 3 4 5
RT
(m
s)
FW
p[S1] = .5
0 1 2 3 4 5
AG
p[S1] = .5
FA
MissesCR
Hits
'ed
TIME
t20 ms 20 ms
SOA
52 ms
S1
Masked(metacontrast)
S2
SPACE
Fixed Motor Threshold
Variable Perceptual Criterion
Sensory Input
Action
with or withoutperceptual awareness
Verbal report
aware / not aware
C. Gorea & Waszak (2004)
Lateral Interactions
ventral
dorsal
Actio
n
with p
erce
ptual
awar
enes
s
The temptation was strong to conclude (Gorea & Waszak, 2004)…
RT for Misses drop with d’ (or contrast) only for the masked condition!
RT appear to depend on Contrast rather than on d’!
Perceptual HITS
220
240
260
280
300
320
0.5 1 1.5 2 2.5
d'
RT
8%13%
27%
13%
8% 13%
27%
13%
Perceptual MISSES
0.5 1 1.5 2 2.5
d'
S1
Masked(metacontrast)
S2 S1
S2
NOT Masked
S2
17 Obs300 trials / d’ / Obs
MaskedNOT Masked
Masked
NOT Masked
NOT Masked
Masked
S1
“prime”
S2
“mask”
t
SOA
13 ms 36 ms
52 ms
Waszak & Gorea, new experiments.
However, the data are more intricate than that…
S1
Masked(metacontrast)
S2
NOT MASKED(6 Obs)
0 10 20 30 40
S1 Contrast (%)
2.6
3.33.2
.14
.451.2
2.1
d’
S1
S2
NOT Masked
S2
S1
“prime”
S2
“mask”
t
SOA
13 ms 36 ms
52 ms
MASKED(6 Obs)
-50
-40
-30
-20
-10
0
10
0 10 20 30 40
S1 Contrast (%)
.21
.32
.75
1.8
2.1
2.42.6
2.82.83.23.1 3.2
3.4 d’RT
ga
in r
el.
to C
R (
ms)
RTHITS-RTCR
RTMISS-RTCR
300-900 trials / C / Obs
HITS(6 Obs)
Not Masked
300-900 trials / C / Obs
Masked
MISSES(6 Obs)
RTHITS-RTCR
RTMISS-RTCR
300-900 trials / C / Obs
Not Masked
300-900 trials / C / Obs
Masked
Waszak & Gorea, new experiments.
S1
Masked(metacontrast)
S2 S1
S2
NOT Masked
S2
S1
“prime”
S2
“mask”
t
SOA
13 ms 36 ms
52 ms
0 1 2 3 4
d’S1
NOT MASKED(6 Obs)
-50
-40
-30
-20
-10
0
10
0 1 2 3 4
d’S1
RT
ga
in r
el.
to C
R (
ms)
MASKED(6 Obs)
RTHITS-RTCR
RTMISS-RTCR
300-900 trials / C / Obs
RTHITS-RTCR
RTMISS-RTCR
300-900 trials / C / Obs
HITS(6 Obs)
Not Masked
300-900 trials / C / Obs
Masked
MISSES(6 Obs)
RTHITS-RTCR
RTMISS-RTCR
300-900 trials / C / Obs
Not Masked
300-900 trials / C / Obs
Masked
Waszak & Gorea, new experiments.
Fixed Motor Threshold
Variable Perceptual Criterion
Sensory Input
Action
with or withoutperceptual awareness
Verbal report
aware / not aware
C. Gorea & Waszak (2004)
Lateral Interactions
ventral
dorsal
Actio
n
with p
erce
ptual
awar
enes
s
…And a simpler, one pathway SDT model can account for them. Instead of:…we propose:
Fixed Motor Threshold
Variable Perceptual Criterion
Sensory Input
Action
with or withoutperceptual awareness
Verbal report
aware / not aware
D. Current view
Lateral Interactions
ventral
ventral
Actio
n
with p
erce
ptual
awar
enes
s
A conceptual model
40 140 240 340
TIME (ms)
S1
Masked(metacontrast)
S2
TIME (ms)
RE
SP
ON
SE
(N
ois
e u
nit
s)
40 140 240 340
-2
-1
0
1
2
3
4
5
6
7
RTd’=1
Motor threshold
d’
Perc
eptu
al M
isse
s
S1
S2
NOT Masked
S2
S1
“prime”
S2
“mask”
t
SOA
Perc
eptu
al M
isse
s
S1
SOA
RTMask
S2
SOA
RT Mask
S2
RT d’=1
S1
S1
d’
-2
-1
0
1
2
3
4
5
6
7
-60 40 140 240 340 440
TIME (ms)
RE
SP
ON
SE
(N
ois
e u
nit
s)
-2
-1
0
1
2
3
4
5
6
7
-60 40 140 240 340 440
TIME (ms)
RE
SP
ON
SE
(N
ois
e u
nit
s)
S1
“prime”
S2
“mask”
t S1
Masked(metacontrast)
S2S1
S2
NOT Masked
S2
RT d’=2RT
Mask RT d’=2RT
Mask
SOA SOA
Perc
eptu
al M
isse
s
Perc
eptu
al M
isse
s
d’
d’
Motor threshold
SOA
-2
-1
0
1
2
3
4
5
6
7
-60 40 140 240 340 440
TIME (ms)
RE
SP
ON
SE
(N
ois
e u
nit
s)
S1
Masked(metacontrast)
S2
S1
“prime”
S2
“mask”
t
SOA
Perc
eptu
al M
isse
s
d’
SOA
1
7
240 340
TIME (ms)
S1
S2
NOT Masked
S2
RT d’=4RT
Mask-2
-1
0
40 140
RTMaskRT d’=4
Motor threshold
-60
RE
SP
ON
SE
(N
ois
e u
nit
s)
SOA6
2d’
Perc
eptu
al M
isse
s
3
4
5
S1
Masked(metacontrast)
S2 S1
S2
NOT Masked
S2
S1
“prime”
S2
“mask”
t
SOA
13 ms 36 ms
52 ms
MASKED6 Obs
230
240
250
260
270
280
290
300
0 1 2 3 4
d’S1
RT
(m
s)
NOT MASKED6 Obs
0 1 2 3 4
d’S1
RTHITS-RTCR
RTMISS-RTCR
300-900 trials / C / Obs
Model Fits
Motor Threshold.8
Sensory Input
A. Layman’s view
Action
with perceptual awareness
PerceptualDecision
Verbal report
aware / not aware
Decision rulenot specified
Decision ruleimplicit
Sensory Input
Actionwith or without
perceptual awareness
Verbal report
(aware / not aware)
B. The 2 pathways view
?
Lateral Interactionsimplicit
ventral
dorsal
Fixed MotorThreshold
Variable PerceptualCriterion
Sensory Input
Actionwith or without
perceptual awareness
Verbal report
(aware / not aware)
C. Gorea & Waszak (2004)
Action
with perc
eptual a
wareness
Lateral Interactions
dorsal
ventral
Fixed MotorThreshold
Variable PerceptualCriterion
Sensory Input
Actionwith or without
perceptual awareness
Verbal report
(aware / not aware)
D. Current view
Action
with perc
eptual a
wareness
Lateral Interactions
ventral
dorsal
ventral
A one-pathway model with two distinct activation levels accounts for the observed perceptual-motor relationship under both masking and non-masking experimental conditions.
There is a fixed motor threshold ( 0.8) to be contrasted with a variable perceptual criterion).
The motor threshold is measured in noise () units as referred to the “absolute” perceptual detection “threshold” (i.e. in ref. to the internal noise).
RT for “unconscious” stimuli (i.e. Misses) depends on the reference (noise) level at which the perceptual task is performed;
as this reference level exceeds the motor threshold, the internal response associated with perceptual Misses also exceeds it and progressively contributes to shortening the RT.
TAKE-HOME MESSAGES (Part II)
PART III
RESPONSE TIME&
TEMPORAL ORDER JUDGMENTS
Time
C2 C1
Trigger-Delay PSS
PSS
RTC2 RTC1
Threshold
Inte
rnal
R
esp
on
se
SOA
PSS
p(S
1 p
erc
eiv
ed
fir
st)
slop
e
Temporal Order Judgments & RT
IFF RT is strictly dependent on the sensory signal (as it determines the TOJ), then the slope and the PSS of the TOJ -function should be direct indicators of the variance of the RT distributions and of their mean difference, respectively: this is a one-pathway sensorimotor model.
a b c
In contrast with previous studies: TOJ and RT were measured in the same trial; It was hence possible to assess RTs for correct & incorrect TOJs.
0
20
40
60
80
100
120
140
160
165 195 225 255 285 315 345 375 405 435 465 495 525
RT (ms)
Fre
qu
en
cy
RT
C2
C1
DG
PSS = 0 ms
DG
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-200 -100 0 100 200
SOA (ms)
%S
1 fi
rst
C1-C1
C2-C2
C1-C2
PSS = 49 ms
500 ms
500-1300 ms
SOA
~
1700 msRT
Left / Right
C1 C2 O1 O2
C1
C2
O1
O2
Stimuli & Paradigm(one trial)
DGC1-C1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC1-C1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
Identical
Different
-RTCorrect / Incorrect
DGC1-O1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC1-O1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
Correct
Incorrect
S1 1st
S2 1st
DGC1-O1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC1-O2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC1-O1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC1-O2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC1-C2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC1-C2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC2-O1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC2-O1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC2-O2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC2-O2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGO1-O2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGO1-O2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC1-C1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC2-C2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGO1-O1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGO2-O2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC1-C1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGC2-C2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGO1-O1
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
DGO2-O2
250
275
300
325
350
375
400
-100 -50 0 50 100SOA
RT
375
Identical Different
-RTCorrect / Incorrect -RT
Correct / Incorrect
4 SjsMean RT
R2 = 0.9174p << .0001
220
240
260
280
300
320
340
360
380
400
220 240 260 280 300 320 340 360 380 400
Predicted
Me
as
ure
d
DG
FC
PC
AG
4 SjsCorrect / Incorrect
R2 = 0.3543p < .0001
R2 = 0.1707p < .0001
220
240
260
280
300
320
340
360
380
400
220 240 260 280 300 320 340 360 380 400
Predicted
Me
as
ure
dCorrect DG
Correct FC
Correct PC
Correct AG
Inorrect DG
Inorrect FC
Inorrect PC
Inorrect AG
4 Sjs
0 vs. 32 df -RT fits
Correct-IncorrectTOJ data
R2 = 0.8415p << .0001
220
240
260
280
300
320
340
360
380
400
220 240 260 280 300 320 340 360 380 400
Measured -RT in "single" (ms)
Fit
ted
-R
T (
ms
)
4 Sjs
0 vs. 32 df -RT fits
Mean TOJ data
R2 = 0.8273p << .0001
220
240
260
280
300
320
340
360
380
400
220 240 260 280 300 320 340 360 380 400
Measured -RT in "single" (ms)
Fit
ted
-R
T (
ms
)
a b
4 Sjs
0 vs. 32 df -RT fits
Correct-IncorrectTOJ data
R2 = 0.8568p << .0001
0
10
20
30
40
50
60
70
80
90
100
0 20 40 60 80 100
Measured -RT in "single" (ms)
Fit
ted
-R
T (
ms
)
4 Sjs
0 vs. 32 df -RT fits
Mean TOJ data
R2 = 0.7094p < .0001
0
10
20
30
40
50
60
70
80
90
100
0 20 40 60 80 100
Measured -RT in "single" (ms)
Fit
ted
-R
T (
ms
)
Red: C1Green: C2Blue: O1
Brown: O2
c d
4 SjsPSSDG
R2 = 0.5893p = .037
AG
R2 = 0.6511p = .026
-150
-100
-50
0
50
100
150
0 50 100 150RT (from "single" - ms)
PS
S (
fro
m T
OJ
- m
s)
DG
FC
PC
AG
4 SjsTOJ -fct Slope
DG
R2 = 0.5337p = .008
AG
R2 = 0.8899p << .0001
0
20
40
60
80
100
120
140
160
180
200
0 20 40 60 80 100 120 140 160 180 200-RT (from "single" - ms)
Slo
pe
TO
J
-fu
nc
tio
n (
ms
)
DG
FC
PC
AG
a b
RT
PS
S
RT
R
M
PC
CHigh
PT1 PT2
RT1
PSS
RT
RT2
M <
PC
M > PCM
= P C
R
t
CLow
For 2 out of 4 Obs, RT to single stimuli also predict their TOJ behavior (PSS and slopes of the TOJ -fct.); their behavior is hence compatible with a one pathway model (no sensorimotor dissociation).
For these 2 Obs, the relationship between RT-differences and PSS also suggests the existence of a motor threshold distinct from and higher than the perceptual criterion.
The behavior of the remaining 2 Obs does not follow any coherent pattern; it is likely that it was perturbed by perceptual response strategy factors, possibly under the influence of “transient Troxler masking” (Kanai & Kamitani, 2003).
Such factors may account for the various inconsistencies in the RT-TOJ literature.
TAKE-HOME MESSAGES (Part III)
The relationship between simple Response Times and perceptual states (Hits, FA, etc. and Temporal Order Judgments) can be accounted for by a single (one-path) system where two distinct decisions are made on the same incoming information.
The decision to act is based on a hard-wired threshold worth about 1 of the internal noise; the perceptual criterion is context dependent (in line with SDT).
TOJ data not complying with this model may reflect variable perceptual response strategies.
GENERAL CONCLUSIONS
THANK YOU
TIME SPACE
OR
OR
What do the Obs. compare?
vs.A
Either, both?C
vs.B
=<
=>
Most likely both(also depending on SOA)