on the relationship between motor and perceptual behavior – a sdt framework andrei gorea with...

ON THE RELATIONSHIP BETWEEN MOTOR AND PERCEPTUAL BEHAVIOR – A SDT FRAMEWORK

Andrei Gorea

with

Pedro Cardoso-LeiteFlorian Waszak

Pascal Mamassian

Laboratoire de Psychologie de la PerceptionCNRS & René Descartes University

71 Ave Edouard Vaillant, 92774 Boulogne-Billancourt, France

1. Background elements

2. RT & Perceptual States with masked and not-masked stimuli

2.1 A one-path – two-decisions model

3. RT & Temporal Order Judgments

3.1 One-path – two-decisions model again

4. General conclusions

SYNOPSYS

PART I

SOME BACKGROUND

Sensory Input

A. Layman’s view

Action

with perceptual awareness

PerceptualDecision

Verbal reportaware / not aware

Priming & Metacontrast + RTPriming & Metacontrast + RT

Fehrer, E. & Raab, D. (1962) Harrison, K. & Fox, R. (1966) Schiller, P. H. & Smith, M. C. (1966) Neumann, O. (1982) Neumann, O., Esselmann, U. & Klotz, W.

(1993) Klotz, W. & Neumann O. (1999) Taylor, J.L. & McCloskey, D.I. (1990) Steglich, C. & Neumann, O. (2000) Schmidt, T. (2002) Ogmen, H., Breitmeyer, B. & Melvin, R. (2003) Scharlau, I. & Ansorge, U. (2003) Scharlau, I. & Neumann, O. (2003) Vorberg, D., Mattler, U., Heinecke, A.,

Schmidt, T. & Schwarzbach, J. (2003) Rossetti, Y. & Pisella, L. (2002) Breitmayer, B., Ro, T. & Singhal, N. S. (2004)

Perception & Gaze pursuitPerception & Gaze pursuit

Beutter, B.B. & Stone, L.S. (1998) Beutter, B.B. & Stone, L.S. (2000). Gegenfurtner, K.R., Xing, D., Scott, B.H. &

Hawken, M.J. (2003) Osborne, L.C., Lisberger, S.G. & Bialek, B.

(2005) Morrone, M.C., Raffele, S., Ma-Wyatt, A. &

Ross, J. (2005)

TOJ & RTTOJ & RT

Roufs, J.A.J. (1974) Jaskowski, P. (1991, 1992, 1993, 1996) Tappe, T., Niepel, M. & Neumann, O (1993) Jaskowski, P. & Verleger, R. (2000) Spence C., Baddeley R., Zampini M., James

R. & Shore D.I. (2003) Adams, W.J & Mamassian, P. (2004)

Most frequently used experimental paradigms

Decision rulenot specified

Decision rule(implicit)

Sensory Input

Action

with or withoutperceptual awareness

Verbal report

aware / not aware

B. The 2 pathways view

?

Lateral Interactions(implicit)

ventral

dorsal

Most frequently used experimental paradigms

PARADIGM Perceptual task Motor Task Comments

Congruent/Incongruent Priming with/without Metacontrast

Detection &/or Discrimination (typically bearing on the “prime”)Appearance (of the "prime“)

Mostly choice- (but also a few simple-) RT (manual, saccades)

Despite consensus, dissociation was not formally proved: methodological problems of many sorts

Conceptually, choice-RT cannot be dissociated from perceptual decisions

Temporal Order Judgments

TOJ Simple-RT, typically to one single stimulus (rather than the 2 used in the perceptual task)

Inconsistent results Methodological problems

Perception & gaze pursuit

Discrimination (typically of speed)

Pointing, gaze pursuit

Contradictory results (presumably due to technical problems)

None of the experimental paradigms used to explore the

sensorimotor dissociation allowed a trial-by-trial analysis of the

relationship between the motor behavior & the state of the

perceptual system (Hits, FA, Misses, CR).

PART II

RESPONSE TIME&

PERCEPTUAL STATE

Simple RT

Correlation S1-RT

Correlation S2-RT

T I M E

T0 : Start Clock

400-1000

0

d’ modulation

criterion modulation

Stimuli & Paradigm(one trial)

SOA: 50-250

S2 : Mask/Primed (p = 1)

20

Masked(metacontrast)

S2

S1: Yes/No?

HitsFAMissesCR

100-200

S1 : Target/Prime (p = .2, .5, .8)

20

S1

7°

SOA: 50-250

S2 : Mask/Primed (p = 1)

20

NOT Masked

S2

22.5°

Waszak & Gorea (2004).

Of the 4 perceptual response categories, Hits & Misses are of particular interest:

They tell us about the motor behavior when the Obs. says he senses and does not sense the test stimulus (S1), hence establishing the relationship between perceptual and motor behavior*.

* RT for FA are not reliable indices as they have an unknown temporal origin; RT for CR are simply used as reference for the relevant RT.

The motor system appears to react if and only if

the stimulus is present and

the observer is “aware” of it (i.e. only for Hits).

The difference between RTs for Hits & Misses points against a full sensori-motor dissociation.

S1


S2

SPACE

4 SjsS1 13 ms

SOA variableS2 36 ms

300 trials / d’ / Obs

TIME

t20 ms 20 ms

SOA

variable

350

400

450

500

550

600

650

0 1 2 3 4 5

RT

(m

s)

AGp[S1] = .5

'ed [S1]

FA

MissesCR

Hits


350

400

450

500

550

600

650

0 1 2 3 4 5

TR

(ms)

350

400

450

500

550

600

650

0 1 2 3 4 5

TR

(ms)

350

400

450

500

550

600

650

0 1 2 3 4 5

TR

(ms)

350

400

450

500

550

600

650

0 1 2 3 4 5

TR

(ms)

350

400

450

500

550

600

650

0 1 2 3 4 5

TR

(ms)

350

400

450

500

550

600

650

0 1 2 3 4 5

TR

(ms)

'ed

Hits Misses FA CR

AG SD

.2

.5

p[S1]

.8

350

400

450

500

550

600

650

0 1 2 3 4 5

RT

(m

s)

350

400

450

500

550

600

650

0 1 2 3 4 5

RT

(m

s)

350

400

450

500

550

600

650

0 1 2 3 4 5

RT

(m

s)FW


Rank Correlations between RT and S1-, S2-onsets as a fct. of d’

c. Correct Rejections

-0.1

0.3

0.7

0 1 2 3 4 5

r tR

-tS

2

a. Hits

-0.1

0.3

0.7

0 1 2 3 4 5

r tR

-tS

1

&

r tR

-tS

2

'ed


TIME

Correlation S1-RT

Correlation S2-RT

S1

Speeded RT

S2

SPACE

S1 S2

SD

AG

FW

Sj

b. Misses

-0.1

0.3

0.7

0 1 2 3 4 5r

tR-t

S1

&

r

tR-t

S2

d. False Alarms

-0.1

0.3

0.7

0 1 2 3 4 5

r tR

-tS

2


Decision rule(implicit)

Sensory Input

Action


Verbal report

aware / not aware

?

Lateral Interactions(implicit)

ventral

dorsal


We’ve thus replaced the standard 2-pathways view…

Actio

n

with p

erce

ptual

awar

enes

s

A’. Layman’s view modified

…with a slightly modified layman’s view

Where does the discrepancy come from?(Aside from potential methodological problems in previous studies)

Common denominator: Most of the previous (whether simple or choice RT)

studies used 100% contrast targets whose ‘invisibility’ was ensured by strong

backward masking.

In contrast, our targets (S1) yielded maximum contrasts of about 20%. To

allow for higher target contrasts while keeping sensitivity constant, shorter

SOA-s (48 ms instead of an average of 162 ms) were used in a second series

of experiments; these entailed S1 contrasts around 30%.


350

400

450

0 1 2 3 4 5

RT

(m

s)

FW

p[S1] = .5

0 1 2 3 4 5

AG

p[S1] = .5

FA

MissesCR

Hits

'ed

TIME

t20 ms 20 ms

SOA

52 ms

S1


S2

SPACE

Fixed Motor Threshold

Variable Perceptual Criterion

Sensory Input

Action


Verbal report

aware / not aware

C. Gorea & Waszak (2004)

Lateral Interactions

ventral

dorsal

Actio

n

with p

erce

ptual

awar

enes

s

The temptation was strong to conclude (Gorea & Waszak, 2004)…

RT for Misses drop with d’ (or contrast) only for the masked condition!

RT appear to depend on Contrast rather than on d’!

Perceptual HITS

220

240

260

280

300

320

0.5 1 1.5 2 2.5

d'

RT

8%13%

27%

13%

8% 13%

27%

13%

Perceptual MISSES

0.5 1 1.5 2 2.5

d'

S1


S2 S1

S2

NOT Masked

S2

17 Obs300 trials / d’ / Obs

MaskedNOT Masked

Masked

NOT Masked

NOT Masked

Masked

S1

“prime”

S2

“mask”

t

SOA

13 ms 36 ms

52 ms

Waszak & Gorea, new experiments.

However, the data are more intricate than that…

S1


S2

NOT MASKED(6 Obs)

0 10 20 30 40

S1 Contrast (%)

2.6

3.33.2

.14

.451.2

2.1

d’

S1

S2

NOT Masked

S2

S1

“prime”

S2

“mask”

t

SOA

13 ms 36 ms

52 ms

MASKED(6 Obs)

-50

-40

-30

-20

-10

0

10

0 10 20 30 40

S1 Contrast (%)

.21

.32

.75

1.8

2.1

2.42.6

2.82.83.23.1 3.2

3.4 d’RT

ga

in r

el.

to C

R (

ms)

RTHITS-RTCR

RTMISS-RTCR

300-900 trials / C / Obs

HITS(6 Obs)

Not Masked


Masked

MISSES(6 Obs)

RTHITS-RTCR

RTMISS-RTCR


Not Masked


Masked


S1


S2 S1

S2

NOT Masked

S2

S1

“prime”

S2

“mask”

t

SOA

13 ms 36 ms

52 ms

0 1 2 3 4

d’S1

NOT MASKED(6 Obs)

-50

-40

-30

-20

-10

0

10

0 1 2 3 4

d’S1

RT

ga

in r

el.

to C

R (

ms)

MASKED(6 Obs)

RTHITS-RTCR

RTMISS-RTCR


RTHITS-RTCR

RTMISS-RTCR


HITS(6 Obs)

Not Masked


Masked

MISSES(6 Obs)

RTHITS-RTCR

RTMISS-RTCR


Not Masked


Masked




Sensory Input

Action


Verbal report

aware / not aware



ventral

dorsal

Actio

n

with p

erce

ptual

awar

enes

s

…And a simpler, one pathway SDT model can account for them. Instead of:…we propose:



Sensory Input

Action


Verbal report

aware / not aware

D. Current view


ventral

ventral

Actio

n

with p

erce

ptual

awar

enes

s

A conceptual model

40 140 240 340

TIME (ms)

S1


S2

TIME (ms)

RE

SP

ON

SE

(N

ois

e u

nit

s)

40 140 240 340

-2

-1

0

1

2

3

4

5

6

7

RTd’=1

Motor threshold

d’

Perc

eptu

al M

isse

s

S1

S2

NOT Masked

S2

S1

“prime”

S2

“mask”

t

SOA

Perc

eptu

al M

isse

s

S1

SOA

RTMask

S2

SOA

RT Mask

S2

RT d’=1

S1

S1

d’

-2

-1

0

1

2

3

4

5

6

7

-60 40 140 240 340 440

TIME (ms)

RE

SP

ON

SE

(N

ois

e u

nit

s)

-2

-1

0

1

2

3

4

5

6

7

-60 40 140 240 340 440

TIME (ms)

RE

SP

ON

SE

(N

ois

e u

nit

s)

S1

“prime”

S2

“mask”

t S1


S2S1

S2

NOT Masked

S2

RT d’=2RT

Mask RT d’=2RT

Mask

SOA SOA

Perc

eptu

al M

isse

s

Perc

eptu

al M

isse

s

d’

d’

Motor threshold

SOA

-2

-1

0

1

2

3

4

5

6

7

-60 40 140 240 340 440

TIME (ms)

RE

SP

ON

SE

(N

ois

e u

nit

s)

S1


S2

S1

“prime”

S2

“mask”

t

SOA

Perc

eptu

al M

isse

s

d’

SOA

1

7

240 340

TIME (ms)

S1

S2

NOT Masked

S2

RT d’=4RT

Mask-2

-1

0

40 140

RTMaskRT d’=4

Motor threshold

-60

RE

SP

ON

SE

(N

ois

e u

nit

s)

SOA6

2d’

Perc

eptu

al M

isse

s

3

4

5

S1


S2 S1

S2

NOT Masked

S2

S1

“prime”

S2

“mask”

t

SOA

13 ms 36 ms

52 ms

MASKED6 Obs

230

240

250

260

270

280

290

300

0 1 2 3 4

d’S1

RT

(m

s)

NOT MASKED6 Obs

0 1 2 3 4

d’S1

RTHITS-RTCR

RTMISS-RTCR


Model Fits

Motor Threshold.8

Sensory Input

A. Layman’s view

Action

with perceptual awareness

PerceptualDecision

Verbal report

aware / not aware


Decision ruleimplicit

Sensory Input

Actionwith or without

perceptual awareness

Verbal report

(aware / not aware)


?

Lateral Interactionsimplicit

ventral

dorsal

Fixed MotorThreshold

Variable PerceptualCriterion

Sensory Input



Verbal report

(aware / not aware)


Action

with perc

eptual a

wareness


dorsal

ventral

Fixed MotorThreshold

Variable PerceptualCriterion

Sensory Input



Verbal report

(aware / not aware)

D. Current view

Action

with perc

eptual a

wareness


ventral

dorsal

ventral

A one-pathway model with two distinct activation levels accounts for the observed perceptual-motor relationship under both masking and non-masking experimental conditions.

There is a fixed motor threshold ( 0.8) to be contrasted with a variable perceptual criterion).

The motor threshold is measured in noise () units as referred to the “absolute” perceptual detection “threshold” (i.e. in ref. to the internal noise).

RT for “unconscious” stimuli (i.e. Misses) depends on the reference (noise) level at which the perceptual task is performed;

as this reference level exceeds the motor threshold, the internal response associated with perceptual Misses also exceeds it and progressively contributes to shortening the RT.

TAKE-HOME MESSAGES (Part II)

PART III

RESPONSE TIME&

TEMPORAL ORDER JUDGMENTS

Time

C2 C1

Trigger-Delay PSS

PSS

RTC2 RTC1

Threshold

Inte

rnal

R

esp

on

se

SOA

PSS

p(S

1 p

erc

eiv

ed

fir

st)

slop

e

Temporal Order Judgments & RT

IFF RT is strictly dependent on the sensory signal (as it determines the TOJ), then the slope and the PSS of the TOJ -function should be direct indicators of the variance of the RT distributions and of their mean difference, respectively: this is a one-pathway sensorimotor model.

a b c

In contrast with previous studies: TOJ and RT were measured in the same trial; It was hence possible to assess RTs for correct & incorrect TOJs.

0

20

40

60

80

100

120

140

160

165 195 225 255 285 315 345 375 405 435 465 495 525

RT (ms)

Fre

qu

en

cy

RT

C2

C1

DG

PSS = 0 ms

DG

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-200 -100 0 100 200

SOA (ms)

%S

1 fi

rst

C1-C1

C2-C2

C1-C2

PSS = 49 ms

500 ms

500-1300 ms

SOA

~

1700 msRT

Left / Right

C1 C2 O1 O2

C1

C2

O1

O2

Stimuli & Paradigm(one trial)

DGC1-C1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC1-C1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

Identical

Different

-RTCorrect / Incorrect

DGC1-O1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC1-O1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

Correct

Incorrect

S1 1st

S2 1st

DGC1-O1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC1-O2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC1-O1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC1-O2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC1-C2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC1-C2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC2-O1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC2-O1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC2-O2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC2-O2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGO1-O2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGO1-O2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC1-C1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC2-C2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGO1-O1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGO2-O2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC1-C1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGC2-C2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGO1-O1

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

DGO2-O2

250

275

300

325

350

375

400

-100 -50 0 50 100SOA

RT

375

Identical Different

-RTCorrect / Incorrect -RT

Correct / Incorrect

4 SjsMean RT

R2 = 0.9174p << .0001

220

240

260

280

300

320

340

360

380

400

220 240 260 280 300 320 340 360 380 400

Predicted

Me

as

ure

d

DG

FC

PC

AG

4 SjsCorrect / Incorrect

R2 = 0.3543p < .0001

R2 = 0.1707p < .0001

220

240

260

280

300

320

340

360

380

400

220 240 260 280 300 320 340 360 380 400

Predicted

Me

as

ure

dCorrect DG

Correct FC

Correct PC

Correct AG

Inorrect DG

Inorrect FC

Inorrect PC

Inorrect AG

4 Sjs

0 vs. 32 df -RT fits

Correct-IncorrectTOJ data

R2 = 0.8415p << .0001

220

240

260

280

300

320

340

360

380

400

220 240 260 280 300 320 340 360 380 400

Measured -RT in "single" (ms)

Fit

ted

-R

T (

ms

)

4 Sjs


Mean TOJ data

R2 = 0.8273p << .0001

220

240

260

280

300

320

340

360

380

400

220 240 260 280 300 320 340 360 380 400


Fit

ted

-R

T (

ms

)

a b

4 Sjs


Correct-IncorrectTOJ data

R2 = 0.8568p << .0001

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100


Fit

ted

-R

T (

ms

)

4 Sjs


Mean TOJ data

R2 = 0.7094p < .0001

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100


Fit

ted

-R

T (

ms

)

Red: C1Green: C2Blue: O1

Brown: O2

c d

4 SjsPSSDG

R2 = 0.5893p = .037

AG

R2 = 0.6511p = .026

-150

-100

-50

0

50

100

150

0 50 100 150RT (from "single" - ms)

PS

S (

fro

m T

OJ

- m

s)

DG

FC

PC

AG

4 SjsTOJ -fct Slope

DG

R2 = 0.5337p = .008

AG

R2 = 0.8899p << .0001

0

20

40

60

80

100

120

140

160

180

200

0 20 40 60 80 100 120 140 160 180 200-RT (from "single" - ms)

Slo

pe

TO

J

-fu

nc

tio

n (

ms

)

DG

FC

PC

AG

a b

RT

PS

S

RT

R

M

PC

CHigh

PT1 PT2

RT1

PSS

RT

RT2

M <

PC

M > PCM

= P C

R

t

CLow

For 2 out of 4 Obs, RT to single stimuli also predict their TOJ behavior (PSS and slopes of the TOJ -fct.); their behavior is hence compatible with a one pathway model (no sensorimotor dissociation).

For these 2 Obs, the relationship between RT-differences and PSS also suggests the existence of a motor threshold distinct from and higher than the perceptual criterion.

The behavior of the remaining 2 Obs does not follow any coherent pattern; it is likely that it was perturbed by perceptual response strategy factors, possibly under the influence of “transient Troxler masking” (Kanai & Kamitani, 2003).

Such factors may account for the various inconsistencies in the RT-TOJ literature.

TAKE-HOME MESSAGES (Part III)

The relationship between simple Response Times and perceptual states (Hits, FA, etc. and Temporal Order Judgments) can be accounted for by a single (one-path) system where two distinct decisions are made on the same incoming information.

The decision to act is based on a hard-wired threshold worth about 1 of the internal noise; the perceptual criterion is context dependent (in line with SDT).

TOJ data not complying with this model may reflect variable perceptual response strategies.

GENERAL CONCLUSIONS

THANK YOU

TIME SPACE

OR

OR

What do the Obs. compare?

vs.A

Either, both?C

vs.B

=<

=>

Most likely both(also depending on SOA)

on the relationship between motor and perceptual behavior – a sdt framework andrei gorea with...

Documents

aware slide

rt perceptual states

background slide

perceptual behavior

france slide

rt t i

simple rt correlation

simple rt manual