experimental design and efficiency edoardo zamuner nicholas wright methods for dummies 17 th...

Experimental design and efficiency

Edoardo Zamuner

Nicholas Wright

Methods for Dummies

17th December 2008

Part I: Experimental design

Asking the Right Question

The Paradigm DesignWhen you design an experiment ask yourself:

1. What do you hope to find about the brain?

2. What would the finding tell you about the cognitive/perceptual process/es

involved?

3. Has the experiment already been done?

4. Would the experiment add anything to what is already known from other

techniques?

5. Could the same question be asked more easily and cheaply with other

techniques?

6. What are the possible confounds?

7. Can you control for those confounds?

What is Experimental Design?

• The basic principle of experimental design in fMRI is to manipulate the subject’s experience and behaviour in some way that is likely to produce a functionally specific neurovascular response.

• What can we manipulate?– Stimulus kind and properties;– Stimulus timing;– Subject instructions.

• What are the goals of experimental design?– To test specific hypotheses

Goals of the Experimental Design

• What is active?

• How does its activity change over time?

-8 -6 -4 -2 0 2 4 6 8 10 12 14 16

Taxonomy of Experimental Designs

1. CategoricalA. Subtraction - Additive factors and pure insertion

B. Conjunction - Testing multiple hypotheses

2. FactorialA. Categorical - Interactions and pure insertion

- Adaptation, modulation and dual-task inference

B. Parametric - Linear and nonlinear interactions

- Psychophysiological Interactions

3. ParametricA. Linear - Cognitive components and dimensions

B. Nonlinear - Polynomial expansions

Categorical Design

Categorical design: it compares the brain activation in one task to that

in another task.

Example:

Stimulus: 30s epoch; visual presentation of 12 common nouns.

Tasks: decide for each noun whether it refers to an animate or inanimate object.

goat bucket

Factorial Design

Factorial design: it combines two or more factors within a task and looks

at the effect of one factor upon the other/s.

Example:

Tasks:

Condition A: decide for each noun whether it refers

to an animate or inanimate object.

Condition B: decide whether the nouns are written

in CAPITAL or lower case letters.

goat bucket

GOAT bucket

Parametric Design

Parametric design: the same task is presented throughout the experiment

but some continuously variable parameter of the task is manipulated.

Example: Semantic task repeated for 5 minutes.

The interval between consecutive stimuli (words) vary from 10s at the

start to 1s at the end.

Assumption: as the task becomes more difficult blood flow to the regions specialised for semantic analysis will increase.

Stimulus Presentation Strategies

fMRI Design Types

1) Blocked Designs

2) Event-Related Designsa) Periodic Single Trial

b) Jittered Single Trial

c) Staggered Single Trial

3) Mixed Designsa) Combination blocked/event-related

b) Variable stimulus probability

Blocked Design

• It involves presenting two conditions – an activation (A) condition and a baseline (B) condition. Each condition is presented for an identical epoch of time.

Task A Task B Task A Task B Task A Task B Task A Task B

Task A Task BREST REST Task A Task BREST REST

Choosing Length of Blocks

• Longer blocks allow for stability of extended patterns of

brain activation.

• Shorter blocks allow for more transitions between tasks.

– Task-related variability increases with increasing

numbers of transitions

Pros and Cons of Blocked Design

Pros:• Avoid rapid task-switching (e.g. patients);• Fast and easy to run; • Good signal to noise ratio.

Cons:• Expectation;• Habituation;• Signal drift;• Poor choice of baseline may preclude meaningful conclusions;• Many tasks cannot be conducted repeatedly.

Event-Related Design

• It allows different trials or stimuli to be presented in arbitrary sequences.

time

Pros and Cons of Event-Related Design

Pros:

• Eliminate predictability of block designs (e.g. expectation);

• Can look at novelty and priming;

• Can look at temporal dynamics of response.

Cons:• More complex design and analysis (esp. timing and baseline issues).

Acknowledgements

Chi-Hua Chen

References

E.T. Bullmore & J. Suckling (2001), “Functional Magnetic Resonance Imaging”, International Review of Psychiatry 13: 24-33.

P. Jezzard, P.M. Matthews, and S.M. Smith (eds), (2001), Functional MRI. An Introduction to Methods, Oxford: OUP.

S.A. Huettel, A.W. Song, G. McCarthy, (2003), Magnetic Resonance Imaging, Sunderland MA: Sinauer Associate, Inc.

R.S.J. Frackowiak, K.J. Friston, C.D. Frith, R.J. Dolan, C.J. Price, S. Zeki, J. Ashburner, and W. Penny, (2004), Human Brain Function, Elsevier Academic Press.

Part II: Example factorial design in SPM

• A – Low attentional load, motion• B – Low attentional load, no motion• C – High attentional load, motion• D – High attentional load, no motion

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTIONLoad task Rees, Frith & Lavie (1997)

Terminology

• Simple main effects

• Main effects

• Interaction terms

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

SIMPLE MAIN EFFECTS

• A – B: Simple main effect of motion (vs. no motion) in the context of low load

• B – D: Simple main effect of low load (vs. high load) in the context of no motion

• D – C: ?• Simple main effect of no

motion (vs. motion) in the context of high load

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

OR

The inverse simple main effect of motion (vs. no motion) in the Context of high load

MAIN EFFECTS

• (A + B) – (C + D): • the main effect of low load

(vs. high load) irrelevant of motion

Main effect of load

• (A + C) – (B + D): ?• The main effect of motion (vs.

no motion) irrelevant of load Main effect of motion

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

INTERACTION TERMS

• (A - B) – (C - D): • the interaction effect of

motion (vs. no motion) greater under low (vs. high) load

• (B - A) – (D - C): ?• the interaction effect of no

motion (vs. motion) greater under low (vs. high) load

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

Factorial design in SPM5

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

• How do we enter these effects in SPM5?

• Simple main effect of motion in the context of low load:

• A vs. B or (A – B)A B C D

[1 -1 0 0]

Factorial design in SPM5

• Main effect of low load: • (A + B) – (C + D)

• Interaction term of motion greater under low load:

• (A – B) – (C – D)

A B C D

A B C D

[1 -1 -1 1]

[1 1 -1 -1]

Factorial design in SPM5

• Interaction term of motion greater under low load:

• (A – B) – (C – D) A B C D

Part III: Design efficiency

• A key question:• How to design fMRI experiments that are sensitive

to (efficient for) a specific hypothesis (i.e. a planned statistical comparison or “contrast”)?

Background: general advice from Rik Henson

1. Scan for as long as possible (incr # vol incr df). Usually for gp results stat power incr more by # subj than # vol per subj.

2. Try not to unnecessarily break scanning into runs.

3. Do not contrast trials far apart in time (low freq noise).

4. Randomise either the order or SOA of trials close together in time (efficiency).

What is efficiency? Ways to think about it

• How to design fMRI experiments that are sensitive to (efficient for) a specific hypothesis (i.e. a planned statistical comparison or “contrast”)

• Ways to think about efficiency:1. Mathematical (statistical)

2. Signal processing

3. Correlations between regressors – not covered today, see Rik Henson’s website.

What is “efficiency”? A mathematical perspective• GLM: Y = X +

• T-statistic for a given contrast: T = cT / var(cT )

• Want to optimise experimental design– minimise the standard error of a contrast (i.e. denominator above)– If we assume that noise variance (2) is unaffected by changes in X, this is

broadly equivalent to maximising the efficiency, e, of a contrast given by

e (c, X) = (2 cT XTX)-1 c)-1

• So,– We can influence e (a priori) by the spacing and sequencing of

epochs/events in our design matrix– e is specific for a given contrast!

Different designs

• If there is only a single event type, we can parameterise any design using:

1. SOAmin (min time between events)2. p(t) (prob of event occurring at each t

• Deterministic design: p(t)=1, low efficiency

• Stationary stochastic: p(t)<1 but constant over time

• Dynamic stochastic: p(t) is a function of time. Extreme is boxcar.

Multiple event types

• With randomised designs, optimal SOA for With randomised designs, optimal SOA for differential effect (A-B) is minimal SOA (>2 differential effect (A-B) is minimal SOA (>2 seconds, and assuming no saturation), whereas seconds, and assuming no saturation), whereas optimal SOA for main effect (A+B) is 16-20s.optimal SOA for main effect (A+B) is 16-20s.

Background: terminology

• Trial: Replications of a condition.• ITI (Inter-trial Interval)• Components: A trial consists of one or more components,

that may be:– “events” or “impulses” (brief bursts of neural activity)– “epochs” (periods of sustained neural activity)

• SOA (Stimulus Onset Asynchrony): The time between the onsets of components.

• ISI (Inter-stimulus Interval): Time between offset of one component and onset of next.

Background: the BOLD impulse response (IR)

• Typical BOLD response to an “impulse” (brief burst) of neural activity.

• Generally in efMRI not necessary to use long SOAs to allow return to baseline (however, note issues of linearity).

Signal processing perspective

• Signal processing is the analysis, interpretation, and manipulation of signals. Processing of such signals includes: filtering, storage and reconstruction, separation of information from noise, compression and feature extraction (Wikipedia, Dec 2008).

• Consider ability to detect the BOLD response to a single event-type versus baseline.

Maximising efficiency from the signal processing perspective

• Basic idea behind maximising efficiency is to maximise the "energy" of the predicted fMRI timeseries.

• This is simply the sum of squared signal values at each scan. It is also proportional to the variance of the signal.

• Therefore, to be best able to detect the signal in the presence of background noise, we want to maximise the variability of that signal. A signal that varies little will be difficult to detect.

An inefficient design v. more efficient design

• Inefficient design (SOA 4s): Overall signal is high, but variance is low. The majority of the stimulus energy will be lost after high pass filtering.

• More efficient design: A “stochastic design” in which min SOA 4s but only 50% probability of event each 4s. Only half as many stimuli, but much larger variability in the signal (which we know!).

Boxcar - even more efficiency!• Systematic SOA variation (e.g. runs of trials & runs of “null events”).

• Why is this even more efficient? Consider Fourier transforms

• Regard the IR as a “filter” that will “pass” low freq but reduce high freq.

• In current example the some higher-freq harmonics reduced but fundamental freq remains!

Design efficiency: Conclusions

• Optimal design for one contrast may not be optimal for another Optimal design for one contrast may not be optimal for another • Blocked designs generally most efficient (with short SOAs, given optimal block Blocked designs generally most efficient (with short SOAs, given optimal block

length is not exceeded)length is not exceeded)• However, However, psychological efficiencypsychological efficiency often dictates intermixed designs, and often often dictates intermixed designs, and often

also sets limits on SOAsalso sets limits on SOAs• With randomised designs, optimal SOA for differential effect (A-B) is minimal With randomised designs, optimal SOA for differential effect (A-B) is minimal

SOA (>2 seconds, and assuming no saturation), whereas optimal SOA for SOA (>2 seconds, and assuming no saturation), whereas optimal SOA for main effect (A+B) is 16-20smain effect (A+B) is 16-20s

• Inclusion of null events improves efficiency for main effect at short SOAs (at Inclusion of null events improves efficiency for main effect at short SOAs (at cost of efficiency for differential effects)cost of efficiency for differential effects)

• If order constrained, intermediate SOAs (5-20s) can be optimal If order constrained, intermediate SOAs (5-20s) can be optimal • If SOA constrained, pseudorandomised designs can be optimal (but may If SOA constrained, pseudorandomised designs can be optimal (but may

introduce context-sensitivity)introduce context-sensitivity)

Useful links and thanks

• http://imaging.mrc-cbu.cam.ac.uk/imaging/DesignEfficiency

• Chapter 40, Human Brain Mapping, 2004• SPM course 2008 (Christian Ruff’s slides)• Tali Sharot’s slides from MfD 2007