2nd level analysis – design matrix, contrasts and inference deborah talmi & sarah white

2nd level analysis – 2nd level analysis – design matrix, contrasts and design matrix, contrasts and

inference inference

Deborah TalmiDeborah Talmi

& Sarah White& Sarah White

OverviewOverview

Fixed, random, and mixed modelsFixed, random, and mixed modelsFrom 1From 1stst to 2 to 2ndnd level analysis level analysis22ndnd level analysis: 1-sample t-test level analysis: 1-sample t-test22ndnd level analysis: Paired t-test level analysis: Paired t-test22ndnd level analysis: 2-sample t-test level analysis: 2-sample t-test22ndnd level analysis: F-tests level analysis: F-testsMultiple comparisonsMultiple comparisons

Fixed effectsFixed effects Fixed effect:Fixed effect: A variable with fixed values A variable with fixed values

E.g. levels of an experimental variable. E.g. levels of an experimental variable. Random effect:Random effect: A variable with values that can A variable with values that can

vary. vary. E.g. the effect ‘list order’ with lists that are randomized E.g. the effect ‘list order’ with lists that are randomized

per subjectper subject The effect ‘Subject’The effect ‘Subject’ can be described as either can be described as either

fixed or randomfixed or random Subjects in the sample are fixedSubjects in the sample are fixed Subjects are drawn randomly from the populationSubjects are drawn randomly from the population Typically treated as a random effect in behavioural Typically treated as a random effect in behavioural

analysisanalysis

Fixed effects analysisFixed effects analysis

• The factor ‘subject’ treated like other experimental variable in the design matrix. • Within-subject variability across condition onsets represented across rows. • Between-subject variability ignored• Case-studies approach: Fixed-effects analysis can only describe the specific sample but does not allow generalization.

Experimental conditions

S1

S2

S3

S4

S5

Constants

Regressors Covariates

Random effects analysisRandom effects analysis

Generalization to the population requires Generalization to the population requires taking between-subject variability into taking between-subject variability into account.account.The question:The question: Would a new subject drawn Would a new subject drawn

from this population show any significant from this population show any significant activity?activity?

Mixed modelsMixed models: the experimental factors : the experimental factors are fixed but the ‘subject’ factor is random.are fixed but the ‘subject’ factor is random.Mixed models take into account both within- Mixed models take into account both within-

and between- subject variability.and between- subject variability.

OverviewOverview


Relationship between 1Relationship between 1stst & 2 & 2ndnd levels levels

11stst-level analysis:-level analysis: Fit the model for each subject Fit the model for each subject using different GLMs for each subject.using different GLMs for each subject. Typically, one design matrix per subjectTypically, one design matrix per subject

Define the effect of interest for each subjectDefine the effect of interest for each subject with a contrast vector. with a contrast vector.

The contrast vector produces a contrast The contrast vector produces a contrast image image containing the contrast of the parameter containing the contrast of the parameter estimates at each voxel.estimates at each voxel.

22ndnd-level analysis: -level analysis: Feed the contrast images Feed the contrast images into a GLM that implements a statistical test.into a GLM that implements a statistical test.

11stst level X values level X values

Convolved with HRF

Convolution with the HRF changes the onsets Convolution with the HRF changes the onsets we enter (1,0) to a gradient of valueswe enter (1,0) to a gradient of values

X values are then ordered on the x-axis to X values are then ordered on the x-axis to predict BOLD data on the Y axis.predict BOLD data on the Y axis.

11stst level parameter estimate level parameter estimate

= ŷ, predicted value

intercept

ŷ = ax + b

ε = residual error

= y i , true value

slope(beta)

Mean activation

Y=data

Contrasts = combination of beta valuesContrasts = combination of beta values

Vowel =23.356.con =23.356

Vowel beta =23.356Tone beta2 =14.4169.con =8.9309

Vowel - baseline Tone - baseline

Vowel - baseline

1 1

Vowel - baseline Tone - baseline Vowel - Tone

• Contrast images for the two classes of stimuli vs. baseline and vs. each other(linear combination of all relevant betas)

Difference from behavioral analysisDifference from behavioral analysis

The ‘1The ‘1stst level analysis’ typical to behavioural data level analysis’ typical to behavioural data is relatively simple: is relatively simple: A single number: categorical or frequency A single number: categorical or frequency A summary statistic, resulting from a simple model of A summary statistic, resulting from a simple model of

the data, typically the mean.the data, typically the mean. SPM 1SPM 1stst level is an extra step in the analysis, level is an extra step in the analysis,

which models the response of one subject. The which models the response of one subject. The statistic generated (statistic generated (ββ) then taken forward to the ) then taken forward to the GLM.GLM. This is possible because This is possible because ββs are normally distributed.s are normally distributed.

A series of 3-D matrices (A series of 3-D matrices (ββ values, error terms) values, error terms)

Both use the GLM model/tests and a Both use the GLM model/tests and a similar SPM machinerysimilar SPM machinery

Both produce design matrices.Both produce design matrices. The columns in the design matrices represent The columns in the design matrices represent

explanatory variables: explanatory variables: 11stst level: All conditions within the experimental design level: All conditions within the experimental design 22ndnd level: The specific effects of interest level: The specific effects of interest

The rows represent observations: The rows represent observations: 11stst level: Time (condition onsets); within-subject level: Time (condition onsets); within-subject

variabilityvariability 22ndnd level: subjects; between-subject variability level: subjects; between-subject variability

Similarities between 1st & 2nd levelsSimilarities between 1st & 2nd levels

Similarities between 1st & 2nd levelsSimilarities between 1st & 2nd levels

The same tests can be used in both levels (but The same tests can be used in both levels (but the questions are different)the questions are different)

.Con images: output at 1.Con images: output at 1stst level, both input and level, both input and output at 2output at 2ndnd level level

There is typically only 1 1There is typically only 1 1stst-level design matrix -level design matrix per subject, but multiple 2per subject, but multiple 2ndnd level design level design matrices for the group – one for each statistical matrices for the group – one for each statistical test.test.

Multiple 2Multiple 2ndnd level analyses level analyses

1-sample t-tests: 1-sample t-tests: Vowel vs. baseline [1 0]Vowel vs. baseline [1 0]Tone vs. baseline [0 1]Tone vs. baseline [0 1]Vowel > Tone [1 -1]Vowel > Tone [1 -1]Vowel or tone >baseline [1 1]Vowel or tone >baseline [1 1]

Vowel Tone

OverviewOverview

Fixed, random, and mixed modelsFixed, random, and mixed models From 1From 1stst to 2 to 2ndnd level analysis level analysis 22ndnd level analysis: 1-sample t-test level analysis: 1-sample t-test

MaskingMaskingCovariatesCovariates

22ndnd level analysis: Paired t-test level analysis: Paired t-test 22ndnd level analysis: 2-sample t-test level analysis: 2-sample t-test 22ndnd level analysis: F-tests level analysis: F-tests Multiple comparisonsMultiple comparisons

1-Sample t-test1-Sample t-test

Enter 1 .con image per subject

All subjects weighted equally – all modeled with a ‘1’

22ndnd level design matrix for 1-sample t-test level design matrix for 1-sample t-test

Values from the design matrix

Y = data (parameter estimates)

1

ŷ = 1*x +β 0

The question: is mean activation significantly greater than zero?

Estimation and resultsEstimation and results

1-Sample t-test figures 1-Sample t-test figures

These data (e.g. beta values) are available in the workspace – useful to create more complex figures

Statistical inference: imaging vs. Statistical inference: imaging vs. behavioural data behavioural data

Inference of imaging data uses some of Inference of imaging data uses some of the same statistical tests as used for the same statistical tests as used for analysis of behavioral data:analysis of behavioral data: t-tests, t-tests, ANOVAANOVAThe effect of covariates for the study of The effect of covariates for the study of

individual-differencesindividual-differencesSome tests are more typical in imaging:Some tests are more typical in imaging:

Conjunction analysisConjunction analysisMultiple comparisons poses a greater Multiple comparisons poses a greater

problem in imaging problem in imaging

MaskingMasking

Implicit mask: the default, excluding voxels with ‘NaN’ or ‘0’ values

Threshold masking: Images are thresholded at a given value and only voxels at which all images exceed the threshold

Explicit mask: only user-defined voxels are included in the analysis

Explicit masksExplicit masks

Group maskSingle subject mask

ROI mask

Segmentation of structural images

Covariates in 1-Sample t-testCovariates in 1-Sample t-test An additional regressor in the design

matrix specifying subject-specific information (e.g. age).

Nuisance covariates, covariate of interest: Included in the model in the same

way. Nuisance: Contrast [1 0] focuses on

mean, partialing out activation due to a variable of no interest

Covariate of interest: contrast [0 1] focuses on the covariate. The parameter estimate represents the magnitude of correlation between task-specific activations and the subject-specific measure.

Covariate optionsCovariate options

Entering single number per subject.

Centering: the vector will be mean- corrected

Covariate resultsCovariate results

Covariate

Parameter estimate

Slope: Parameter estimate of 2nd level covariate

Mean activation

Centred covariate mean

OverviewOverview


Factorial designFactorial design First…back to first level analysis First…back to first level analysis

here, 2 factors with 2/3 levels making 6 conditionshere, 2 factors with 2/3 levels making 6 conditions For each subject we could create a number of effects of For each subject we could create a number of effects of

interest, eg.interest, eg. each condition separatelyeach condition separately each level separatelyeach level separately contrast between levels within a factorcontrast between levels within a factor interaction between factorsinteraction between factors

A1

B1

A2

B2

1 2

4 5

B3

3

6

[1,-1,0,-1,1,0,0]

1 2 3 4 5 6

[1,1,1,-1,-1,-1,0]

1 2 3 4 5 6

[1,0,0,0,0,0,0]

1 2 3 4 5 6

[1,1,1,0,0,0,0]

1 2 3 4 5 6

Paired t-testsPaired t-tests

This is when we start being interested in This is when we start being interested in contrasts at 2nd levelcontrasts at 2nd level

Within groupWithin group between subject variance is greater than within between subject variance is greater than within

subject variancesubject variance better use of time to have more subjects for shorter scanning better use of time to have more subjects for shorter scanning

slots than vice versaslots than vice versa

Paired t-testsPaired t-tests This is when we start being interested in contrasts at 2nd This is when we start being interested in contrasts at 2nd

levellevel Within groupWithin group

whether, across subjects, one effect of interest (A1) is whether, across subjects, one effect of interest (A1) is significantly greater than another effect of interest (A2)significantly greater than another effect of interest (A2)

contrast vector [1,-1]contrast vector [1,-1] one-tailed / directionalone-tailed / directional

asks specific Q, eg. is A1>A2?asks specific Q, eg. is A1>A2?

You could equally do this same analysis byYou could equally do this same analysis bycreating the contrast at the 1st level analysiscreating the contrast at the 1st level analysisand then running a one-sample t-test at theand then running a one-sample t-test at the2nd level2nd level

A1 A2

A1

B1

A2

B2

1 2

4 5

B3

3

6

Factorial designFactorial design

This is when we start being interested in This is when we start being interested in contrasts at 2nd levelcontrasts at 2nd level

Within groupWithin group whether, across subjects, one effect of interest (A1) is whether, across subjects, one effect of interest (A1) is

significantly greater than another effect of interest (A2)significantly greater than another effect of interest (A2) contrast vector [1,1,1,-1,-1,-1]contrast vector [1,1,1,-1,-1,-1] one-tailed / directionalone-tailed / directional

asks specific Q, eg. is A1>A2?asks specific Q, eg. is A1>A2?

A1

B1

A2

B2

1 2

4 5

1 2 3 4 5 6

B3

3

6

Factorial designFactorial design Conjunction analysisConjunction analysis Simple example within groupSimple example within group

whether, across subjects, those voxels significantly activated in whether, across subjects, those voxels significantly activated in one contrast are also significantly activated in anotherone contrast are also significantly activated in another

eg. whether the difference between A1 and A2 is significant eg. whether the difference between A1 and A2 is significant across all three conditions B1, B2 and B3across all three conditions B1, B2 and B3

contrast vector ???contrast vector ??? [1,1,1,-1,-1, -1] given [1,1,1,-1,-1, -1] given

[1,0,0,-1,0,0] & [0,1,0,0,-1,0] and [0,0,1,0,0,-1][1,0,0,-1,0,0] & [0,1,0,0,-1,0] and [0,0,1,0,0,-1] basically testing whether there is a mainbasically testing whether there is a main

effect in the absence of an interactioneffect in the absence of an interaction

A1

B1

A2

B2

1 2

4 5

1 2 3 4 5 6

B3

3

6

OverviewOverview


Two sample t-testsTwo sample t-tests

This is a contrast againThis is a contrast again but can’t be done at the 1st level of analysis this timebut can’t be done at the 1st level of analysis this time

Between groupsBetween groups both groups must have same design matrixboth groups must have same design matrix

Two sample t-testsTwo sample t-tests This is a contrast againThis is a contrast again

but can’t be done at the 1st levelbut can’t be done at the 1st level Between groupsBetween groups

whether, across conditions, the difference between two groups whether, across conditions, the difference between two groups of subjects (M & F) is significantof subjects (M & F) is significant

one-tailed / directionalone-tailed / directional asks specific Q, eg. is M>F?asks specific Q, eg. is M>F?

contrast vector [1,-1]contrast vector [1,-1]

Unlike the paired samples t-test, there’s noUnlike the paired samples t-test, there’s noother way to do this analysis as you haven’tother way to do this analysis as you haven’tbeen able to collapse data across subjectsbeen able to collapse data across subjectsbeforebefore

A1

B1

A2

B2

1 2

4 5

B3

3

6

A1

B1

A2

B2

1 2

4 5

B3

3

6

M F

Factorial designFactorial design

This is a contrast againThis is a contrast again but can’t be done at the 1st levelbut can’t be done at the 1st level

Between groupsBetween groups whether, across conditions, the difference between whether, across conditions, the difference between

two groups of subjects (M & F) is significanttwo groups of subjects (M & F) is significant one-tailed / directionalone-tailed / directional

asks specific Q, eg. is M>F?asks specific Q, eg. is M>F? contrast vectorcontrast vector

[1,1,1,1,1,1,-1,-1,-1 ,-1 ,-1 ,-1][1,1,1,1,1,1,-1,-1,-1 ,-1 ,-1 ,-1]

M F123456 123456

A1

B1

A2

B2

1 2

4 5

B3

3

6

A1

B1

A2

B2

1 2

4 5

B3

3

6

OverviewOverview


F-testsF-tests This is for multiple contrastsThis is for multiple contrasts Within and between groupsWithin and between groups

whether, across conditions and/orwhether, across conditions and/orsubjects, a number of differentsubjects, a number of differentcontrasts are significantcontrasts are significant

gives differences in both directions (+ve & -ve)gives differences in both directions (+ve & -ve) equivalent to lots of 2-tailed t-testequivalent to lots of 2-tailed t-test asks general question: A1 asks general question: A1 ≠≠ A2 A2

contrast vector for main effect of A:contrast vector for main effect of A:[1,-1,0,0][1,-1,0,0][0,0,1,-1][0,0,1,-1]

A1

B1

A2

B2

1 2

4 5

B3

3

6

A1

B1

A2

B2

1 2

4 5

B3

3

6

M FA1 A2 A1 A2



gives differences in both directions (+ve & -ve)gives differences in both directions (+ve & -ve) equivalent to lots of 2-tailed t-testequivalent to lots of 2-tailed t-test asks general question: A1 asks general question: A1 ≠≠ A2 A2

contrast vector for main effect of A:contrast vector for main effect of A:[1,0,0,-1,0,0,0,0,0,0,0,0][1,0,0,-1,0,0,0,0,0,0,0,0][0,1,0,0,-1,0,0,0,0,0,0,0][0,1,0,0,-1,0,0,0,0,0,0,0][0,0,1,0,0,-1,0,0,0,0,0,0][0,0,1,0,0,-1,0,0,0,0,0,0][0,0,0,0,0,0,1,0,0,-1,0,0][0,0,0,0,0,0,1,0,0,-1,0,0][0,0,0,0,0,0,0,1,0,0,-1,0][0,0,0,0,0,0,0,1,0,0,-1,0][0,0,0,0,0,0,0,0,1,0,0,-1][0,0,0,0,0,0,0,0,1,0,0,-1]

A1

B1

A2

B2

1 2

4 5

B3

3

6

A1

B1

A2

B2

1 2

4 5

B3

3

6

M F123 456 123 456



gives differences in both directions (+ve & -ve)gives differences in both directions (+ve & -ve) equivalent to lots of 2-tailed t-testequivalent to lots of 2-tailed t-test asks general question: M asks general question: M ≠ F≠ F

contrast vector for main effect of sex:contrast vector for main effect of sex:[1,0,0,1,0,0,0,0,0,0,0,0][1,0,0,1,0,0,0,0,0,0,0,0][0,1,0,0,1,0,0,0,0,0,0,0][0,1,0,0,1,0,0,0,0,0,0,0][0,0,1,0,0,1,0,0,0,0,0,0][0,0,1,0,0,1,0,0,0,0,0,0][0,0,0,0,0,0,-1,0,0,-1,0,0][0,0,0,0,0,0,-1,0,0,-1,0,0][0,0,0,0,0,0,0,-1,0,0,-1,0][0,0,0,0,0,0,0,-1,0,0,-1,0][0,0,0,0,0,0,0,0,-1,0,0,-1][0,0,0,0,0,0,0,0,-1,0,0,-1]

A1

B1

A2

B2

1 2

4 5

B3

3

6

A1

B1

A2

B2

1 2

4 5

B3

3

6

M F123 456 123 456



gives differences in both directions (+ve & -ve)gives differences in both directions (+ve & -ve) equivalent to lots of 2-tailed t-testequivalent to lots of 2-tailed t-test asks general question: M(A1-A2)asks general question: M(A1-A2) ≠ F(A1-A2) ≠ F(A1-A2)

contrast vector for interaction between A and sex:contrast vector for interaction between A and sex:[1,0,0,-1,0,0,0,0,0,0,0,0][1,0,0,-1,0,0,0,0,0,0,0,0][0,1,0,0,-1,0,0,0,0,0,0,0][0,1,0,0,-1,0,0,0,0,0,0,0][0,0,1,0,0,-1,0,0,0,0,0,0][0,0,1,0,0,-1,0,0,0,0,0,0][0,0,0,0,0,0,-1,0,0,1,0,0][0,0,0,0,0,0,-1,0,0,1,0,0][0,0,0,0,0,0,0,-1,0,0,1,0][0,0,0,0,0,0,0,-1,0,0,1,0][0,0,0,0,0,0,0,0,-1,0,0,1][0,0,0,0,0,0,0,0,-1,0,0,1]

A1

B1

A2

B2

1 2

4 5

B3

3

6

A1

B1

A2

B2

1 2

4 5

B3

3

6

M F123456 123456

OverviewOverview

Fixed, random, and mixed modelsFixed, random, and mixed modelsFrom 1From 1stst to 2 to 2ndnd level analysis level analysis22ndnd level analysis: 1-sample t-test level analysis: 1-sample t-test22ndnd level analysis: Paired t-test level analysis: Paired t-test22ndnd level analysis: 2-sample t-test level analysis: 2-sample t-test22ndnd level analysis: ANOVA level analysis: ANOVAMultiple comparisonsMultiple comparisons

Multiple comparisonsMultiple comparisons

we’re still doing these comparisons for each we’re still doing these comparisons for each voxel involved in the analysis (even though voxel involved in the analysis (even though we’ve collapsed across time) -> lots of we’ve collapsed across time) -> lots of comparisonscomparisons

also multiple contrastsalso multiple contrasts problem of false positivesproblem of false positives correction for multiple comparisonscorrection for multiple comparisons cf talk on random field theorycf talk on random field theory

ReferencesReferences

Previous MFD presentationsPrevious MFD presentationsSPM5 Manual, The FIL Methods Group SPM5 Manual, The FIL Methods Group

(2007)(2007)Poline, Kherif, Pallier & Penny, Chapter 9, Poline, Kherif, Pallier & Penny, Chapter 9,

Statistical Parametric Mapping (2007)Statistical Parametric Mapping (2007)Penny & Holmes, Chapter 12, Human Penny & Holmes, Chapter 12, Human

Brain function (2Brain function (2ndnd edition) edition)

2nd level analysis – design matrix, contrasts and inference deborah talmi & sarah white

Documents

nd level analysis

fixed effects analysis

fixedeffects analysis

nd levels

behavioural analysis

sample ttest

effect subject

level x values