decoding representational spaces with multivariate pattern ...€¦ · decoding representational...
TRANSCRIPT
![Page 1: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/1.jpg)
Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data
Jim HaxbyCenter for Cognitive Neuroscience, Dartmouth College
Center for Mind/Brain Sciences (CIMeC), University of Trento
![Page 2: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/2.jpg)
Hyperalignment Swaroop Guntupalli Post-doctoral fellow
2
Haxby Lab Analysis of similarity structure, representation of biological classes Andy Connolly Post-doctoral fellow
Yaroslav Halchenko Research scientist
Face & person perception M Ida Gobbini Associate professor Ricercatrice, U Bologna
Person perception Dylan Wagner Post-doctoral fellow
Action representation, computational methods Nick Oosterhof Post-doctoral fellow
NeuroDebian
Attention Sam Nastase Graduate student
![Page 3: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/3.jpg)
Hyperalignment Swaroop Guntupalli Post-doctoral fellow
![Page 4: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/4.jpg)
With help from
Peter RamadgeElectrical EngineeringPrinceton University
Mert Rory Sabuncunow at MGH
Bryan ConroyPhilips Research
Alex LorbertSuperfish, Israel
and EE grad students, past and present
Hao XuGoogle
Cameron Chencurrent
![Page 5: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/5.jpg)
• Neural decoding: understanding representational spaces
• Statement of the problem: capturing fine-grained distinctions in a common model
• Conceptual framework: high-dimensional representational spaces
• Deriving the common space and individual transformation matrices with hyperalignment
• Validation
• Conclusions
Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data
5
![Page 6: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/6.jpg)
• Neural decoding: understanding representational spaces
• Statement of the problem: capturing fine-grained distinctions in a common model
• Conceptual framework: high-dimensional representational spaces
• Deriving the common space and individual transformation matrices with hyperalignment
• Validation
• Conclusions
A common high-dimensional linear model of representational spaces in human cortex
6
![Page 7: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/7.jpg)
fMRI data matrix
![Page 8: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/8.jpg)
fMRI data matrix Columns are voxel response tuning profiles
![Page 9: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/9.jpg)
fMRI data matrix Columns are voxel response tuning profiles
![Page 10: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/10.jpg)
fMRI data matrix Columns are voxel response tuning profiles
![Page 11: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/11.jpg)
fMRI data matrix Columns are voxel response tuning profiles
![Page 12: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/12.jpg)
fMRI data matrix Rows are multivoxel response patterns
![Page 13: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/13.jpg)
fMRI data matrix Rows are multivoxel response patterns
![Page 14: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/14.jpg)
fMRI data matrix Rows are multivoxel response patterns
![Page 15: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/15.jpg)
fMRI data matrix Rows are multivoxel response patterns
![Page 16: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/16.jpg)
What is a neural representational space? A pattern of activity (distributed over cortex) can be analyzed as a
vector in an n-dimensional space, where n = number of voxels (fMRI) or neurons (single unit recordings) or …
![Page 17: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/17.jpg)
Neural decoding using multivariate pattern analysis (MVPA)
• Pattern classification (MVPC)
• Representational similarity analysis (RSA)
![Page 18: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/18.jpg)
MVP Classification divides the representational space into sectors, each of which is associated with a different category
![Page 19: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/19.jpg)
Representational similarity analysis (RSA) indexes similarities between vectors as distances to analyze
representational geometry
![Page 20: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/20.jpg)
MVPC of fMRI data • Each observation (pattern) is
treated as a high-dimensional vector
• Each dimension is a single feature - usually a voxel
Condition a (e.g chairs) Condition b (e.g. shoes)
Illustration: 2 voxel pattern classifier
![Page 21: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/21.jpg)
MVPC of fMRI data • Each observation (pattern) is
treated as a high-dimensional vector
• Each dimension is a single voxel (or other feature)
• Classifiers find a rule (e.g. decision surface) that optimally differentiates observations for different conditions
Condition a (e.g chairs) Condition b (e.g. shoes)
Illustration: 2 voxel pattern classifier
![Page 22: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/22.jpg)
MVPC of fMRI data • Each observation (pattern) is
treated as a high-dimensional vector
• Each dimension is a single voxel (or other feature)
• Classifiers find a rule (e.g. decision surface) that optimally differentiates observations for different conditions
• The validity of that rule is tested on independent test data that played no role in deriving that rule
Illustration: 2 voxel pattern classifier
Training data
Test data
![Page 23: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/23.jpg)
Building and Testing Pattern Classifiers
1. Divide observations into training and test data sets
2. Based on the training data only
a. Select features (usually voxels)
b. Develop decision rule
3. Test decision rule on test data set
A new classifier is built for each individual
![Page 24: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/24.jpg)
Multivariate Pattern Classification Example: Classifying responses during viewing of animal species
(VT cortex, SVM)
(Haxby et al. 2011; Connolly et al. 2012)
monkey lemurPrimates
warbler mallardBirds
luna moth ladybugInsects
Classifier output
Vie
wed
stim
ulus
100
90
80
70
60
50
40
30
20
10
0
Percent ofclassifications
![Page 25: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/25.jpg)
MVP classification of animal species is significant in both early visual cortex and the ventral visual pathway (LOC) (Connolly et al. 2012)
![Page 26: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/26.jpg)
Data-driven cluster analysis finds distinct representational geometries in the ventral visual pathway (LOC) and early visual cortex (EV) (Connolly et al. 2012)
![Page 27: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/27.jpg)
Data-driven cluster analysis finds distinct representational geometries in the ventral visual pathway (LOC) and early visual cortex (EV) (Connolly et al. 2012)
![Page 28: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/28.jpg)
DSMs in early visual and LOC cortices correlate highly with semantic ratings and V1 models but not with each other (Connolly et al. 2012)
Correlation with ratings model = 0.76
Correlation with V1 model = 0.78
Correlation between LOC and EV DSMs = 0.09
![Page 29: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/29.jpg)
• Neural decoding: understanding representational spaces
• Statement of the problem: capturing fine-grained distinctions in a common model
• Conceptual framework: high-dimensional representational spaces
• Deriving the common space and individual transformation matrices with hyperalignment
• Validation
• Conclusions
A common high-dimensional linear model of representational spaces in human cortex
29
![Page 30: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/30.jpg)
• Neural decoding: understanding representational spaces
• Statement of the problem: capturing fine-grained distinctions in a common model
• Conceptual framework: high-dimensional representational spaces
• Deriving the common space and individual transformation matrices with hyperalignment
• Validation
• Conclusions
A common high-dimensional linear model of representational spaces in human cortex
30
![Page 31: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/31.jpg)
The problem: Building model representational spaces that are common across brains
• MVPA detects fine distinctions carried by fine-grained patterns of neural activity
• Anatomical alignment of brain spaces blurs these fine-grained distinctions
• Can a model of functional brain architecture capture these fine-grained distinctions among representations in a common framework?
– If so, how would such a model be structured?– Will it work? Do brains share a common basis for neural coding?
![Page 32: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/32.jpg)
The problem: Loss of fine-grained distinctions among representations after
anatomical alignment of brainsWithin-subject classification(new model for each subject)
Between-subject classification(common model based on anatomy)
WSC (1000 voxels)
BSC (1000anatomically-aligned voxels)
Chance (16.7%)
(Haxby et al. 2011; Connolly et al. 2012)
monkey lemurPrimates
warbler mallardBirds
luna moth ladybugInsects
![Page 33: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/33.jpg)
Modeling functional architecture of the human cortex: Anatomical alignment
Individual brainsTransformations
(affine or nonlinear warps) Brain atlas
![Page 34: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/34.jpg)
Modeling functional architecture of the human cortex: Anatomical alignment
Individual brainsTransformations
(affine or nonlinear warps) Atlas brain
![Page 35: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/35.jpg)
Modeling functional architecture of the human cortex: Anatomical alignment
Individual brainsTransformations
(affine or nonlinear warps) Atlas brain
P2
P1
![Page 36: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/36.jpg)
• Statement of the problem: capturing fine-grained distinctions in a common model
• Conceptual framework: high-dimensional representational spaces
- A pattern of activity is a response vector
- Dimensions are local features, e.g. voxels, of the pattern of activity
- Model space is based on features (dimensions) with common tuning profiles
• Deriving the common space and individual transformation matrices with hyperalignment
• Validation
• Conclusions
A common high-dimensional linear model of representational spaces in human cortex
36
![Page 37: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/37.jpg)
Conceptual framework: High-dimensional representational spaces
Brain activation patterns Data matrix Representational space (2 voxels)
![Page 38: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/38.jpg)
The locations of response pattern vectors for the same stimuli differ across subjects
![Page 39: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/39.jpg)
Modeling functional architecture of the human cortex: Individual representational spaces <=> common representational space
voxel1
voxel2
voxel3, v4, …,vi
voxel1
voxel2
voxel3 v4, …,vj
voxel1
voxel2
Individual representational spaces
dim1
dim2
dim3, dim4, …, dimm
Common modelrepresentational spaceIndividual brains
Transformations(improper rotations)
voxel3 v4, …,vk
![Page 40: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/40.jpg)
2
Modeling functional architecture of the human cortex: Individual representational spaces <=> common representational space
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
Individual brainsIndividual
representational spaces
dim1
dim2
dim3 ….
Common modelrepresentational space
1
3
123
Transformations(improper rotations)
![Page 41: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/41.jpg)
Modeling functional architecture of the human cortex: Individual representational spaces <=> common representational space
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
Individual brainsIndividual
representational spaces
dim1
dim2
dim3 ….
Common modelrepresentational space
2
1
3
123
Transformations(improper rotations)
2
1
3
213
![Page 42: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/42.jpg)
Modeling functional architecture of the human cortex: Individual representational spaces <=> common representational space
dim1
dim2
dim3 ….
Common modelrepresentational spaceIndividual brains
X
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
Individual representational spaces
X
X
X
Transformations(transposed rotations)
![Page 43: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/43.jpg)
Modeling functional architecture of the human cortex: Anatomical alignment
Individual brainsTransformations
(affine or nonlinear warps) Atlas brain
![Page 44: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/44.jpg)
Modeling functional architecture of the human cortex: Individual representational spaces <=> common representational space
dim1
dim2
dim3 ….
Common modelrepresentational spaceIndividual brains
X
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
Individual representational spaces
X
X
X
Transformations(transposed rotations)
![Page 45: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/45.jpg)
• Statement of the problem: capturing fine-grained distinctions in a common model
• Conceptual framework: high-dimensional representational spaces
• Deriving the common space and individual transformation matrices with hyperalignment
- Hyperalignment algorithm based on Procrustes transformations
- A rich sampling of response vectors using natural stimulus
• Validation
• Conclusions
A common high-dimensional linear model of representational spaces in human cortex
45
![Page 46: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/46.jpg)
Matrix math: Individual transformation matrices rotate individual brain spaces
into common model space coordinates
Individual brain spaces Transformation matrices Common model space
![Page 47: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/47.jpg)
Individual transformation matrices are the key to building the common model: How can the parameters be derived?
Transformation matrices
![Page 48: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/48.jpg)
48
![Page 49: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/49.jpg)
49
Subject 1 Subject 2
![Page 50: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/50.jpg)
Broad sampling of a neural representational space with a movie
Response patterns in cortex
15 response pattern vectors in individual 3D representational spaces (full exp’t has >2600 vectors in >50,000D space)
Subject 1 Subject 2
![Page 51: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/51.jpg)
Individual representational spaces
Common modelrepresentational space
Procrustes transformations(improper rotations)
x [ ] =
=S1
S2
![Page 52: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/52.jpg)
Individual representational spaces
S1
S2
S3
Common modelrepresentational space
Procrustes transformations(improper rotations)
x [ ]s2 =
=
x [ ]s3 =
![Page 53: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/53.jpg)
Individual representational spaces
S4
S5
S6
…
Common modelrepresentational space
Procrustes transformations(improper rotations)
x [ ]s5 =
x [ ]s6 =
x [ ]s4 =
![Page 54: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/54.jpg)
Individual representational spaces
S4
S5
S6
…
Common modelrepresentational space
Procrustes transformations(improper rotations)
x [ ]s5 =
x [ ]s6 =
x [ ]s4 =
![Page 55: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/55.jpg)
Movie data in Subject-specific Movie data in Brain Space Transformation Matrix Model Space
55
X =
S1
![Page 56: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/56.jpg)
Experiment 2 data in Subject-specific Experiment 2 data in Brain Space Transformation Matrix Model Space
56
The key that unlocks an individual’s neural code
![Page 57: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/57.jpg)
Experiment 2 data in Subject-specific Experiment 2 data in Brain Space Transformation Matrix Model Space
X =
![Page 58: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/58.jpg)
Modeling representational spaces in all human cortex with searchlight hyperalignment
d1
d2
d3, d4, …, dk
Voxels in overlapping searchlights Overlapping searchlight transformation matrices are hyperaligned across subjects are aggregated into a whole cortex matrix
Data in individual brain anatomy Data in common model space
![Page 59: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/59.jpg)
• Statement of the problem: capturing fine-grained distinctions in a common model
• Conceptual framework: high-dimensional representational spaces
• Deriving the common space and individual transformation matrices with hyperalignment
• Validation
– Between-subject correlations of time-series
– Between-subject classification of movie time segments
– Between-subject correlations of local similarity structures
– Applying transformation matrices to data from an unrelated experiment
• Conclusions
A common high-dimensional linear model of representational spaces in human cortex
59
![Page 60: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/60.jpg)
Whole-brain hyperalignment increases between-subject MVPC (bsMVPC) of 15 s movie time segments in occipital, temporal, parietal, and frontal cortices
5% 30%
Classificationaccuracy (%)
![Page 61: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/61.jpg)
Increased bsMVPC of movie time-segmentsin visual, auditory, and cognitive regions of interest (ROIs)
(coordinates from NeuroSynth)
![Page 62: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/62.jpg)
Whole-brain hyperalignment increases between-subject classification of 15 s movie time segments for the whole brain (after SVD dimensionality reduction)
bsMVP
C accuracy (%
± SE)
![Page 63: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/63.jpg)
How much movie data is necessary to calculate transformation matrices? Answer – the more the better, but ~20 minutes isn’t too bad
Number of Bme-‐points for hyperalignment
![Page 64: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/64.jpg)
Smoothing reduces BSC accuraciesIn all visual, auditory, and cognitive regions of interest (ROIs)
Smoothing filter (FWHM)
![Page 65: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/65.jpg)
Point spread function (intersubject correlations of movie time series):Fine spatial scale of alignment of function
ROI mean
![Page 66: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/66.jpg)
Whole-brain hyperalignment increases between-subject correlation of high-dimensional representational geometries
(correlations between movie time-points)
Correlation0.15 0.45
![Page 67: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/67.jpg)
Increased intersubject correlations of representational geometriesin visual, auditory, and cognitive regions of interest (ROIs)
![Page 68: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/68.jpg)
Second-order RSA: Between-ROI representational geometry dissimilarities
![Page 69: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/69.jpg)
Dimension 1
Dim
ension
3
Dimension 1
Dim
ension
2
V1-‐le&
V1-‐right
PPA-‐right PPA-‐le&
FFA-‐le& FFA-‐right VWFA
MT-‐right MT-‐le&
Math-‐le& Math-‐right
WM-‐right WM-‐le&
Broca
A1-‐le& A1-‐right
Voice-‐le&
Music-‐le& Music-‐right Voice-‐right
V1-‐le& V1-‐right
PPA-‐right PPA-‐le&
FFA-‐le& FFA-‐right
VWFA
MT-‐right MT-‐le&
Math-‐le&
Math-‐right WM-‐right
WM-‐le&
Broca
A1-‐le&
A1-‐right Voice-‐le&
Music-‐le& Music-‐right
Voice-‐right
Multidimensional scaling (MDS) of similarity structuresin visual, auditory, and cognitive regions of interest* (ROIs)
1st subspace (dimensions 1 & 2) 2nd subspace (dimensions 1 & 3)
* ROI coordinates from Neurosynth
![Page 70: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/70.jpg)
Dimension 1
Dim
ension
2
Dimension 1
Dim
ension
2
V1-‐le&
V1-‐right
PPA-‐right PPA-‐le&
FFA-‐le&
FFA-‐right VWFA
MT-‐right
MT-‐le&
A1-‐le&
A1-‐right
Voice-‐le&
Music-‐le&
Music-‐right
Voice-‐right
Multidimensional scaling (MDS) fit separately tovisual and auditory ROIs
MDS of visual regions only MDS of auditory regions only
![Page 71: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/71.jpg)
Whole-brain hyperalignment increases between-subject correlation of high-dimensional representational geometries
that reflect widely divergent domains of information
Correlation0.15 0.45
![Page 72: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/72.jpg)
A common high-dimensional linear model of representational spaces in human cortex James V Haxby1,2, J Swaroop Guntupalli1,3, Michael Hanke4, Peter J Ramadge5 1Dartmouth College, 2CIMeC, University of Trento, 3Caltech, 4University of Magdeburg, 5Princeton University
Within-subject classification (new model for each subject)
Between-subject classification (common model based on anatomy)
WSC (1000 voxels) BSC (1000 anatomically- aligned voxels) Chance (16.7%)
(Haxby et al. 2011; Connolly et al. 2012)
monkey lemur Primates
warbler mallard Birds
luna moth ladybug Insects
The problem Models based on anatomical alignment fail to capture fine-scale topographies that carry fine-grained distinctions among representations
The solution Searchlight hyperalignment of all cortex into a high-dimensional common model space
Validation highlights
Next: Connectivity hyperalignment Model dimensions have common functional connectivity profiles.
• The Procrustes transformation is used to develop the common space and to derive individual transformation matrices.
• General validity afforded by using a complex, rich stimulus to obtain a broad sample of response vectors.
Room for improvement • General validity can be increased with better stimulus and task paradigms (e.g. motor
execution and music) • Hyperalignment may be improved with better algorithm (e.g. regularized CCA, Xu,
Lorbert, et al. 2012)
Classification accuracy (%)
30% 5%
Between-subject classification of 15s movie time-segments (chance<1%)
Significance • Our model captures fine distinctions among neural population responses in a high-
dimensional representational space based on response tuning functions that are common across brains. • The model is valid for diverse domains of information. • Functional cortical topographies are modeled with individual basis functions that are
grounded in common tuning functions. • Potential basis for a new kind of functional brain atlas. – Report results as vectors in common model space rather than as anatomical
coordinates – Afford comparison and interpretation of results at a far more fine-grained level – Allow arbitrarily large, multi-subject data sets for MVPA
Between- and within-subject classification of 6 animal species (chance=17%) (BSC accuracy significantly higher than WSC, hyperalignment based on movie data)
Between-subject classification Within-subject classification Anatomical alignment Hyperalignment
Classification accuracy (%)
60% 30%
Mapping retinotopy by projecting other subjects’ polar angle maps into a different subject’s occipital topography (hyperalignment based on movie data)
Polar angle from subject’s own retinotopy data
Polar angle from other subjects’ retinotopy data
Correlation between measured and projected
Horizontal meridian
Vertical meridian
Intersubject correlations of connectivity vectors
Anatomical alignment
Connectivity hyperalignment
References Connolly AC, Guntupalli JS, Gors J, Hanke M, Halchenko YO, Wu YC, Abdi H, Haxby JV. (2012). The representation of biological classes in the human brain. Journal of Neuroscience. 32:2608-2618.
Guntupalli JS, Hanke M, Halchenko YO, Connolly AC, Ramadge PJ, Haxby JV. (under review). A model of representational spaces in human cortex.
Haxby JV, Guntupalli JS, Connolly AC, Halchenko YO, Conroy BR, Gobbini MI, Hanke M, Ramadge PJ. (2011). A common high-dimensional model of the representational space in human ventral temporal cortex. Neuron 72:404-416.
Haxby JV, Connolly AC, Guntupalli JS. (2014). Decoding neural representational spaces using multivariate pattern analysis. Annual Review of Neuroscience, 37, 435-456.
Xu H, Lorbert A, Ramadge PJ, Guntupalli JS, Haxby JV. (2012). Regularized hyperalignment of multi-set fMRI data. Proc. IEEE Signal Processing Workshop, Ann Arbor Michigan, 229-232
Supported by NSF1129764, “US-German collaboration: Building common high-dimensional models of neural representational spaces”
Anatomical alignment Hyperalignment
Brain connectivity patterns are better aligned in the common model space
Hyperalignment
Inter-subject correlation of connectivity vectors
![Page 73: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/73.jpg)
Experiment 2 data in Subject-specific Experiment 2 data in Brain Space Transformation Matrix Model Space
X =
![Page 74: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/74.jpg)
Whole-brain hyperalignment based on movie affords between-subject classification of responses in a visual category experiment (6 animal species)
at levels of accuracy that exceed within-subject classification
30% 60%
Classificationaccuracy (%)
Between-subject classification Within-subject classificationAnatomical alignment Hyperalignment
monkey lemurPrimates
warbler mallardBirds
luna moth ladybugInsects
![Page 75: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/75.jpg)
Similar results for other domains
• Action observation and attention (Sam Nastase, Dartmouth)
• Action execution (Nick Oosterhof, CIMeC, University of Trento)
• Connectivity hyperalignment and Music (Swaroop Guntupalli, Dartmouth, Caltech)
• Person knowledge (Dylan Wagner, Dartmouth, now Ohio State)
Further validation testing and algorithm development
![Page 76: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/76.jpg)
Raiders of the Lost ArkLife on Earth
The Wire
Hyperalignment parameters are estimated from responses
recorded during movie viewing
![Page 77: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/77.jpg)
Projecting group data from common model space into individual subject’s anatomy
dim1
dim2
dim3 ….
Common modelrepresentational spaceIndividual brains
X
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
voxel1
voxel2
voxel3 ….
Individual representational spaces
X
X
X
Transformations(transposed rotations)
![Page 78: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/78.jpg)
Modeling functional architecture of the human cortex: common model dimensions ≠ voxels
Individual representational spaces
dim2
dim3 ….
Common modelrepresentational spaceIndividual brains
Transformations(improper rotations)
![Page 79: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/79.jpg)
Topographies of weights for three model dimensions in two subjects
![Page 80: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/80.jpg)
Topographies for response patterns are modeled in different brains as weighted sums of individual-specific topographic basis functions
using the same weights for common model dimensions
Weights: ß1 ß2 ß3 ß4 ß5 ß6 ... ß35 ∑(ß*PC)
…
…
=>
=>
PC1 PC2 PC3 PC4 PC5 PC6 … PC35
S1
S2
![Page 81: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/81.jpg)
Individual VT topographies for face vs object dimension in the model agrees well with the topographies of individually-defined FFAs
Subject 1 Subject 2
81
![Page 82: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/82.jpg)
Mapping retinotopy by projecting other subjects’ polar angle maps into a different subject’s occipital topography
![Page 83: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/83.jpg)
The topographic basis functions for PCs individually show little correspondence to category-selective face and place areas or the
domain-specific divisions for animate and inanimate stimuli
Weights: ß1 ß2 ß3 ß4 ß5 ß6 ... ß35 ∑(ß*D)
…
…
=>
=>
PC1 PC2 PC3 PC4 PC5 PC6 … PC35
S1
S2
![Page 84: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/84.jpg)
Single dimensions (simple contrasts) are inadequate for modeling the functional architecture of cortex
Model dimensions (from PCA)
Varia
nce
acco
unte
d fo
r (%
)
The face vs object LD accounts for only 7% of movie response variance in VT cortex
• <20% of VAF by top 3 model dimensions
• <13% of VAF by 35 model dimensions
0
10
20
30
40
50
60
70
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
35D common model
1D model (face versus object LD)
84
![Page 85: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/85.jpg)
Category-selective regions!
Domain specificity! (animate vs inanimate)
Animacy continuum!
(human to bug)
It’s expertise!
Foveal versus peripheral
vision!
Stimulus size! 85
![Page 86: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/86.jpg)
• Our model captures fine distinctions among neural population responses in a high-dimensional representational space based on response tuning functions that are common across brains– Valid for diverse domains of information
• Functional cortical topographies are modeled with basis functions that are grounded in common tuning functions- Accounts for structure-function relationships in individual
brains with high fidelity• Single dimensions (or small numbers of dimensions) are
inadequate to capture fine distinctions and the fine-grained structure of topographies that carry these distinctions
Common model: Structure and validation testing
![Page 87: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/87.jpg)
Why are anatomical coordinates inadequate for capturing neural representation?
• Response tuning functions for voxels with the same anatomical coordinates are highly variable across brains.
• The basic unit for neural representation is the population response, not the responses of single voxels (or single neurons).
![Page 88: Decoding representational spaces with multivariate pattern ...€¦ · Decoding representational spaces with multivariate pattern analysis (MVPA) of fMRI data Jim Haxby Center for](https://reader033.vdocuments.us/reader033/viewer/2022060607/605c89821241500a373f5914/html5/thumbnails/88.jpg)
Software for ROI hyperalignment and data are on PyMVPA (www.pymvpa.org)
See the NeuroDebian/PyMVPA booth in exhibits
New massive data release of 7T fMRI with natural stimulus and lots more:data website: http://www.studyforrest.orgpaper: Hanke et al. (2014) Nature Scientific Data, 1: 140003.