learning to identify overlapping and hidden cognitive processes from fmri data rebecca hutchinson,...
Post on 22-Dec-2015
218 views
TRANSCRIPT
![Page 1: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/1.jpg)
Learning to Identify Overlapping and Hidden
Cognitive Processes from fMRI Data
Rebecca Hutchinson, Tom Mitchell, Indra Rustandi
Carnegie Mellon University
![Page 2: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/2.jpg)
Decide whether consistent
How can we track hidden cognitive processes?
Read sentence
View picture
Cognitive processes:
?
Observed fMRI:
cortical region 1:
cortical region 2:
Observed button press:
![Page 3: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/3.jpg)
Typical BOLD response
Sig
nal
Am
plitu
de
Time (seconds)
• At left is a typical averaged BOLD response
• Here, subject reads a word, decides whether it is a noun or verb, and pushes a button in less than 1 second.
![Page 4: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/4.jpg)
Related Work
• General linear model (GLM) applied to fMRI– E.g., [Dale 1999]; SPM;– Accommodates multiple, overlapping processes,– But not unknown process timing
• Dynamic Bayesian Networks – Family of probabilistic models for time series– E.g., Factorial HMMs [Ghahramani & Jordan 1998]– Accommodate hidden timings/states– But do not capture convolution of overlapping states– Require learning detailed next-state function
![Page 5: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/5.jpg)
Approach: Hidden Process Models
• Probabilistic model– Can evaluate P(model | data), P(data | model)
• Describe hidden processes by their– Type, duration, start time, fMRI signature
• Algorithms for learning model, interpreting data– Learn maximum likelihood models and data
interpretations
![Page 6: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/6.jpg)
Hidden Process Models
Process ID = 3
Process ID = 2Process Instances:
Observed fMRI:
Processes:
ID: 1 Timing: P(start=+O)Response:
ID: 2Timing: P(start=+O)Response:
ID: 3 Timing: P(start=+O)Response:
Process ID = 1 Process ID = 1
Time landmarks: ¢ 1¢ 2¢ 1 ¢ 3
![Page 7: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/7.jpg)
Process: ViewPicture
Duration d: 11 sec. P(Offset times): , Response signature W:
Configuration C of Process Instances h 1, 2, … i
Observed data Y:
Input Stimulus :
14
Timing landmarks : ¢ 2¢ 1 ¢ 3
2
Process instance: 2
Process h: ViewPicture Timing landmark : 2
Offset time O: 1 sec Start time ´ + O
sentencepicture
sentence
3
Hidden Process Models
![Page 8: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/8.jpg)
HPMs More Formally…
Process h =
h d, , W i
Process Instance =
h h, , O i
Configuration C = set of Process Instances
Hidden Process Model HPM = h H, , C, i
• H: set of processes• : prior probs over H• C: set of candidate
configurations
• : h 1 … v i voxel noise model
![Page 9: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/9.jpg)
HPM Generative ModelProbabilistically generate data using a configuration
of N process instances with known landmarks:
1. Generate a configuration C of process instances: For i=1 to N, generate process instance i
• Choose a process hi according to P(h| i , )• Choose an offset Oi according to P(O| (h) )
2. Generate all observed fMRI data ytv given C:
![Page 10: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/10.jpg)
HPM Inference
• Given:– An HPM,
• including a set of candidate configurations• we typically assume processes known, but not timing
– Observed data Y
• Determine:– The most probable process instance
configuration c– P(C=c|Y, HPM) P(Y|C=c, HPM) P(C=c | HPM)
![Page 11: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/11.jpg)
Inference: Example
Configuration 1:
Observed data
ProcessID=1, S=1
ProcessID=2, S=17
ProcessID=3, S=21
Configuration 2:
ProcessID=2, S=1
ProcessID=1, S=17
ProcessID=3, S=23
Prediction 1
Prediction 2
![Page 12: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/12.jpg)
Learning HPMs with unknown timing O(), known processes h()
EM (Expectation-Maximization) algorithm
• E-step– Estimate the conditional distribution over start
times of the process instances given observed data, P(O(1)…O(N) | Y, h(1)… h(N), HPM).
• M-step– Use the distribution from the E step to get
maximum-likelihood estimates of the HPM parameters.
* In real problems, some timings are often known
![Page 13: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/13.jpg)
HPMs are learnable from realistic amounts of data
![Page 14: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/14.jpg)
Figure 1. The learner was given 80 training examples with known start times for only the first two processes. It chooses the correct start time (26) for the third process, in addition to learning the HDRs for all three processes.
true signal
Observed noisy signal
true response W
learned W
Process 1 Process 2 Process 3
![Page 15: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/15.jpg)
fMRI Study: Pictures and Sentences
• Each trial: determine whether sentence correctly describes picture
• 40 trials per subject.• Picture first in 20 trials, Sentence first in other 20• Images acquired every 0.5 seconds.
Read Sentence
View Picture Read Sentence
View PictureFixation
Press Button
4 sec. 8 sec.t=0
Rest
![Page 16: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/16.jpg)
Decide whether consistent
HPM model for Picture-Sentence Comparison
Read sentence
View picture
Cognitive processes:
?
Observed fMRI:
cortical region 1:
cortical region 2:
Observed button press:
![Page 17: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/17.jpg)
Learned HPM with 3 processes (S,P,D), and R=13sec (TR=500msec).
P PS S
D?
observed
Learned models:
S
P
D
D start time chosen by program as t+18
reconstructed
P PS S
D D
D?
![Page 18: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/18.jpg)
HPMs provide more accurate classification of unknown
processes than earlier methods
(e.g., Gaussian Naïve Bayes (GNB) classifier)
![Page 19: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/19.jpg)
Standard classifier formulation
View PictureOr
Read Sentence
Read SentenceOr
View PictureFixation
Press Button
4 sec. 8 sec.t=0
Rest
picture or sentence? picture or sentence?
16 sec.
GNB:
Standard formulation of classification problem (e.g., Gaussian Naïve Bayes (GNB)):
Train on labeled data: known Processes, known StartTimes
Test on unlabeled data: unknown Processes, known StartTimes
![Page 20: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/20.jpg)
HPM classifier accounts for overlap
View PictureOr
Read Sentence
Read SentenceOr
View PictureFixation
Press Button
4 sec. 8 sec.t=0
Rest
picture or sentence? picture or sentence?
16 sec.
GNB:
picture or sentence?
picture or sentence?
HPM:
![Page 21: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/21.jpg)
View PictureOr
Read Sentence
Read SentenceOr
View PictureFixation
Press Button
4 sec. 8 sec.t=0
Rest
picture or sentence? picture or sentence?
16 sec.
GNB:
picture or sentence?
picture or sentence?
HPM:
Results
HPM with overlapping processes improves accuracy by 15% on average.
![Page 22: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/22.jpg)
HPMs allow detecting and examining hidden processes
with unknown timing
![Page 23: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/23.jpg)
Decide whether consistent
Two cognitive processes, or three?
Read sentence
View picture
Cognitive processes:
?
Observed fMRI:
cortical region 1:
cortical region 2:
Observed button press:
![Page 24: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/24.jpg)
Choosing Between Alternative HPM Models
• Train 2-process HPM2 on training data
• Train 3-process HPM3 on training data
• Test HPM2 and HPM3 on separate test data
– Which predicts process identities better?– Which has higher probability given the test data?– (use n-fold cross-validation for test)
![Page 25: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/25.jpg)
2-process HPM, 3-process HPM, GNB
![Page 26: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/26.jpg)
Summary
• Hidden Process Model formalism
• Superiority over earlier classification methods
• Basis for studying hidden cognitive processes
![Page 27: Learning to Identify Overlapping and Hidden Cognitive Processes from fMRI Data Rebecca Hutchinson, Tom Mitchell, Indra Rustandi Carnegie Mellon University](https://reader030.vdocuments.us/reader030/viewer/2022032523/56649d815503460f94a65825/html5/thumbnails/27.jpg)
Future Directions
• Add temporal and/or spatial smoothness constraints to process fMRI signatures
• Allow variable duration processes
• Give processes input arguments, output results
• Feature selection for HPMs
• Process libraries, hierarchies