convolutional neural networks for mesh-based … · "convolutional neural networks on graphs...
TRANSCRIPT
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
Guillem Cucurull, Konrad Wagstyl, Arantxa Casanova, Petar Veličković, Estrid Jakobsen, Michal Drozdzal, Adriana Romero, Alan Evans, Yoshua Bengio
�1
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
CORTEX PARCELLATION
▸ Different areas of the cerebral cortex are involved in different cognitive processes:
• Visual processing, language comprehension
▸ Mapping these areas helps us understand how the cortex is organized
Insert Cortex Parcellation Example Image
�2
Cortex Parcellation. From Glasser et al. 2016
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
WHAT IS A CORTICAL MESH
▸ Common coordinate system
▸ Can represent multiple imaging modalities and features
▸ Can be used to coregister cortical surfaces between different individuals
Insert Cortical Mesh Example Image
�3
Zoom of a cortical mesh
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
PREVIOUS WORK
▸ Glasser et al. 2016
Used an MLP classifier to recognize each cortical area, trained on semi-automatic labels of cortex parcellation.
▸ Jakobsen et al. 2016
Compared connectivity templates of each vertex of the cortical surface to parcellate Broca’s region.
�4
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
WHY CONVOLUTIONS
▸ Convolutional models achieve state of the art on image segmentation, which is analogous to this task.
▸ Important properties:
▸ Parameter sharing
▸ Exploits data structure
�5
Image from http://petar-v.com/GAT/
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
GRAPH CONVOLUTIONS
▸ Graph Convolutional Networks [Kipf 2017], ChebNets [Defferrard 2016]:
▸ Graph Attention Networks [Veličković 2018]:
�6
h′�i = σ( ∑j∈𝒩i
1cij
Whj)
h′�i = σ( ∑j∈𝒩i
αijWhj)
From the normalized
adjacency matrix
Attention coefficient
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
BASELINES▸ NodeMLP
• Independently classify each node
▸ NodeAVG
• Predict the most frequent label at each node, computed in the training set
▸ MeshMLP
• Process the full mesh at once, concatenating all nodes together
�7
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
DIFFERENCES BETWEEN MODELS
�8
Method
Properties NodeAVG NodeMLP MeshMLP GraphConv
Neigh Info
Global Info
Features
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
BROCA’S REGION PARCELLATION
▸ Broca’s region has an important role in language processing
It can be subdivided into two adjacent cytoarchitectonic areas: 44 and 45.
�9
▸ Dataset
▸ Manually labeled parcellation of Broca’s area from 100 different subjects.
▸ Each mesh has 1195 nodes, with 9 features per node, and 3 possible labels.
▸ Augment the features with x,y,z absolute position of the vertex
From Geranmayeh, 2014
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
EXPERIMENTAL SETUP
▸ Given the limited size of the dataset we use 10-fold cross validation to report results:
- 8 folds for training, 1 for validation, 1 for test
- The models are trained using Adam optimizer
- The optimized loss function is either weighted cross entropy or Dice loss
�10
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
RESULTS
�11
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
RESULTS
�12
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
RESULTS
�13
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
RESULTS
�14
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
RESULTS
�15
Using node
position
as features
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
RESULTS
�16
Subject 1
Subject 2
Subject 1
Subject 2
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
SUMMARY
▸ Investigated the use of graph CNNs for cortical mesh parcellation
▸ Evaluated on Broca’s area segmentation
▸ Graph CNNs models outperform the baselines
�17
CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX
REFERENCES‣ Glasser, Matthew F., Timothy S. Coalson, Emma C. Robinson, Carl D. Hacker, John Harwell, Essa Yacoub, Kamil Ugurbil et
al. "A multi-modal parcellation of human cerebral cortex." Nature 536, no. 7615 (2016): 171-178. ‣ Jakobsen, Estrid, Franziskus Liem, Manousos A. Klados, Şeyma Bayrak, Michael Petrides, and Daniel S. Margulies.
"Automated individual-level parcellation of Broca's region based on functional connectivity." Neuroimage (2016). ‣ Defferrard, Michaël, Xavier Bresson, and Pierre Vandergheynst. "Convolutional neural networks on graphs with fast
localized spectral filtering." In Advances in Neural Information Processing Systems, pp. 3844-3852. 2016. ‣ Kipf, Thomas N., and Max Welling. "Semi-supervised classification with graph convolutional networks." International
Conference on Learning Representations, ICLR, (2017). ‣ Veličković, Petar, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. "Graph Attention
Networks.” International Conference on Learning Representations, ICLR, (2018). ‣ Geranmayeh, Fatemeh, Sonia LE Brownsett, and Richard JS Wise. "Task-induced brain activity in aphasic stroke patients:
what is driving recovery?." Brain 137, no. 10 (2014): 2632-2648.
�18
Guillem Cucurull, Konrad Wagstyl, Arantxa Casanova, Petar Veličković, Estrid Jakobsen, Michal Drozdzal, Adriana Romero, Alan Evans, Yoshua Bengio
�19
THANKS!CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEXhttps://openreview.net/forum?id=rkKvBAiiz