topics in ai (cpsc 532s)lsigal/532s_2018w2/lecture18a.pdf · conclusions — deep learning on...
TRANSCRIPT
![Page 1: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/1.jpg)
Lecture 18: Graph Neural Networks
Topics in AI (CPSC 532S): Multimodal Learning with Vision, Language and Sound
![Page 2: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/2.jpg)
Logistics
— All slides will be up today!
— Last lecture by me
— Paper list is up (volunteers)?
![Page 3: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/3.jpg)
Traditional Neural Networks
* slide from Thomas Kipf, University of Amsterdam
![Page 4: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/4.jpg)
Graph-structured DataA lot of real-world data does not “live” on grids
* slide from Thomas Kipf, University of Amsterdam
![Page 5: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/5.jpg)
Graph-structured DataA lot of real-world data does not “live” on grids
* slide from Thomas Kipf, University of Amsterdam
![Page 6: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/6.jpg)
Graph-structured DataA lot of real-world data does not “live” on grids
* slide from Thomas Kipf, University of Amsterdam
![Page 7: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/7.jpg)
Graph-structured DataA lot of real-world data does not “live” on grids
* slide from Thomas Kipf, University of Amsterdam
![Page 8: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/8.jpg)
Graph-structured DataA lot of real-world data does not “live” on grids
Standard CNN and RNN architectures don’t work on this data
* slide from Thomas Kipf, University of Amsterdam
![Page 9: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/9.jpg)
Graph Neural Networks (GNNs)
Main Idea: Pass massages between pairs of nodes and agglomerate
Alternative Interpretation: Pass massages between nodes to refine node (and possibly edge) representations
* slide from Thomas Kipf, University of Amsterdam
![Page 10: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/10.jpg)
Graph Neural Networks (GNNs)
Main Idea: Pass massages between pairs of nodes and agglomerate
Alternative Interpretation: Pass massages between nodes to refine node (and possibly edge) representations
* slide from Thomas Kipf, University of Amsterdam
![Page 11: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/11.jpg)
Recap: Convolutional Neural Networks (CNNs) on Grids
* slide from Thomas Kipf, University of Amsterdam
![Page 12: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/12.jpg)
Recap: Convolutional Neural Networks (CNNs) on Grids
* slide from Thomas Kipf, University of Amsterdam
![Page 13: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/13.jpg)
Recap: Convolutional Neural Networks (CNNs) on Grids
* slide from Thomas Kipf, University of Amsterdam
![Page 14: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/14.jpg)
Recap: Convolutional Neural Networks (CNNs) on Grids
* slide from Thomas Kipf, University of Amsterdam
![Page 15: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/15.jpg)
Kipf & Welling (ICLR 2017), related previous works by Duvenaud et al. (NIPS 2015) and Li et al. (ICLR 2016) Graph Convolutional Networks (GCNs)
* slide from Thomas Kipf, University of Amsterdam
![Page 16: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/16.jpg)
Kipf & Welling (ICLR 2017), related previous works by Duvenaud et al. (NIPS 2015) and Li et al. (ICLR 2016) Graph Convolutional Networks (GCNs)
* slide from Thomas Kipf, University of Amsterdam
![Page 17: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/17.jpg)
Kipf & Welling (ICLR 2017), related previous works by Duvenaud et al. (NIPS 2015) and Li et al. (ICLR 2016) Graph Convolutional Networks (GCNs)
* slide from Thomas Kipf, University of Amsterdam
![Page 18: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/18.jpg)
Kipf & Welling (ICLR 2017), related previous works by Duvenaud et al. (NIPS 2015) and Li et al. (ICLR 2016) Graph Convolutional Networks (GCNs)
* slide from Thomas Kipf, University of Amsterdam
![Page 19: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/19.jpg)
GNNs with Edge EmbeddingsBattaglia et al. (NIPS 2016), Gilmer et al. (ICML 2017), Kipf et al. (ICML 2018)
* slide from Thomas Kipf, University of Amsterdam
![Page 20: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/20.jpg)
GNNs with Edge EmbeddingsBattaglia et al. (NIPS 2016), Gilmer et al. (ICML 2017), Kipf et al. (ICML 2018)
* slide from Thomas Kipf, University of Amsterdam
![Page 21: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/21.jpg)
GNNs with Edge EmbeddingsBattaglia et al. (NIPS 2016), Gilmer et al. (ICML 2017), Kipf et al. (ICML 2018)
* slide from Thomas Kipf, University of Amsterdam
![Page 22: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/22.jpg)
Graph Neural Networks (GNNs) with Attention Monti et al. (CVPR 2017), Hoshen (NIPS 2017), Veličković et al. (ICLR 2018)
* slide from Thomas Kipf, University of Amsterdam
![Page 23: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/23.jpg)
Graph Neural Networks (GNNs) with Attention Monti et al. (CVPR 2017), Hoshen (NIPS 2017), Veličković et al. (ICLR 2018)
* slide from Thomas Kipf, University of Amsterdam
![Page 24: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/24.jpg)
Graph Neural Networks (GNNs) with Attention Monti et al. (CVPR 2017), Hoshen (NIPS 2017), Veličković et al. (ICLR 2018)
* slide from Thomas Kipf, University of Amsterdam
![Page 25: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/25.jpg)
Graph Neural Networks (GNNs) with Attention Monti et al. (CVPR 2017), Hoshen (NIPS 2017), Veličković et al. (ICLR 2018)
* slide from Thomas Kipf, University of Amsterdam
![Page 26: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/26.jpg)
A Brief History of Graph Neural Nets
* slide from Thomas Kipf, University of Amsterdam
![Page 27: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/27.jpg)
How do we use GNN / GCN for real problems?
![Page 28: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/28.jpg)
Classification and Link Prediction with GNNs / GCNs
* slide from Thomas Kipf, University of Amsterdam
![Page 29: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/29.jpg)
Classification and Link Prediction with GNNs / GCNs
* slide from Thomas Kipf, University of Amsterdam
![Page 30: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/30.jpg)
Classification and Link Prediction with GNNs / GCNs
* slide from Thomas Kipf, University of Amsterdam
![Page 31: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/31.jpg)
Classification and Link Prediction with GNNs / GCNs
* slide from Thomas Kipf, University of Amsterdam
![Page 32: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/32.jpg)
Semi-supervised Classification on Graphs
* slide from Thomas Kipf, University of Amsterdam
![Page 33: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/33.jpg)
Semi-supervised Classification on Graphs
* slide from Thomas Kipf, University of Amsterdam
![Page 34: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/34.jpg)
Semi-supervised Classification on Graphs
* slide from Thomas Kipf, University of Amsterdam
![Page 35: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/35.jpg)
Conclusions— Deep learning on graphs works and is very effective!
— Exciting area: lots of new applications and extensions (hard to keep up)
Open problems: — Theory
— Scalable, stable generative models
— Learning on large, evolving data
— Multi-modal and cross-model learning (e.g., sequence2graph) * slide from Thomas Kipf, University of Amsterdam
![Page 36: Topics in AI (CPSC 532S)lsigal/532S_2018W2/Lecture18a.pdf · Conclusions — Deep learning on graphs works and is very effective! — Exciting area: lots of new applications and](https://reader033.vdocuments.us/reader033/viewer/2022052022/6036b128e9b7db3daf2d8d02/html5/thumbnails/36.jpg)
Graph Neural Nets (GNNs) are strict Generalizations of Traditional Neural Nets
(CNNs / RNNs can be implemented using GNNs / GCNs, but this is inefficient)