matrix completion with graph neural...
TRANSCRIPT
![Page 1: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/1.jpg)
Matrix Completion with Graph Neural Networks
Qin Zhang2017. 2. 23
![Page 2: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/2.jpg)
Matrix CompletionMonti F, Bronstein M, Bresson X. Geometric matrix completion with recurrent multi-graph neural networks. NIPS 2017: 3700-3710.
van den Berg R, Kipf T N, Welling M. Graph Convolutional Matrix Completion[J]. stat, 2017, 1050: 7.
![Page 3: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/3.jpg)
![Page 4: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/4.jpg)
Matrix Completion
Mathematically, a recommendation method can be posed as a matrix completion problem, where columns and rows represent users and items, respectively, and matrix values represent scores determining whether a user would like an item or not.
Given a small subset of known elements of the matrix, the goal is to fill in the rest.
![Page 5: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/5.jpg)
Matrix Completion
NP-hard combinatorial problem
Non-differentiable
rank(X): the maximal number of linearly independent columns of X;
Why minimize rank(X): assume that the variables ie in a smaller subspace, i.e., the matrix is of low rank. → if two users have same tastes, their rating scores are the same or very similar; and there are many users have same or similar testes;
![Page 6: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/6.jpg)
equal to the sum of its singular values
Assumption: users are independent with each other; items are independent with each other.
Matrix Completion
is the tightest possible convex relaxation of the rank(.) [1]
[1] Candès E J, Recht B. Exact matrix completion via convex optimization[J]. Foundations of Computational mathematics, 2009, 9(6): 717.
![Page 7: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/7.jpg)
Geomatric Matrix Completion
![Page 8: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/8.jpg)
Geomatric Matrix Completion
Laplacian Eigenmaps is a popular technique for dimensionality reduction on graphs.
Graph laplacian minimization tries to make the connected nodes in the original space still be close to each other in the reduced-dimensional space. Thus, the structure of the graph can be still maintained
u and uc are trade-off parameters;
graph Laplacian
![Page 9: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/9.jpg)
Graph Laplacian
![Page 10: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/10.jpg)
Dirichlet norm: The smaller the norm, the slower the change, the better the structure reserved.
Graph Laplacian
Dirichlet norm
![Page 11: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/11.jpg)
Geomatric Matrix Completion
Laplacian Eigenmaps is a popular technique for dimensionality reduction on graphs.
Graph laplacian minimization tries to make the connected nodes in the original space still be close to each other in the reduced-dimensional space. Thus, the structure of the graph can be still maintained
u and uc are trade-off parameters;
graph Laplacian
![Page 12: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/12.jpg)
Geometric Matrix completion
![Page 13: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/13.jpg)
Geometric Matrix Completion
Ultimate objective function
![Page 14: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/14.jpg)
Methodology
Multi-Graph CNN
![Page 15: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/15.jpg)
CNN
Euclidean data: images, texts
![Page 16: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/16.jpg)
Graph Convolutional Networks(GCN)
12 nodes 6 nodes 3 nodes
![Page 17: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/17.jpg)
Geometric Matrix completion
Multi-graph CNN (GCN)
![Page 18: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/18.jpg)
2D Fourier transform
![Page 19: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/19.jpg)
2D Fourier transform
![Page 20: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/20.jpg)
2D Fourier transform
![Page 21: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/21.jpg)
![Page 22: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/22.jpg)
Recurrent MGCNN
O(mn) variables
![Page 23: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/23.jpg)
Factorized matrix completion models
O(m+n) variables
![Page 24: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/24.jpg)
Factorized matrix completion models
O(m+n) variables
![Page 25: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/25.jpg)
Separable MGCNN
![Page 26: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/26.jpg)
sRMGCNN
![Page 27: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/27.jpg)
Experimental resultsDatasets: Synthetic dataset*, MovieLens, Flixster, Douban, and YahooMusic.
Baseline: Classical Matrix Completion (MC), Inductive Matrix Completion (IMC), Geometric Matrix Completion(GMC), and Graph Regularized Alternating Least Squares (GRALS).
Algorithm: Adam stochastic optimization algorithm.
Implement: TensorFlow
*in which the user and item graphs have strong communities structure
![Page 28: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/28.jpg)
Experimental results
![Page 29: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/29.jpg)
Experimental results
![Page 30: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/30.jpg)
ConclutionA new deep learning approach for matrix completion based on multi-graph convolutional neural network architecture was proposed.
Key advantages: low computational complexity: O(m+n) variablesconstant number of degrees (independent to the matrix size)
![Page 31: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/31.jpg)
![Page 32: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/32.jpg)
0 - missing observations
![Page 33: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/33.jpg)
Graph auto-encoder
![Page 34: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/34.jpg)
Graph auto-encoder
GAE
Encoder
![Page 35: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/35.jpg)
Graph auto-encoder
GAE
Decoder
![Page 36: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/36.jpg)
Graph auto-encoder
Loss function:
![Page 37: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/37.jpg)
Only rating data
rating data+user/item features
![Page 38: Matrix Completion with Graph Neural Networksyunshengb.com/wp-content/uploads/2018/02/Geometric... · A new deep learning approach for matrix completion based on multi-graph convolutional](https://reader034.vdocuments.us/reader034/viewer/2022042308/5ed4501b4e1aa219885a94f7/html5/thumbnails/38.jpg)
ConclusionA graph convolutional matrix completion (GC-MC) was proposed to learn bipartite user-item interaction graphs. Our model takes the form of a graph
Advantage:side information for both types of nodes can be included.
Disadvantage:complexity is O(mn) while sRMGCNN’s is O(m+n)