Neural Graph Collaborative Filtering

Advisor: Jia-Ling Koh

Presenter: You-Xiang Chen

Source: SIGIR ‘19

Data: 2019/12/20

1

Content

Introduction

Method

Experiment

01

Conclusion

02

03

042

Introduction

3

Recommended System

4

Collaborative Filtering

Matrix Factorization

Neural Collaborative Filtering

5

Introduction

1. Embedding , which transforms users and items to

vectorized representations

2. Interaction modeling , which reconstructs historical

interactions based on the embeddings.

Components of learnable CF models

6

Motivation

Inherent drawback of existing methods

• Most existing methods build the embedding

function with the descriptive features only.

• The embedding function lacks an explicit

encoding of the crucial collaborative signal .

7

Goal

• Proposing a new recommendation framework

based on graph neural network .

• The work explicitly encodes the collaborative

signal in the form of high-order connectivities .

8

Method

9

Neural graph collaborative filtering

Embedding

Embedding Propagation Layer

Prediction Layer

10

Embedding Layer

11

Embedding Propagation LayerFirst-order Propagation

• Message Construction

• Message Aggregation

For a connected user-item pair (u,i) , we define

the interaction message as:

We aggregate the messages propagated

from 𝒖′𝒔 neighborhood as 𝑢′𝑠representation :

𝒆𝒖(𝒍)

𝒆𝒖(𝟏)

𝒆𝒖(𝟐)

12

Embedding Propagation LayerFirst-order Propagation

• Message Construction𝒇 ∙ : 𝒆𝒏𝒄𝒐𝒅𝒊𝒏𝒈 𝒇𝒖𝒏𝒄𝒕𝒊𝒐𝒏

𝒑𝒖𝒊: 𝒄𝒐𝒆𝒇𝒇𝒊𝒄𝒊𝒆𝒏𝒕 𝒕𝒐 𝒄𝒐𝒏𝒕𝒓𝒐𝒍 𝒅𝒆𝒄𝒂𝒚 𝒇𝒂𝒄𝒕𝒐𝒓

𝐖𝟏,𝐖𝟐 ∈ ℝ𝒅′×𝒅

• Message Aggregation

𝑮𝑪𝑵 𝒄𝒐𝒏𝒔𝒊𝒅𝒆𝒓 𝒆𝒊 𝒐𝒏𝒍𝒚

13

Embedding Propagation LayerHigh-order Propagation

• Message Aggregation

𝐖𝟏𝒍 ,𝐖𝟐

𝒍 ∈ ℝ𝒅𝒍×𝒅𝒍−𝟏

14

Embedding Propagation LayerHigh-order Propagation

• Propagation Rule in Matrix Form

It allows us to discard the node sampling procedure

Simultaneously updating the representations for all

users and items in a rather efficient way

(facilitate batch implementation)

15

Embedding Propagation LayerHigh-order Propagation

• Matrix Form of layer-wise propagation rule

𝑹 ∈ ℝ𝑵×𝑴 :user-item interaction matrix

A :adjacency matrix

D :Diagonal degree matrix

𝑬(𝒍) ∈ ℝ 𝑵+𝑴 ×𝒅𝒍

Node degree

16

Laplacian Matrix

1

3

4

2

5

ℒ = 𝐷 − 𝐴 =

−1 −1 0 0 0−1 2 −1 −1 00 −1 3 −1 −10 −1 −1 3 −10 0 −1 −1 2

𝐴 =

0 1 0 0 01 0 1 1 00 1 0 1 10 1 1 0 10 0 1 1 0

𝐷 =

1 0 0 0 00 2 0 0 00 0 3 0 00 0 0 3 00 0 0 0 2

• Adjacency Matrix & Diagonal Degree Matrix

• Laplacian Matrix

17

Laplacian Matrix

ℒ 𝑠𝑦𝑚 = 𝐷−12 𝐿 𝐷−

12 = 𝐼 − 𝐷−

12 𝐴 𝐷−

12

ℒ𝑖,𝑗𝑠𝑦𝑚

• Symmetric normalized Laplacian

18

Model Prediction

19

OptimizationBayesian Personalized Ranking

Dislike? Missing value?

20

OptimizationBayesian Personalized Ranking

𝑦𝑢𝑖𝑗 = 𝑦𝑢𝑖 − 𝑦𝑢𝑗

• Learning models with BPR

• Objective function

𝓡+: observed interactions𝓡−: unobserved interactions

𝐿2 𝑟𝑒𝑔𝑢𝑙𝑎𝑟𝑖𝑧𝑎𝑡𝑖𝑜𝑛

𝑓𝑜𝑟 𝑒𝑣𝑒𝑟𝑦 𝑟𝑎𝑛𝑑𝑜𝑚𝑙𝑦 𝑠𝑎𝑚𝑝𝑙𝑒𝑑 𝒕𝒓𝒊𝒑𝒍𝒆 𝒖, 𝒊, 𝒋 ∈ 𝓞

BPR

21

Experiment

22

Dataset

Dataset Interaction# Item# User# Density

Gowalla 29,858 40,981 1,027,370 0.00084

Yelp2018 31,831 40,841 1,666,869 0.00128

Amazon-Book

52,643 91,599 2,984,108 0.00062

• Statistics of the evaluation datasets

23

Evaluation metrics

𝐷𝐶𝐺𝑘 =

𝑖=1

𝑘2𝑟𝑒𝑙𝑖 − 1

log2(𝑖 + 1)

𝑁𝐷𝐶𝐺@𝐾 =𝐷𝐶𝐺@𝐾

𝐼𝐷𝐶𝐺

• Normalized Discounted cumulative gain

24

Comparison Baselines

• Model-based CF methods

MF

NeuMF

CMN

• Graph-based CF methods

HOP-Rec

• Graph Convolutional Network

PinSage

GC-MC

25

Performance Comparison

26

Comparison w.r.t. Sparsity Level

27

Comparison w.r.t. Sparsity Level

28

Study of NGCF

29

Study of NGCF

30

Conclusion

31

Conclusion

• Extensive results demonstrate the state-of-the-art performance

of NGCF and its effectiveness in improving the embedding

quality with neural embedding propagation.

• The research present the NGCF model, a new recommendation

framework based on graph neural network, which explicitly

encodes the collaborative signal in the form of high-order

connectivities by performing embedding propagation.

32