Slide 1

User Profiling by Social Curation By GENG Xue Supervised by Prof. Chua Tat-Seng

Slide 2

A Crowd of Social Network Platforms

Slide 3

Changes in Concerts & Pope Inauguration 19902010

Slide 4

Exponential Multimedia 2013 Internet Trends http://www.kpcb.com/insights/2013-internet-trends http://www.kpcb.com/insights/2013-internet-trends The Visual nature of the web increases exponentially

Slide 5

One of Kind Big Multimedia

Slide 6

How to Deliver Meaningful Contents to the Right Person ? User Profiling

Slide 7

Definition A process to establish user profiles by extracting & representing the characteristics and preferences of users. Better Service Better Experience

Slide 8

Recommendation Recommend ? Similar A B Recommend !

Slide 9

Recommendation Similar Basic info & Social relationships A B

Slide 10

So, User Profiling by Multimedia Analysis

Slide 11

However, Multimedia data are very diverse & unorganized. Traditional approaches fail.

Slide 12

Solutions Structured multimedia & Social Intelligence. Flickr GalleriesFacebook Like

Slide 13

Social Curation Modern Funnel: Social Curation. People select organize keep track of items they like.

Slide 14

Social Curation Services (Pinterest)

Slide 15

Boards to which image are re-pinned Board name and holder

Slide 16

Why SCSs good? Organized Contents Content-centric network Better Content Models Refine Content Models

Slide 17

Why SCSs good? Organized Contents Better Content Models

Slide 18

Why SCSs good? Content-centric network Social intelligence to refine content models

Slide 19

Framework Profile Structure (Ontology) Learning Profile Structure Refinement User Profiles

Slide 20

Ontology-based user profiles (e.g., Fashion Domain)

Slide 21

Profile Ontology Construction

Slide 22

An example (pegged pants)

Slide 23

Profile Ontology Learning Idea : sibling samples are more visually similar, classifiers should be more distinct. V dresses, Strapless dresses, Halter dress, One Shoulder dress

Slide 24

Failure Cases Features may be Wrong

Slide 25

Refinement of user profiles Organized contents without social intelligence (content-centric network). Social intelligence to refine user profiles.

Slide 26

If images are shared by more users/boards simultaneously, they more likely belong to the same preference. User/Board-level Connection

Slide 27

Observation: Shopping Sites Recommendation T shirt People also see these

Slide 28

Observation: Movie Recommendation Recommendation

Slide 29

If images are shared by more users/boards simultaneously, they more likely belong to the same preference. User/Board-level Connection

Slide 30

Content-level Connection Similar images share similar visual cues and semantics. More Similar

Slide 31

Mathematical Social Intelligence

Slide 32

Refinement of User Profiles Multi-level connections are incorporated into the low-rank method Before refinement After refinement User-level connection Bundle-level connection Visual-level connection Semantic-level connection

Slide 33

Visualization of User Profiles It is a vector: (, 0.13, , 0.23, , 0.3, )

Slide 34

Experimental Data Data collection 1,239 users, 1,538,658 images. Profile learning and refinement We split labeled images equally into training/testing sets as the ground truths. Image recommendation We split the dataset by pin-time for training/testing We added half noisy data out of fashion domain to simulate real world recommendation system.

Slide 35

Evaluation of Profile Learning Failure Cases: a)Some image samples are too fine-grained. b)Some concepts tend to co-occur in the same image frequently.

Slide 36

Evaluation of Profile Refinement Failure Cases: a)Sparse & noisy connections from some outdated items. b)Some items are co-repined leading to similar multi-level connections

Slide 37

Evaluation of Image Recommendation

Slide 38

Conclusion Social Curation is NEW! It has Well-organized Contents Social Intelligence We test it on Pinterest (fashion domain).

Slide 39

Thank You