Tag Ranking

Present by Jie Xiao

Dept. of Computer Science

Univ. of Texas at San Antonio

jxiao@cs.utsa.edu 2

Outline

ProblemProbabilistic tag relevance estimationRandom walk tag relevance refinementExperimentConclusion

jxiao@cs.utsa.edu 3

Problem

There are millions of social images on internet, which are very attractive for the research purpose.

The tags associated with images are not ordered by the relevance.

Problem (Cont.)

jxiao@cs.utsa.edu 4

Tag relevance

There are two types of relevance to be considered.

The relevance between a tag and an image

The relevance between two tags for the same image.

jxiao@cs.utsa.edu 5

Probabilistic Tag Relevance Estimation

Similarity between a tag and an image

jxiao@cs.utsa.edu 6

x : an imaget : tag i associated with image xP(t|x) : the probability that given an image x, we have the tag t.P(t) : the prior probability of tag t occurred in the dataset

After applying Bayes’ rule, we can derive that

Probabilistic Relevance Estimation (Cont)

Since the target is to rank that tags for the individual image and p(x) is identical for these tags, we refine it as

jxiao@cs.utsa.edu 7

Density Estimation

Let (x1, x2, …, xn) be an iid sample drawn from some distribution with an unknown density ƒ.

Two types of methods to describe the densityHistogramKernel density estimator

jxiao@cs.utsa.edu 8

Histogram

jxiao@cs.utsa.edu 9

Credit: All of Nonparametric Statistics via UTSA library

Kernel Density Estimation

jxiao@cs.utsa.edu 10

Smooth function K is used to estimate the density

Kernel Density Estimation (Cont.)

Its kernel density estimator is

jxiao@cs.utsa.edu 11

Probabilistic Relevance Estimation (Cont)

Kernel Density Estimation (KDE) is adopted to estimate the probability density function p(x|t).

jxiao@cs.utsa.edu 12

Xi : the image set containing tag tixk : the top k near neighbor image in image set XiK : density kernel function used to estimate the probability|x| : cardinality of Xi

Relevance between tags

ti, tag i associated with image xtj, tag j associated with image x , the image set containing tag i , the image set containing tag jN: the top N nearest neighbor for image x

jxiao@cs.utsa.edu 13

Relevance between tags (Cont.)

jxiao@cs.utsa.edu 14

Relevance between tags (Cont.)

Co-occurrence similarity between tags

jxiao@cs.utsa.edu 15

f(ti) : the # of images containing tag tif(ti,tj) : the # of images containing both tag ti and tag tjG : the total # of images in Flickr

Relevance between tags (Cont.)

jxiao@cs.utsa.edu 16

Relevance between tags (Cont.)

Relevance score between two tags

jxiao@cs.utsa.edu 17

where

Random walk over tag graph

P: n by n transition matrix. pij : the probability of the transition from node i to j

jxiao@cs.utsa.edu 18

rk(j): relevance score of node i at iteration k

Random walk

jxiao@cs.utsa.edu 19

Random walk over tag graph (Cont.)

jxiao@cs.utsa.edu 20

Experiments

Dataset: 50,000 image crawled from FlickrPopular tags:Raw tags: more than 100,000 unique tagsFiltered tags: 13,330 unique tags

jxiao@cs.utsa.edu 21

Performance Metric

Normalized Discounted Cumulative Gain(NDCG)

jxiao@cs.utsa.edu 22

r(i) : the relevance level of the i - th tag

Zn : a normalization constant that is chosen so that the optimalranking’s NDCG score is 1.

Experimental Result

Comparison among different tag ranking approaches

jxiao@cs.utsa.edu 23

jxiao@cs.utsa.edu 24

Conclusion

Estimate the tag - image relevance by kernel density estimation.

Estimate the tag – tag relevance by visual similarity and tag co-occurrence.

A random walk based approach is used to refine the ranking performance.

jxiao@cs.utsa.edu 25

jxiao@cs.utsa.edu 26

Thank you!