multiple objectives in collaborative filtering (recsys 2010)

Multiple Objectives in Collaborative Filtering

Tamas Jambor and Jun Wang

University College London

Structure of the talk

• Motivation• Multiple objectives• User perspective

– Promoting less popular items

• System perspective– Stock management

Motivation

• In the RecSys community, many research efforts are focused on recommendation accuracy

• And yet accuracy is not a only concern • Practical recommender systems might have

multiple goals

Improved Accuracy != Improved User experience

Algorithm Additional factors

External factors

System related

User related

Speed

Accuracy

Available resources

Cost of delivery

User interface

Diverse choices

Profitability per item

Advertisement

Improved user experience

Available resources

Cost of delivery

User interface

Diverse choices

Profitability per item

Advertisement

Additional factors

Accuracy

Improved user experience

Handling Multiple objectives

• Accuracy is the main objective– Defined in the baseline algorithm

• User perspective– Define and consider user satisfaction as priority

• System perspective– Consider additional system related objectives

• Objectives of the system might contradict

Where to optimize?

• In the objective function or as a post-filter?• Post-filters have the advantage to

– Add to any baseline algorithm– Extend easily – Add multiple goals

The proposed optimization framework(for each user)

• Add additional constraints of w

0

11:tosubject

ˆmax

ww

rwT

Tw

Properties of the framework

• Linear optimization problem• Recommendation as a ranking problem• Constraints provide the means of biasing the

ranking

User case – Promoting the Long Tail

Current systems are biased towards popular items

1 2 3 4 5 6 7 8 9 100

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

SVD

User-based

Item-based

Random Sample

Ranking Position

Pro

bab

ilit

y o

f an

y o

f 10

0 m

ost

po

pu

lar

item

bei

ng

at

ran

k-in

g p

osi

tio

n

Promoting the Long Tail

• Does that reflect real user needs?• Popular items might not be interesting for the user• Discovering unknown item could be more valuable• The aim is to reduce recommending popular items

– if the user is likely to be an interested in alternative choices

– keep recommending popular items otherwise

Promoting the Long Tail

• Extending the optimization framework

0

11

:tosubject

ˆmax

ww

mwm

rw

T

uT

Tw

Promoting the Long Tail and Diversification

• Diversifying the results

0

11

:tosubject

ˆmax

ww

mwm

wwrw

T

uT

TTw

Diversification

• Increase the covariance between recommended items– Reduce the risk of expanding the system– Provide a wider range of choice

Experimental setup

• MovieLens 1m dataset• 3900 movies, 6040 users• Five-fold cross validation

Evaluation metrics

• Recommendation as a ranking problem• IR measures

– Normalized discounted cumulative gain (NDCG)– Precision– Mean reciprocal rank (MRR)

• Constraint specific measures

Results: Promoting the Long Tail

1 2 3 4 5 6 7 8 9 100

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Baseline (SVD)

Long Tail Constraint

Long Tail Constraint and Diversi-fication (λ=6)

Random Sample

Ranking Position

Pro

bab

ilit

y o

f an

y o

f 10

0 m

ost

po

pu

lar

item

bei

ng

at

ran

k-in

g p

osi

tio

n

Results: Promoting the Long Tail

Baseline (SVD) LTC LTC and Div (λ=6)

NDCG@10 0.8808 0.8780 (-0.3%) 0.8715 (-1.0%)

P@10 0.8204 0.8207 (+0.2%) 0.8177 (-0.3%)

MRR 0.9518 0.9453 (-0.6%) 0.9349 (-1.7%)

System case – Resource Constraint

• Introducing external factors to the system• Stock availability of recommended items• The aim is to rank items lower, if less of them are

available• Minimizing performance loss

Simulation

• Online DVD-Rental company– Operates a warehouse– Only a limited number of items are available

• Recommend items that are in stock higher in the ranking list

Simulation

• User choice is based purely on recommendation• Simulating the stock level for 50 days

– Present a list of items to a random number of users– The probability that the item is taken depends on the

rank– Cumulative probability depends on how many times the

item was shown and at which rank position

Cut-off point

• Threshold c controls the cut-off point from which the system starts re-ranking items

cpcpss ti

tititi

,

,,,

if0 if

Resource Constraint

• Extending the optimization framework

0

11

:tosubject

ˆmax

w

w

sws

rw

T

u

Tw

Evaluation: Monitoring the waiting list size

• Waiting list– If item is not in stock, user puts it on their waiting list– When item returns, it goes out to the next user

• Waiting list size represents how long a user has to wait to get their favourite items

Results: Resource Constraint

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200

20

40

60

80

100

120

140

160

180 baseline c=1.6 c=1.2 c=0.0

Time (days)

Nu

mb

er o

f it

ems

on

th

e w

aiti

ng

lis

t

Results: Resource Constraint

• Trade-off between low waiting list size and good performance

0.750000000000001

0.770000000000001

0.790000000000001

0.810000000000001

0.830000000000001

0.850000000000001

0.870000000000001

0.890000000000001

0.910000000000001baseline c=1.6 c=1.2 c=0.0

Time (days)

ND

CG

@3

Results: Resource Constraint

c=0 c=0.4 c=0.8 c=1.2 c=1.6

NDCG@3(mean) -12.3% -4.32% -1.03% -0.43% -0.13%

NDCG@3(max) -14.7% -5.12% -1.34% -0.56% -0.50%

P@10(mean) -6.42% -3.37% -0.86% -0.06% -0.03%

P@10(max) -8.42% -3.91% -1.11% -0.24% -0.18%

Performance loss over 50 days

Conclusion

• Recommender systems have multiple objectives• Multiple optimization framework

– Expand the system with minor performance loss– It is designed to add objectives flexibly– It can be added to any recommender system

• Two scenarios that offer practical solutions– Long-tail items– Stock simulation

Future plan

• Personalized digital content delivery– Reduce delivery cost

• Diversification and the long tail– Does recommendation kill diversity?

• Evaluate improved user experience– User studies

Thank you.

References

• Deshpande, M., Karypis, G.: Item-based top-N recommendation algorithms. ACM Trans. Inf. Syst. 22(1) (2004)

• Herlocker, J.L., Konstan, J.A., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative filtering. In: SIGIR '99. (1999)

• Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8) (2009)

• Wang, J., de Vries, A.P., Reinders, M.J.T.: Unifying user-based and item-based collaborative filtering approaches by similarity fusion. In: SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, New York, NY, ACM Press