lecture 23: recommender systemshzhang/math574m/2020...introduction terminology long-tail property...
TRANSCRIPT
Lecture 23: Recommender Systems
Hao Helen Zhang
Hao Helen Zhang Lecture 23: Recommender Systems
Outlines
Netflix Challenge
Introduction
terminologylong-tail propertycontent-based vs collaborative filteringchallengesevaluation
key recommendation techniques
content-based recommendationneighborhood-based methodslatent factor modelsmore techniques
Google’s Adwords problem
Hao Helen Zhang Lecture 23: Recommender Systems
References
Agarwal, D., and Chen, B.C. (2009). Regression-based latentfactor models. KDD’09
garwal, D., Zhang, L., and Mazumder, R. (2011). Modelingitem similarities for personalized recommendations on Yahoo!front page. Annals of Applied Statistics, 5, 1839-1875.
Feuerverger, A., He, Y., and Khatri, S. (2012). Statisticalsignificance of the Netflix challenge. Statistical Sciences, 27,202-231.
Rajaraman, A., Leskovec, J., and Ullman, J.D. (2012). Miningof Massive Datasets. Chapters 8, 9.
Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (Eds.) (2011).Recommender Systems Handbook. Springer.
Hao Helen Zhang Lecture 23: Recommender Systems
Hao Helen Zhang Lecture 23: Recommender Systems
Hao Helen Zhang Lecture 23: Recommender Systems
Netflix Challenge
Netfix, the worlds largest internet-based movie rental company
October 2, 2006, publicly released a set of data, and offered aGrand Prize of one million US dollars
goal: produce a system for recommending movies to usersbased on predicting how much someone is going to like anyparticular movie
Data: ratings (from 1 star to 5 stars) that users have assigned tomovies they have seen
training set: ∼ 100 million ratings, 480,000 users, 18,000movies (user-movie rating matrix is ∼ 99% sparse)
quiz set: ∼ 1.5 million ratings but withheld
test set: ∼ 1.5 million ratings but withheld
evaluation metric: root mean squared error (RMSE)
Hao Helen Zhang Lecture 23: Recommender Systems
Netflix Challenge Performance
overall average: 1.0528 for quiz, 1.0540 for test
Cinematch: 0.9514 for quiz, 0.9525 for test (∼ 9.5%improvement)
grand prize: RMSE 0.8572 (10%), or better, on the test set
progress prize: 50k best with at least 1% better than previousyear
July 26, 2009, the grand prize was won
Hao Helen Zhang Lecture 23: Recommender Systems
Introduction
Recommendation systems:
predict user response to item
item examples: news article, produce/service ads, movie, ...
Examples:
offer news articles to online newspaper readers, based on aprediction of reader interests
offer customers of an online retailer suggestions(products/ads) about what they might like to buy, based ontheir past history of purchases and/or product searches
Hao Helen Zhang Lecture 23: Recommender Systems
Hao Helen Zhang Lecture 23: Recommender Systems
Long-tail Property
Long-tail graph shows the distribution of ratings or popularityamong items or products in marketplace.
On the x-column, items are ordered by their popularity orrating frequencies
y-column shows the popularity in terms of ratings, demand etc
Three important facts:
popularity
diversity
sparsity
Hao Helen Zhang Lecture 23: Recommender Systems
Popularity
Products on left side (or in blue area) are called as “popular”because their popularity is higher than green or long-tail area.
popular products are generally competitive products.
Products in green long-tail area are thought to be “unpopular” or“new products” in market.
The threshold which discriminates popular and unpopular items inmarket is an hyper-parameter.
Hao Helen Zhang Lecture 23: Recommender Systems
Physical store vs Online store
Some researches show that even though popular products mean tobe sold a lot, unpopular products or those in long-tail generallyreturns in better profit.
physical institutions provide only the most popular items,
while on-line institutions provide the entire range of items
Hao Helen Zhang Lecture 23: Recommender Systems
Diversity
Recommender algorithms are generally designed to giverecommendations for popular items because they are popular.
However, a good recommendation system should provide diversity.
Same and known items can make the customers bored.
Adjusting the threshold, starting point of long-tail, inrecommendation system is an important research to take intoaccount.
Moving it right in the graph can increase the diversity inrecommendations made.
Hao Helen Zhang Lecture 23: Recommender Systems
Sparsity (Missing Values)
Sparsity: Items in the right side of graph are less rated than thethose in left side.
There are much more sparsity or unobserved areas forunpopular items in ratings matrix.
Due to sparsity, a recommender system which relies onneighborhood algorithms may produce bad results.
The more we move the threshold to right side, The worserecommendation system results.
Sparsity and long-tail are 2 important properties of a recommendersystem to take into account in design and process.
Hao Helen Zhang Lecture 23: Recommender Systems
Hao Helen Zhang Lecture 23: Recommender Systems
Different Similarity Measures
content-based system: measures similarity by looking forcommon features of items/users; e.g., if a Netflix user haswatched many cowboy movies, then recommend a movieclassified in the database as having the cowboy genre.
collaborative filtering: measure similarity of users by theiritem preferences; measure similarity of items by the users wholike them (user-item interaction)
hybrid system: uses both types of information
Hao Helen Zhang Lecture 23: Recommender Systems
Challenges
user-item interactions are extremely sparse
cold start problem
for content-based system, user has to dedicate an amount ofeffort using the system, so to construct their user profile,before the system can start providing any recommendation(fail for new user)
for collaborative filtering, it would fail to consider it would failto consider items which no on has rated previously (fail fornew item)
Hao Helen Zhang Lecture 23: Recommender Systems
Evaluation of Recommender Systems
Accuracy of the predicted scores:
root mean squared error (RMSE)
mean absolute error
Accuracy of the recommended list:
precision and recall:
precision(L) =1
Nuser
∑u
|L(u) ∩ T (u)||L(u)|
recall(L) =1
Nuser
∑u
|L(u) ∩ T (u)||T (u)|
average precision for a given user u:∑k=1,...,K precision(k)
number of items clicked in M recommended items,
where precision(k) is the precision at cut-off k , i.e., the ratio ofnumber of clicked items up to the position k over the number k
Hao Helen Zhang Lecture 23: Recommender Systems
Key recommendation techniques
content-based recommendation
neighborhood-based methods
latent factor models
more techniques
Hao Helen Zhang Lecture 23: Recommender Systems
Content-based recommendation
construction of feature profile:
item profile: movie features (genre, release date, cast); newsarticle features (topic, word frequencies); image tagsuser profile: browsing history; demographical information
recommend items for a given user:
recommend similar items based on item profilesbuild a decision / classification rule given item features ascovariateswould fail for new users
Hao Helen Zhang Lecture 23: Recommender Systems
Neighborhood-based Recommendation
Key Idea:
recommend similar items, or items of similar users
simple, efficient, stable
key components:
similarity measure / weight: Pearson correlation or othermeasures; based on user and item profiles
neighborhood selection: to address data sparsity, can clusterusers and/or items into small groups with strong similarity first
Hao Helen Zhang Lecture 23: Recommender Systems
Item-based recommendation
Predict user u’s rating of item i , rui , as
r̂ui = r̄i +
∑j∈Nu(i)
wij(ruj − r̄j)∑j∈Nu(i)
wij,
for user u, the rating for item i is the weighted average of thesame user’s ratings on similar items j ∈ Nu(i)
r̄i is the average rating of all users have given to item i :mean-centering normalization.
Hao Helen Zhang Lecture 23: Recommender Systems
User-based recommendation
Predict user u’s rating of item i , rui , as
r̂ui = r̄u +
∑v∈Ni (u)
wuv (rvi − r̄v )∑v∈Ni (u)
wuv,
for item i , the rating by user u is the weighted average of theratings from similar users v ∈ Ni (u)
r̄u is the average rating of all items that user i has given:mean-centering normalization.
Hao Helen Zhang Lecture 23: Recommender Systems
Latent Factor Models
also known as matrix factorization
key idea: map both users and items to a joint latent factorspace of dimensionality k , such that user-item interactions aremodeled as inner products in that space
each item i is associated with a vector qi ∈ Rk , and eachuser u is associated with a vector pu ∈ Rk .
qTi pu captures the interaction between user u and item i , i.e.,the overall interest of the user in characteristics of the item
model only the observed ratings, avoid over-fitting viaregularization.
Hao Helen Zhang Lecture 23: Recommender Systems
Latent Factor Model Estimation
Model: for a known link function g ,
g(E (rui )) = b0 + bi + bu + qTi pu
Estimation: take g = identity as an example, minimize∑observed
(rui − b0 − bi − bu − qTi pu)2 + λ(b2i + b2u + ‖qi‖2 + ‖pu‖2).
Hao Helen Zhang Lecture 23: Recommender Systems
Latent factor models incorporating user/item features:
Key Idea:
model item/user latent factors based on item/user features
use a Bayesian framework for computation
Model setup
rui ∼ Normal(µui , σ2)
rui ∼ Bernoulli(µui )
g(rui ) = b0 + bi + bu + qTi pu
bi = αT zi + εib, εib ∼ N(0, σ2i )
bu = βT xu + εub, εub ∼ N(0, σ2u)
qi = φzi + εiq, εiq ∼ MVN(0,Σi )
pu = ψxu + εup, εup ∼ MVN(0,Σu)
Hao Helen Zhang Lecture 23: Recommender Systems
Adwords Problem
Adwords Problem:
a fundamental problem of search advertising (about 24 billionsin 2012 for US market only)
we term the “adwords problem,” because it was firstencountered in the Google Adwords system
Hao Helen Zhang Lecture 23: Recommender Systems
History of Search Advertising
Around 2000, a company called Overture (later bought by Yahoo!)introduced a new (revolutionary) kind of search - advertisers bid onkeywords (words in a search query),
when a user searched for that keyword, the links to all theadvertisers who bid on that keyword are displayed in the orderhighest-bid-first;
if the advertisers link was clicked on, they paid what they hadbid
useful for when the search queryer was looking for advertisements,but rather useless if someone was just looking for information
Hao Helen Zhang Lecture 23: Recommender Systems
Google’s Adwords System
Years later, Google adapted the idea in a system called “Adwords”
by that time, the reliability of Google was well established, sopeople were willing to trust the ads they were shown
Google kept the list of responses based on PageRank (or othercriteria) separate from the list of ads, so the same system wasuseful for both types of users who wanted information orlooked to buy something
Hao Helen Zhang Lecture 23: Recommender Systems
Further Refinement/Complications
show only a limited number of ads with each query - Googlehad to decide both which ads to show and in what order
users of the Adwords system specified a budget: the amountthey were willing to pay for all clicks on their ads in a month
Google did not simply order ads by the amount of the bid, butby the amount they expect to receive for display of the ad -the value of an ad was taken to be the product of the bid andthe click-through rate
the decision regarding which ads to show must be madeon-line
Hao Helen Zhang Lecture 23: Recommender Systems
Adwords Problem: Input
a set of bids by advertisers for search queries
a click-through rate for each advertiser-query pair
a budget for each advertiser (for a month, or other timelength)
a limit on the number of ads to be displayed with each searchquery
Hao Helen Zhang Lecture 23: Recommender Systems
Adwords Problem: Output
respond to each search query with a set of advertisers s.t.
the size of the set is no larger than the limit on the number ofads per query
each advertiser has bid on the search query
each advertiser has enough budget left to pay for the ad if itis clicked upon
greedy algorithm, balance algorithm, implementation
Hao Helen Zhang Lecture 23: Recommender Systems