research © 2008 yahoo! statistical challenges in online advertising deepak agarwal deepayan...
TRANSCRIPT
![Page 1: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/1.jpg)
Research © 2008 Yahoo!
Statistical Challenges in Online Advertising
Deepak Agarwal
Deepayan Chakrabarti(Yahoo! Research)
![Page 2: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/2.jpg)
Research © 2008 Yahoo!
Online Advertising
• Multi-billion dollar industry, high growth
– $9.7B in 2006 (17% increase), total $150B
• Why this will continue?
– Broadband cheap, ubiquitous
– “Getting things done” easier on the internet
– Advertisers shifting dollars
• Why does it work?
– Massive scale, automated, low marginal cost
– Key: Monetize more and better, “learn from data”
– New discipline “Computational Advertising”
![Page 3: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/3.jpg)
Research © 2008 Yahoo!
What is “Computational Advertising”?
New scientific sub-discipline, at the intersection of – Large scale search and text analysis
– Information retrieval
– Statistical modeling
– Machine learning
– Optimization
– Microeconomics
![Page 4: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/4.jpg)
Research © 2008 Yahoo!
Online advertising: 6000 ft Overview
Ad
vert
iser
s
Ad Network
Ads
Content
Pick ads
User
Content Provider
Examples:Yahoo, Google,
MSN, RightMedia, …
![Page 5: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/5.jpg)
Research © 2008 Yahoo!
Outline
• Background on online advertising
– Sponsored Search, Content Match, Display, Unified marketplace
• The Fundamental Problem
• Statistical sub-problems:
– Description
– Existing methods
– Challenges
![Page 6: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/6.jpg)
Research © 2008 Yahoo!
Different flavors
Online Advertising
Revenue Models
Advertising Setting
Misc.
CPM CPC CPA
Display Content Match
Sponsored Search
Ad exchanges
![Page 7: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/7.jpg)
Research © 2008 Yahoo!
Revenue Models
CPM CPC CPA
Ad
vert
iser
s
Ads
Content
Pick ads
User
Cost Per iMpression
$$
$Content Provider
Ad Network
![Page 8: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/8.jpg)
Research © 2008 Yahoo!
Revenue Models
CPM CPC CPA
Ad
vert
iser
s
Ads
Content
Pick ads
User
Cost Per Click
$$
$Content Provider
Ad Networkclick
![Page 9: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/9.jpg)
Research © 2008 Yahoo!
Revenue Models
CPM CPC CPA
Ad
vert
iser
s
Ads
Content
Pick ads
User
Cost Per Action
$$
$Content Provider
Ad Networkclick
Adver
tiser
land
ing
page
![Page 10: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/10.jpg)
Research © 2008 Yahoo!
Revenue Models
• Example: Suppose we show an ad N times on the same spot
• Under CPM: Revenue = N * CPM
• Under CPC: Revenue = N * CTR * CPC
CPM CPC CPA
Click-through Rate(probability of a click given an impression)
Depends on auction
mechanism
![Page 11: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/11.jpg)
Research © 2008 Yahoo!
Auction Mechanism
• Revenue depends on type of auction– Generalized First-price:
• CPC = bid on clicked ad
– Generalized Second-price: • CPC = bid of ad below clicked ad (or the reserve price)
• CPC could be modified by additional factors
• [Optimal Auction Design in a Multi-Unit Environment: The Case of Sponsored Search Auctions] by Edelman+/2006
• [Internet Advertising and the Generalized Second Price Auction…] by Edelman+/2006
![Page 12: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/12.jpg)
Research © 2008 Yahoo!
Revenue Models
• Example: Suppose we show an ad N times on the same spot
• Under CPM: Revenue = N * CPM
• Under CPC: Revenue = N * CTR * CPC
• Under CPA: Revenue = N * CTR * Conv. Rate * CPA
CPM CPC CPA
Conversion Rate(probability of a user conversion on the advertiser’s landing page
given a click)
![Page 13: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/13.jpg)
Research © 2008 Yahoo!
Revenue Models
CPM
website traffic
CPC
website traffic +ad relevance
Revenue dependence
CPA
website traffic +ad relevance +landing page quality
Relevance to advertisers
Prices and Bids
Ease of picking ads
![Page 14: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/14.jpg)
Research © 2008 Yahoo!
Background
Online Advertising
Revenue Models
Advertising Setting
Misc.
CPM CPC CPA
Display Content Match
Sponsored Search
Ad exchanges
![Page 15: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/15.jpg)
Research © 2008 Yahoo!
Advertising Setting
Ad
vert
iser
s
Ad Network
Content
Pick ads
User
Content Provider
Ads
• What do you show the user?
• How does the user interact with the ad system?
![Page 16: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/16.jpg)
Research © 2008 Yahoo!
Advertising Setting
Display Content Match
Sponsored Search
![Page 17: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/17.jpg)
Research © 2008 Yahoo!
Advertising Setting
Display Content Match
Sponsored Search
Pick ads
![Page 18: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/18.jpg)
Research © 2008 Yahoo!
Advertising Setting
• Graphical display ads
• Mostly for brand awareness
• Revenue model is typically CPM
Display Content Match
Sponsored Search
![Page 19: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/19.jpg)
Research © 2008 Yahoo!
Advertising Setting
Display Content Match
Sponsored Search
Content match ad
![Page 20: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/20.jpg)
Research © 2008 Yahoo!
Advertising Setting
Display Content Match
Sponsored Search
Pick ads
Text ads
Match ads to the content
![Page 21: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/21.jpg)
Research © 2008 Yahoo!
Advertising Setting
• The user intent is unclear
• Revenue model is typically CPC
• Query (webpage) is long and noisy
Display Content Match
Sponsored Search
![Page 22: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/22.jpg)
Research © 2008 Yahoo!
Advertising Setting
Display Content Match
Sponsored Search
Search Query
Sponsored Search Ads
![Page 23: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/23.jpg)
Research © 2008 Yahoo!
Advertising Setting
Display Content Match
Sponsored Search
Pick ads
Text ads
Search Query
Match ads to the query
![Page 24: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/24.jpg)
Research © 2008 Yahoo!
Advertising Setting
• User “declares” his/her intention
• Click rates generally higher than for Content Match
• Revenue model is typically CPC (recently some CPA)
• Query is short and less noisy than Content Match
Display Content Match
Sponsored Search
![Page 25: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/25.jpg)
Research © 2008 Yahoo!
Summary
• Different revenue models
– Depends on the goal of the advertiser campaign
• Brand awareness
– Display advertising
– Pay per impression (CPM)
• Attracting users to advertised product
– Content Match, Sponsored Search
– Pay per click (CPC), Pay per action (CPA)
![Page 26: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/26.jpg)
Research © 2008 Yahoo!
Background
Online Advertising
Revenue Models
Advertising Setting
Misc.
CPM CPC CPA
Display Content Match
Sponsored Search
Ad exchanges
![Page 27: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/27.jpg)
Research © 2008 Yahoo!
Unified Marketplace
• Publishers, Ad-networks, advertisers participate together in a singe exchange
• Publishers put impressions in the exchange; advertisers/ad-networks bid for it
• CPM, CPC, CPA are all integrated into a single auction mechanism
![Page 28: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/28.jpg)
Research © 2008 Yahoo!
Overview: The Open Exchange
Transparency and value
Has ad impression to sell --AUCTIONS
Bids $0.50Bids $0.75 via Network…
… which becomes $0.45 bid
Bids $0.65—WINS!
AdSenseAd.com
Bids $0.60
![Page 29: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/29.jpg)
Research © 2008 Yahoo!
Unified scale: Expected CPM
• Campaigns are CPC, CPA, CPM
• They may all participate in an auction together
• Converting to a common denomination is a challenge
![Page 30: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/30.jpg)
Research © 2008 Yahoo!
Outline
• Background on online advertising
• The Fundamental Problem
• Statistical sub-problems:
– Description
– Existing methods
– Challenges
![Page 31: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/31.jpg)
Research © 2008 Yahoo!
Outline
• Background on online advertising
• The Fundamental Problem
– Display advertising
– Sponsored Search and Content Match
• Statistical sub-problems:
– Description
– Existing methods
– Challenges
![Page 32: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/32.jpg)
Research © 2008 Yahoo!
Display Advertising
![Page 33: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/33.jpg)
Research © 2008 Yahoo!
Display Advertising
• Main goal of advertisers: Brand Awareness
• Revenue Model: Primarily Cost per impression (CPM)
• Traditional Advertising Model:
1. Ads are targeted at particular demographics (user characteristics)
1. GM ads on Y! autos shown to “males above 55”
2. Mortgage ad shown to “everybody on Y! Front page”
2. Book a slot well in advance– “2M impressions in Jan next year”
– These future impressions must be guaranteed by the ad network
![Page 34: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/34.jpg)
Research © 2008 Yahoo!
Display Advertising
• Fundamental Problem: Guarantee impressions to advertisers
3
24
2 2
1
1
Young US
FemaleY! Mail
1. Predict Supply:
• How many impressions will be available?
• Demographics overlap
2. Predict Demand:
• How much will advertisers want each demographic?
![Page 35: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/35.jpg)
Research © 2008 Yahoo!
Display Advertising
• Fundamental Problem: Guarantee impressions to advertisers
3
24
2 2
1
1
Young US
FemaleY! Mail
1. Predict Supply
2. Predict Demand
3. Find the optimal allocation
• subject to supply and demand constraints
![Page 36: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/36.jpg)
Research © 2008 Yahoo!
Display Advertising
• Fundamental Problem: Guarantee impressions to advertisers
1. Predict Supply
2. Predict Demand
3. Find the optimal allocation, subject to constraints
• Optimal in terms of what objective function?
![Page 37: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/37.jpg)
Research © 2008 Yahoo!
Allocation through Optimization
• Optimal in terms of what objective function?
– E.g. Maximize value of remaining inventory • Cherry-picks valuable inventory, saves it for later
– Fairness• “Spreads the wealth” subject to constraints
sisupply demand
dj
xij
![Page 38: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/38.jpg)
Research © 2008 Yahoo!
Example
324
2 2
1
1
Young US
FemaleY!
US & Y(2)
Supply Pools
DemandUS, Y, nFSupply = 2Price = 1
US, Y, FSupply = 3Price = 5
Supply Pools
How should we distribute impressions from the supply pools to satisfy this
demand?
![Page 39: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/39.jpg)
Research © 2008 Yahoo!
Example (Cherry-picking)
• Cherry-picking: Fulfill demands at least cost
US & Y(2)
Supply Pools
DemandUS, Y, nFSupply = 2Price = 1
US, Y, FSupply = 3Price = 5
How should we distribute impressions from the supply pools to satisfy this
demand?
(2)
![Page 40: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/40.jpg)
Research © 2008 Yahoo!
Example (Fairness)
• Cherry-picking: Fulfill demands at least cost
• Fairness:Equitable distribution of available supply pools US & Y
(2)
Supply Pools
DemandUS, Y, nFSupply = 2
Cost = 1
US, Y, FSupply = 3
Cost = 5
How should we distribute impressions from the supply pools to satisfy this
demand?
(1)
(1)
![Page 41: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/41.jpg)
Research © 2008 Yahoo!
Objective functions
jV
jy
yV
j
j
jjj
pool of Value:
poolfor inventory remaining :
Maximize
.0
)~/log(
Minimize :function Objective
.V valueof
function decreasinglly monotonica becan general,In
allocation alproportion1 ..
;)/(~
j
:
k
jkj
jkjkk
k
j
j
SSjjjwkwjjjk
xxx
w
wge
wxXdXwxxkj
Fairness""
![Page 42: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/42.jpg)
Research © 2008 Yahoo!
Display Advertising
• Fundamental Problem: Guarantee impressions to advertisers
1. Predict Supply
2. Predict Demand
3. Find the optimal allocation, subject to constraints– Pick the right objective function
• Further issues:
– Risk Management: Supply and demand forecasts should have both mean and variance
– Forecast aggregation: Forecasts may be needed over multiple resolutions, in time and in demographics
![Page 43: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/43.jpg)
Research © 2008 Yahoo!
Display Advertising
• Fundamental Problem: Guarantee impressions to advertisers
1. Predict Supply
2. Predict Demand
3. Find the optimal allocation, subject to constraints– Pick the right objective function
• Forecasting accuracy is critical!
– Overshoot under-delivery of impressions unhappy advertisers
– Undershoot loss in revenue
![Page 44: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/44.jpg)
Research © 2008 Yahoo!
Outline
• Background on online advertising
• The Fundamental Problem
– Display advertising
– Sponsored Search and Content Match
• Statistical sub-problems:
– Description
– Existing methods
– Challenges
![Page 45: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/45.jpg)
Research © 2008 Yahoo!
Sponsored Search and Content Match
• Given a query:
– Select the top-k ads to be shown on the k slots to maximize total expected revenue
• What is total expected revenue?
![Page 46: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/46.jpg)
Research © 2008 Yahoo!
Example (Content Match)
Ad Position 1
Ad Position 2
Ad Position 3
![Page 47: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/47.jpg)
Research © 2008 Yahoo!
Example (Content Match)
![Page 48: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/48.jpg)
Research © 2008 Yahoo!
Reminder: Auction Mechanism
• Revenue depends on type of auction– Generalized First-price:
• CPC = bid on clicked ad
– Generalized Second-price: • CPC = bid of ad below clicked ad (or the reserve price)
• CPC could be modified by additional factors
• Total expected revenue = revenue obtained in a given time window
• [Optimal Auction Design in a Multi-Unit Environment: The Case of Sponsored Search Auctions] by Edelman+/2006
• [Internet Advertising and the Generalized Second Price Auction…] by Edelman+/2006
![Page 49: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/49.jpg)
Research © 2008 Yahoo!
Sponsored Search and Content Match
• Given a query:
– Select the top-k ads to be shown on the k slots to maximize total expected revenue
• What affects the total revenue?
– Relevance of the ad to the query
– Bids on the ads
– User experience on the ad landing page (ad “quality”)
– Expected total revenue is some function of these.
![Page 50: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/50.jpg)
Research © 2008 Yahoo!
Sponsored Search and Content Match
• Given a query:
– Select the top-k ads to be shown on the k slots to maximize total expected revenue
• Fundamental Problem:
– Estimate relevance of the ad to the query
![Page 51: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/51.jpg)
Research © 2008 Yahoo!
Ad Relevance Computation
![Page 52: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/52.jpg)
Research © 2008 Yahoo!
Overview
• Information Retrieval (IR)
– Techniques
– Challenges
• Machine Learning using Click Feedback
• Online Learning
![Page 53: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/53.jpg)
Research © 2008 Yahoo!
IR-based ad matching
• “Why not use a search engine to match ads to context?”– Ads are the “documents”
– Context (user query or webpage content) is the “query”
• Three broad approaches:– Vector space models
– Probabilistic models
– Language models
• Open-source software is available:
– Lemur (www.lemurproject.org)
![Page 54: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/54.jpg)
Research © 2008 Yahoo!
IR-based ad matching
• Vector space models:
– Each word/phrase in the vocabulary is a separate dimension
– Each ad and query is a point in this vector space
– Example: cosine similarity
• Probabilistic models
• Language models
![Page 55: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/55.jpg)
Research © 2008 Yahoo!
• Q1: How can we score the goodness of an ad for a context?
• Cosine similarity:
• Advantages:
– Simple and easy to interpret
– Normalizes for different ad and context lengths
IR-based ad matching
Ad vectorQuery vector
![Page 56: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/56.jpg)
Research © 2008 Yahoo!
IR-based ad matching
• Vector space models
• Probabilistic models:
– Predict, for every (ad, query) pair, the probability that the ad is relevant to the query
– Example: Okapi BM25
• Language models
![Page 57: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/57.jpg)
Research © 2008 Yahoo!
• Q1: How can we score the goodness of an ad for a context?
• Okapi BM25:
IR-based ad matching
Term Frequency
in ad
Parameters
Norm. document
length
Inverse Document Frequency
Term Frequency
in query
![Page 58: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/58.jpg)
Research © 2008 Yahoo!
• Q1: How can we score the goodness of an ad for a context?
• Okapi BM25:
• Advantages:– Different terms are weighted differently
– Tunable parameters
– Good performance
IR-based ad matching
Term Frequency
in ad
Norm. document
length
Term Frequency
in query
![Page 59: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/59.jpg)
Research © 2008 Yahoo!
IR-based ad matching
• Vector space models
• Probabilistic models
• Language models:
– Ads and queries are generated by statistical models of how words are used in the language
– What statistical models can be used?
– How do we translate query and ad generation probabilities into relevance?
![Page 60: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/60.jpg)
Research © 2008 Yahoo!
IR-based ad matching
• What statistical models can be used?
– Bigram model
– Multinomial model• Given any ad or query, we can compute the parameter
setting most likely to have generated the document
Term Frequency
Term probability (model parameters)Total length
![Page 61: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/61.jpg)
Research © 2008 Yahoo!
IR-based ad matching
How do we translate query and ad generation probabilities into relevance?
Method 1
• Compute most likelyquery and ad params
• Generate ad usingquery params
• High probability high relevance
QueryQuery
params
AdAd
params
![Page 62: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/62.jpg)
Research © 2008 Yahoo!
IR-based ad matching
How do we translate query and ad generation probabilities into relevance?
Method 2
• Compute most likelyquery and ad params
• Generate query usingad params
• High probability high relevance
QueryQuery
params
AdAd
params
![Page 63: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/63.jpg)
Research © 2008 Yahoo!
IR-based ad matching
How do we translate query and ad generation probabilities into relevance?
Method 3
• Compute most likelyquery and ad params
• Compute KL-divergencebetween params
• Low KL-divergence high relevance
QueryQuery
params
AdAd
params
![Page 64: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/64.jpg)
Research © 2008 Yahoo!
IR-based ad matching
• New methods to combine syntactic and semantic information
• For example, “A Semantic Approach to Contextual Advertising” by Broder+/SIGIR/2007– Words only provide syntactic clues
– Classify ads and queries into a common taxonomy
– Taxonomy matches provide semantic clues
![Page 65: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/65.jpg)
Research © 2008 Yahoo!
Overview
• Information Retrieval (IR)
– Techniques
– Challenges
• Machine Learning using Click Feedback
• Online Learning
![Page 66: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/66.jpg)
Research © 2008 Yahoo!
Challenges of IR-based ad matching
• Word matches might not always work
![Page 67: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/67.jpg)
Research © 2008 Yahoo!
Woes of word matching
Extract Topical info
Increases coverage,more relevant match
![Page 68: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/68.jpg)
Research © 2008 Yahoo!
Challenges of IR-based ad matching
• Word matches might not always work
• Works well for frequent words, what about rare words? Long tail, big revenue impact.– Remedy: Add more matching dimensions (phrase,…)
• Static, does not capture effect of external factors– E.g. high interest in basketball page due to an event;
dies off after the event
– Click feedback a powerful way of capturing such latent effects; difficult to do it through relevance only
• Relevance scores may not correspond to CTR; does not provide estimates of expected revenue
![Page 69: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/69.jpg)
Research © 2008 Yahoo!
Challenges of IR-based ad matching
• Heterogeneous corpus (query, ads). Single tfidf scores not applicable.
• In content match, queries long and noisy
• Partial feedback does not work
– Not scalable
• Ads are small, relevance of landing page difficult to determine (video, image, text)
![Page 70: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/70.jpg)
Research © 2008 Yahoo!
Machine Learning using Click Feedback
![Page 71: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/71.jpg)
Research © 2008 Yahoo!
Overview
• Information Retrieval (IR)
• Machine Learning using Click Feedback
– Advantages and Challenges of Click Feedback
– Feature-based models• Description
• Case Studies
– Hierarchical Models
– Matrix Factorization and Collaborative Filtering
– Challenges and Open Problems
• Online Learning
![Page 72: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/72.jpg)
Research © 2008 Yahoo!
Learning from Click Feedback
• Learning relevance from partial human-labeled training data
– Attractive but not scalable
• Users provide us direct feedback through ad clicks
– Low cost and automated learning mechanism
– Large amounts of feedback for big ad-networks
• Estimation problem:
– Estimate CTR = Pr(click| query, ad, user)
![Page 73: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/73.jpg)
Research © 2008 Yahoo!
Learning from Clicks: Challenges
• Noisy labels– Clicks (unscrupulous users gaming the system)
– Negatives (not clear; I never click on ads )
• Sparseness– (query, ad) matrix has billions of cells; long tail
• Too few data points in large number of cells; MLE has high variance
• Goal is to learn the best cells, not all cells
• Dynamic and seasonal effects– CTRs evolve; subject to seasonal effects
• Summer, Halloween,..
• Palin ads popular yesterday, not today
![Page 74: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/74.jpg)
Research © 2008 Yahoo!
Challenges continued
• Selection bias– We never showed watch ads on golf pages
• Positional bias, presentation bias– Same ad performs differently at different positions
• Slate bias– Performance of ad depends on other ads that were
displayed
![Page 75: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/75.jpg)
Research © 2008 Yahoo!
Overview
• Information Retrieval (IR)
• Machine Learning using Click Feedback
– Advantages and Challenges of Click Feedback
– Feature-based models• Description
• Case Studies
– Hierarchical Models
– Matrix Factorization and Collaborative Filtering
– Challenges and Open Problems
• Online Learning
![Page 76: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/76.jpg)
Research © 2008 Yahoo!
Feature based approach
• Query, Ad characterized by features– Query: bag-of-words, phrases, topic,…
– Ads: bag-of-words, keywords, size,…
• Query feature vector: q
• Ad feature vector: a
• Pr(Click|Q,A) = f(q,a;θ)
• Example: Logistic regression– log-odds(Pr(Click|Q,A)) = q’ W a
– W estimated from data
![Page 77: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/77.jpg)
Research © 2008 Yahoo!
Feature based models: Challenges
• Challenges– High dimensional, need to regularize (Priors)
– De-bias for positional and slate effects
– Negative events to be weighted appropriately
• Go through case studies reported in literature
![Page 78: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/78.jpg)
Research © 2008 Yahoo!
Predicting Clicks: Estimating the Click-through rates of new ads: Richardson et al, WWW 2007
• Estimate CTR of new ads in Sponsored search
• Log-odds(CTR(ad)) = wifi(ad)
• Features used:
– Bid term CTRs of related ads (from other accounts)• CTRs of all other ads with keyword “camera”
– Appearance, attention, advertiser reputation, landing page quality, relevance of bid terms to ad, bag-of-words in ad.
• Does not capture interactions between (query, ad), main focus is to estimate CTR of new ads only
• Negative events down-weighted based on eye-tracking study
![Page 79: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/79.jpg)
Research © 2008 Yahoo!
Combining relevance with Click Feedback, Chakrabarti et al, WWW 08
• Content Match application
• CTR estimation for arbitrary (page, ad) pairs
• Features :
– Bag-of-words in query, ads; relevance scores from IR
– Cross-product of words: Occurs in both page and ad
• Learn to predict click data using such features
• Prediction function amenable to WAND algorithm
– Helps with fast retrieval at serve time
![Page 80: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/80.jpg)
Research © 2008 Yahoo!
Proposed Method
• A logistic regression method model for CTR
CTR Main effect for page
(how good is the page)
Main effect for ad
(how good is the ad)
Interaction effect
(words shared by page and ad)
Model parameters
![Page 81: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/81.jpg)
Research © 2008 Yahoo!
Proposed Method
• Mp,w = tfp,w
• Ma,w = tfa,w
• Ip,a,w = tfp,w * tfa,w
• So, IR-based term frequency measures are taken into account
![Page 82: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/82.jpg)
Research © 2008 Yahoo!
Proposed Method
• Two sources of complexity
– Adding in IR scores
– Word selection for efficient learning
![Page 83: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/83.jpg)
Research © 2008 Yahoo!
Proposed Method
• How can IR scores fit into the model?
– What is the relationship between logit(pij) and cosine score?
– Quadratic relationship
Cosine scorelo
git(
p ij)
![Page 84: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/84.jpg)
Research © 2008 Yahoo!
Proposed Method
• How can IR scores fit into the model?
• This quadratic relationship can be used in two ways
– Put in cosine and cosine2 as features
– Use it as a prior
![Page 85: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/85.jpg)
Research © 2008 Yahoo!
Proposed Method
• Word selection
– Overall, nearly 110k words in corpus
– Learning parameters for each word would be:• Very expensive
• Require a huge amount of data
• Suffer from diminishing returns
– So we want to select ~1k top words which will have the most impact
![Page 86: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/86.jpg)
Research © 2008 Yahoo!
Proposed Method
• Word selection
– Data based:• Define an interaction measure for each word
• Higher values for words which have higher-than-expected CTR when they occur on both page and ad
![Page 87: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/87.jpg)
Research © 2008 Yahoo!
Experiments
Recall
Pre
cisi
on
25% lift in precision at 10% recall
![Page 88: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/88.jpg)
Research © 2008 Yahoo!
Overview
• Information Retrieval (IR)
• Machine Learning using Click Feedback
– Advantages and Challenges of Click Feedback
– Feature-based models• Description
• Case Studies
– Hierarchical Models
– Matrix Factorization and Collaborative Filtering
– Challenges and Open Problems
• Online Learning
![Page 89: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/89.jpg)
Research © 2008 Yahoo!
Regelsen and Fain, 2006
• Estimate CTR of terms by “borrowing strength” at multiple resolutions
• Hierarchical clustering of related terms
– Clustering advertiser keyword matrix
• Estimating CTR at finer resolutions by using information at coarser resolutions
– Weighted average, more weight to finer resolutions
– Weights selected heuristically, no principled approach
![Page 90: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/90.jpg)
Research © 2008 Yahoo!
Estimation in the “tail”
• A more principled approach to “Estimating Rates of Rare Events at Multiple Resolutions” [KDD/2007]
• Contextual Advertising
– Show an ad on a webpage (“impression”)
– Revenue is generated if a user clicks
– Problem: Estimate the click-through rate (CTR) of an ad on a page
• Most (ad, page) pairs have very few impressions, if any,
• and even fewer clicks
Severe data sparsity
![Page 91: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/91.jpg)
Research © 2008 Yahoo!
Estimation in the “tail”
• Use an existing, well-understood hierarchy
– Categorize ads and webpages to leaves of the hierarchy
– CTR estimates of siblings are correlated
The hierarchy allows us to aggregate data
• Coarser resolutions
– provide reliable estimates for rare events
– which then influences estimation at finer resolutions
![Page 92: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/92.jpg)
Research © 2008 Yahoo!
System overview
Retrospective data[URL, ad, isClicked]
Crawl URLs
Classify pages and ads
Rare event estimation using
hierarchy
a sample of URLs
Impute impressions, fix sampling bias
![Page 93: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/93.jpg)
Research © 2008 Yahoo!
Sampling of webpages
• Naïve strategy: sample at random from the set of URLsSampling errors in impression volume AND click
volume
• Instead, we propose:
– Crawling all URLs with at least one click, and
– a sample of the remaining URLs
Variability is only in impression volume
![Page 94: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/94.jpg)
Research © 2008 Yahoo!
Imputation of impression volume
Ad classes
Pag
e cl
asse
s
sums to #impressions on ads of this ad class
[column constraint]
sums to ∑nij + K.∑mij
[row constraint]
sums toTotal impressions
(known)
#impressions = nij + mij + xij
Clicked pool
Sampled Non-clicked
pool
Excess impressions(to be imputed)
![Page 95: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/95.jpg)
Research © 2008 Yahoo!
Imputation of impression volume
Level 0
Level i
Page hierarchy Ad hierarchy
• Region= (page node, ad node)
• Region Hierarchy A cross-product of the page
hierarchy and the ad hierarchy
Page classes Ad classes
Region
![Page 96: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/96.jpg)
Research © 2008 Yahoo!
Imputation of impression volume
sums to
[block constraint]
Level i
Level i+1
![Page 97: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/97.jpg)
Research © 2008 Yahoo!
Imputing xij
Level i
Level i+1
Iterative Proportional Fitting [Darroch+/1972]
• Initialize xij = nij + mij
• Iteratively scale xij values to match row/col/block constraint
• Ordering of constraints: top-down, then bottom-up, and repeat
blockPage classes Ad classes
![Page 98: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/98.jpg)
Research © 2008 Yahoo!
Imputation: Summary
• Given
– nij (impressions in clicked pool)
– mij (impressions in sampled non-clicked pool)
– # impressions on ads of each ad class in the ad hierarchy
• We get
– Estimated impression volume Ñij = nij + mij + xij
in each region ij of every level
![Page 99: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/99.jpg)
Research © 2008 Yahoo!
System overview
Retrospective data[page, ad, isclicked]
Crawl Pages
Classify pages and ads
Rare event estimation using
hierarchy
a sample of pages
Impute impressions, fix sampling bias
![Page 100: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/100.jpg)
Research © 2008 Yahoo!
Rare rate modeling
1. Freeman-Tukey transform: – yij = F-T(clicks and impressions at ij)
≈ transformed-CTR
– Variance stabilizing transformation: Var(y) is independent of E[y] needed in further modeling
![Page 101: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/101.jpg)
Research © 2008 Yahoo!
SijSparent(ij)
Rare rate modeling
2. Generative Model (Tree-structured Markov Model)
yij yparent(ij)
covariates βij variance Vij
Unobserved “state”
variance Wij
Vparent(ij)
βparent(ij)
Wparent(ij)
![Page 102: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/102.jpg)
Research © 2008 Yahoo!
Rare rate modeling
• Model fitting with a 2-pass Kalman filter:
– Filtering: Leaf to root
– Smoothing: Root to leaf
• Linear in thenumber of regions
![Page 103: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/103.jpg)
Research © 2008 Yahoo!
Tree-structured Markov model
rVrWRootWRootSrpaSrwrWNrw
rwrpaSrS
rSd(r).rd
rVrSrdT
ruNry
/on Depends :Smoothing
.0;)( indep );,0(~)(
smoothing) (requireregion per one effects, random: levelat covariatesfor t vector coefficien:)(
),)((~
Model Markov
)(
1 1 1
'
)(
'
/),Corr( ;)(
;/rd
i
l
i
l
i
iiir
rdrrr
WWllWSVar
WWNVV
![Page 104: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/104.jpg)
Research © 2008 Yahoo!
Scalable Model fitting Multi-resolution Kalman filter
1994) Rubin, and(Liu algorithm ECME :componets Variance
downtree , Uptree:steps Two
n computatio )3regionchildren O(# region,parent each At
regionsparent ofnumber on Dependsregions ofnumber in thelinear y"essentiall" Algorithm
2002) Cressie, and (Huang algorithmfilter Kalman -
:
smoothingfiltering
}r{S states ofPosterior
![Page 105: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/105.jpg)
Research © 2008 Yahoo!
Multi-Resolution Kalman filter: Mathematical overview
parent from info usingchildren on info Update
)(
parentfor available info recombine
children fromn informatio Combine
parentfor child ofon contributiCollect
);1)(),(( ;)(
equations; state Invert the
updatesBayesian standard using nodes leaf ofposterior Update
)(
step Smoothing
:step Filtering
downtree
rdrdcorrrBrrSrBrpa
S
uptree
![Page 106: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/106.jpg)
Research © 2008 Yahoo!
Experiments
• 503M impressions
• 7-level hierarchy of which the top 3 levels were used
• Zero clicks in
– 76% regions in level 2
– 95% regions in level 3
• Full dataset DFULL, and a 2/3 sample DSAMPLE
![Page 107: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/107.jpg)
Research © 2008 Yahoo!
Experiments
• Estimate CTRs for all regions R in level 3 with zero clicks in DSAMPLE
• Some of these regions R>0 get clicks in DFULL
• A good model should predict higher CTRs for R>0 as against the other regions in R
![Page 108: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/108.jpg)
Research © 2008 Yahoo!
Experiments
• We compared 4 models
– TS: our tree-structured model
– LM (level-mean): each level smoothed independently
– NS (no smoothing): CTR proportional to 1/Ñ
– Random: Assuming |R>0| is given, randomly predict the membership of R>0 out of R
![Page 109: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/109.jpg)
Research © 2008 Yahoo!
Experiments
TS
Rando
m
LM, N
S
![Page 110: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/110.jpg)
Research © 2008 Yahoo!
Experiments
Enough impressions little “borrowing”
from siblings
Few impressions Estimates depend more on siblings
![Page 111: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/111.jpg)
Research © 2008 Yahoo!
Related Work
• Multi-resolution modeling
– studied in time series modeling and spatial statistics [Openshaw+/79, Cressie/90, Chou+/94]
• Imputation
– studied in statistics [Darroch+/1972]
• Application of such models to estimation of such rare events (rates of ~10-3) is novel
![Page 112: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/112.jpg)
Research © 2008 Yahoo!
Summary
• A method to estimate
– rates of extremely rare events
– at multiple resolutions
– under severe sparsity constraints
• The method has two parts
– Imputation incorporates hierarchy, fixes sampling bias
– Tree-structured generative model extremely fast parameter fitting
![Page 113: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/113.jpg)
Research © 2008 Yahoo!
Overview
• Information Retrieval (IR)
• Machine Learning using Click Feedback
– Advantages and Challenges of Click Feedback
– Feature-based models• Description
• Case Studies
– Hierarchical Models
– Matrix Factorization and Collaborative Filtering
– Challenges and Open Problems
• Online Learning
![Page 114: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/114.jpg)
Research © 2008 Yahoo!
Collaborative Filtering
• Collaborative filtering– Similarity based methods
)()(
/iNj
ijujiNj
ijuisrsr
Rating (CTR) for query u of ad i
Ad-ad similarity matrix
Local neighborhood of ad i
![Page 115: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/115.jpg)
Research © 2008 Yahoo!
Collaborative Filtering
• Collaborative filtering– Similarity based methods
– Possible adaptation
– Challenges: • Learning similarity
• Simultaneously incorporating query and ad similarities
)()(
/iNj
ijujiNj
ijuisrsr
)()(/
);,()(odds-log
aNjqjqj
aNjqjqa
qaqa
szsz
zfp θaq
Feature-based model
Collaborative filtering model
![Page 116: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/116.jpg)
Research © 2008 Yahoo!
Matrix Factorization
• Matrix Factorization– Each query (ad) is a linear
combination of latent factors
– Solve for factors, under someregularization and constraints
r
kakqkqa
vufp1
)()(odds-log θa;q,
Factor coefficients for query
Factor coefficients
for ad
![Page 117: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/117.jpg)
Research © 2008 Yahoo!
Matrix Factorization
• Matrix Factorization
• Bi-clustering
– Predictive Discrete latent factor models, Agarwal and Merugu, KDD 07.
r
kakqkqa
vufp1
)()(odds-log θa;q,
cluster ad:(a)cluster;Query :)(
);,()(odds-log)(),(
q
zaqfpaqqa
![Page 118: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/118.jpg)
Research © 2008 Yahoo!
Overview
• Information Retrieval (IR)
• Machine Learning using Click Feedback
– Advantages and Challenges of Click Feedback
– Feature-based models• Description
• Case Studies
– Hierarchical Models
– Matrix Factorization and Collaborative Filtering
– Challenges and Open Problems
• Online Learning
![Page 119: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/119.jpg)
Research © 2008 Yahoo!
Challenges of Feature-based models
• Learns from clicks but still misses context in many instances as in relevance based approach
• Introducing features that are too granular makes it hard to learn CTR reliably
• Does not capture the dynamics of the system
• Training cost is high
• Slow prediction functions inadmissible due to latency constraints
![Page 120: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/120.jpg)
Research © 2008 Yahoo!
Challenges of Feature-based models
• Other methods– Boosting, Neural nets, Decision Trees, Random Forests, ……
• Local models– Mixture of experts: Fit local, think global
• Hierarchical modeling with multiple trees– User interest, query, ad,..
– Each tree is different
– How to perform smoothing with multiple disparate trees?
L
1kk
A)Q,|click(A)Q,|P(clickk
P
![Page 121: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/121.jpg)
Research © 2008 Yahoo!
Challenges of Feature-based models
• Combining cold start with warm start together main challenge in collaborative filtering based methods
• We believe, solving basic issues more challenging
– Positional bias
– Selection bias
– Correlation in ads on a slate
– Dynamic CTR; seasonal variations
![Page 122: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/122.jpg)
Research © 2008 Yahoo!
Online learning
![Page 123: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/123.jpg)
Research © 2008 Yahoo!
Overview
• Information Retrieval (IR)
• Machine Learning using Click Feedback
• Online Learning
![Page 124: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/124.jpg)
Research © 2008 Yahoo!
Online learning for ad matching
• All previous approaches learn from historical data
• This has several drawbacks:
– Slow response to emerging patterns in the data• due to special events like elections, …
– Initial systemic biases are never corrected• If the system has never shown “sound system dock” ads
for the “iPod” query, it can never learn if this match is good
– System needs to be retrained periodically
![Page 125: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/125.jpg)
Research © 2008 Yahoo!
Online learning for ad matching
• Solution: Combining exploitation with exploration
– Exploitation: Pick ads that are good according to current model
– Exploration: Pick ads that increase our knowledge about the entire space of ads
• Multi-armed bandits
– Background
– Applications to online advertising
– Challenges and Open Problems
![Page 126: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/126.jpg)
Research © 2008 Yahoo!
Background: Bandits
Bandit “arms”
p1 p2 p3(unknown payoff
probabilities)
• “Pulling” arm i yields a reward:
• reward = 1 with probability pi (success)
• reward = 0 otherwise (failure)
![Page 127: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/127.jpg)
Research © 2008 Yahoo!
Background: Bandits
• Goal: Pull arms sequentially so as to maximize the total expected reward
– Estimate payoff probabilities pi
– Bias the estimation process towards better arms
Bandit “arms”
p1 p2 p3(unknown payoff
probabilities)
![Page 128: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/128.jpg)
Research © 2008 Yahoo!
Background: Bandits
• An algorithm to sequentially pick the arms is called a bandit policy
• Regret of a policy = how much extra payoff could be gained in expectation if the best arm is always pulled
– Of course, the best arm is not known to the policy
– Hence, the regret is the price of exploration
– Low regret implies that the policy quickly converges to the best arm
• What is the optimal policy?
![Page 129: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/129.jpg)
Research © 2008 Yahoo!
Background: Bandits
• Which arm should be pulled next?– Not necessarily what looks best right now, since it might have
had a few lucky successes
– Seems to depend on some complicated function of the successes and failures of all arms
argmax g(s1, f1, s2, f2, …, sk, fk) ?
Number of successes
Number of failures
![Page 130: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/130.jpg)
Research © 2008 Yahoo!
Background: Bandits
• What is the optimal policy?
• Consider a bandit which
– has an infinite time horizon, but
– future rewards are geometrically discountedRtotal = R(1) + γ.R(2) + γ2.R(3) + … (0<γ<1)
• Theorem [Gittins/1979]: The optimal policy decouples and solves a bandit problem for each arm independently
argmax {g1(s1, f1), g2(s2, f2), …, gk(sk, fk)}
argmax g(s1, f1, s2, f2, …, sk, fk) ?
![Page 131: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/131.jpg)
Research © 2008 Yahoo!
Background: Bandits
• What is the optimal policy?
• Theorem [Gittins/1979]: The optimal policy decouples and solves a bandit problem for each arm independently
– Significantly reduces the dimension of the problem space
– Gives a minimum regret bound of O(log T)
– But, the optimal functions gi(si, fi) are hard to compute
– Need approximate methods…
![Page 132: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/132.jpg)
Research © 2008 Yahoo!
Background: Bandits
Bandit Policy
1. Assign priority to each arm
2. “Pull” arm with max priority, and observe reward
3. Update priorities
Priority 1
Priority 2
Priority 3
Allocation
Estimation
![Page 133: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/133.jpg)
Research © 2008 Yahoo!
Background: Bandits
• One common policy is UCB1 [Auer/2002]
Number of successes
Number of failures
Total number of observations
Number of observations of
arm i
Observed payoff
Factor representing uncertainty
![Page 134: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/134.jpg)
Research © 2008 Yahoo!
Background: Bandits
• As total observations T becomes large:
– Observed payoff tends asymptotically towards the true payoff probability
– The system never completely “converges” to one best arm; only the rate of exploration tends to zero
Observed payoff
Factor representing uncertainty
![Page 135: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/135.jpg)
Research © 2008 Yahoo!
Background: Bandits
• Sub-optimal arms are pulled O(log T) times
• Hence, UCB1 has O(log T) regret
• This is the lowest possible regret
Observed payoff
Factor representing uncertainty
![Page 136: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/136.jpg)
Research © 2008 Yahoo!
Online learning for ad matching
• Solution: Combining exploitation with exploration
– Exploitation: Pick ads that are good according to current model
– Exploration: Pick ads that increase our knowledge about the entire space of ads
• Multi-armed bandits
– Background
– Applications to online advertising
– Challenges and Open Problems
![Page 137: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/137.jpg)
Research © 2008 Yahoo!
Background: BanditsW
ebp
age
1
Bandit “arms”
We
bpa
ge 2
We
bpa
ge 3
= ads
~106 ads
~109 pages
![Page 138: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/138.jpg)
Research © 2008 Yahoo!
Background: Bandits
Ads
Web
page
s
Content Match = A matrix
• Each row is a bandit
• Each cell has an unknown CTR
One bandit
Unknown CTR
![Page 139: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/139.jpg)
Research © 2008 Yahoo!
Background: Bandits
Why not simply apply a bandit policy directly to our problem?
• Convergence is too slow ~109 bandits, with ~106 arms per bandit
• Additional structure is available, that can help Taxonomies
![Page 140: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/140.jpg)
Research © 2008 Yahoo!
Taxonomies for dimensionality reduction
Root
Apparel Computers Travel
• Already exist
• Actively maintained
• Existing classifiers to map pages and ads to taxonomy nodes
Page/Ad
A bandit policy that uses this structure can be faster
![Page 141: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/141.jpg)
Research © 2008 Yahoo!
Outline
Multi-level Bandit Policy for Content Match
• Experiments
• Summary
![Page 142: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/142.jpg)
Research © 2008 Yahoo!
Multi-level Policy
Ads
Webpages
… …
……
……
classes
classes
Consider only two levels
![Page 143: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/143.jpg)
Research © 2008 Yahoo!
Multi-level Policy
ApparelCompu-
ters Travel
… …
……
……
Consider only two levels
Tra
vel
Co
mp
u-
ters
Ap
pare
l
Ad parent classes
Ad child classes
Block
One bandit
![Page 144: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/144.jpg)
Research © 2008 Yahoo!
Multi-level Policy
ApparelCompu-
ters Travel
… …
……
……
Key idea: CTRs in a block are homogeneous
Ad parent classes
Block
One bandit
Tra
vel
Co
mp
u-
ters
Ap
pare
l Ad child classes
![Page 145: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/145.jpg)
Research © 2008 Yahoo!
Multi-level Policy
• CTRs in a block are homogeneous
– Used in allocation (picking ad for each new page)
– Used in estimation (updating priorities after each observation)
![Page 146: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/146.jpg)
Research © 2008 Yahoo!
Multi-level Policy
• CTRs in a block are homogeneous
Used in allocation (picking ad for each new page)
– Used in estimation (updating priorities after each observation)
![Page 147: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/147.jpg)
Research © 2008 Yahoo!C
A C T
AT
Multi-level Policy (Allocation)
?
Page classifier
• Classify webpage page class, parent page class
• Run bandit on ad parent classes pick one ad parent class
![Page 148: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/148.jpg)
Research © 2008 Yahoo!C
A C T
AT
Multi-level Policy (Allocation)
• Classify webpage page class, parent page class
• Run bandit on ad parent classes pick one ad parent class
• Run bandit among cells pick one ad class
• In general, continue from root to leaf final ad
?
Page classifier
ad
![Page 149: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/149.jpg)
Research © 2008 Yahoo!C
A C T
AT
ad
Multi-level Policy (Allocation)
Bandits at higher levels
• use aggregated information
• have fewer bandit arms Quickly figure out the best ad parent class
Page classifier
![Page 150: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/150.jpg)
Research © 2008 Yahoo!
Multi-level Policy
• CTRs in a block are homogeneous
Used in allocation (picking ad for each new page)
Used in estimation (updating priorities after each observation)
![Page 151: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/151.jpg)
Research © 2008 Yahoo!
Multi-level Policy (Estimation)
• CTRs in a block are homogeneous
– Observations from one cell also give information about others in the block
– How can we model this dependence?
![Page 152: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/152.jpg)
Research © 2008 Yahoo!
Multi-level Policy (Estimation)
• Shrinkage Model
Scell | CTRcell ~ Bin (Ncell, CTRcell)
CTRcell ~ Beta (Paramsblock)
# clicks in cell# impressions
in cell
All cells in a block come from the same distribution
![Page 153: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/153.jpg)
Research © 2008 Yahoo!
Multi-level Policy (Estimation)
• Intuitively, this leads to shrinkage of cell CTRs towards block CTRs
E[CTR] = α.Priorblock + (1-α).Scell/Ncell
Estimated CTR
Beta prior (“block CTR”)
Observed CTR
![Page 154: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/154.jpg)
Research © 2008 Yahoo!
Experiments
Root
20 nodes
221 nodes…
~7000 leaves
Taxonomy structure
We use these 2 levels
Depth 0
Depth 7
Depth 1
Depth 2
![Page 155: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/155.jpg)
Research © 2008 Yahoo!
Experiments
• Data collected over a 1 day period
• Collected from only one server, under some other ad-matching rules (not our bandit)
• ~229M impressions
• CTR values have been linearly transformed for purposes of confidentiality
![Page 156: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/156.jpg)
Research © 2008 Yahoo!
Experiments (Multi-level Policy)
Multi-level gives much higher #clicks
Number of pulls
Clic
ks
![Page 157: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/157.jpg)
Research © 2008 Yahoo!
Experiments (Multi-level Policy)
Multi-level gives much better Mean-Squared Error it has learnt more from its explorations
Mea
n-S
quar
ed E
rror
Number of pulls
![Page 158: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/158.jpg)
Research © 2008 Yahoo!
Experiments (Shrinkage)
Number of pulls Number of pullsMea
n-S
quar
ed E
rror
Clic
ks without shrinkage
with shrinkage
Shrinkage improved Mean-Squared Error, but no gain in #clicks
![Page 159: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/159.jpg)
Research © 2008 Yahoo!
Summary
• Taxonomies exist for many datasets
• They can be used for
– Dimensionality Reduction
– Multi-level bandit policy higher #clicks
– Better estimation via shrinkage models better MSE
![Page 160: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/160.jpg)
Research © 2008 Yahoo!
Online learning for ad matching
• Solution: Combining exploitation with exploration
– Exploitation: Pick ads that are good according to current model
– Exploration: Pick ads that increase our knowledge about the entire space of ads
• Multi-armed bandits
– Background
– Applications to online advertising
– Challenges and Open Problems
![Page 161: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/161.jpg)
Research © 2008 Yahoo!
Challenges and Open Problems
• Bandit policies typically assume stationarity
• But, sudden changes are the norm in the online advertising world:
– Ads may be suddenly removed when they run out of budget
– New ads are constantly added to the system
– The total number of ads is huge, and full exploration may be too costly
– Mortal multi-armed bandits [NIPS/2008]
![Page 162: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/162.jpg)
Research © 2008 Yahoo!
Mortal Multi-armed Bandits
• Traditional bandit policies like UCB1 spend a large fraction of their initial pulls on exploration
– Hard-earned knowledge may be lost due to finite arm lifetimes
• Method 1 (Sampling):
– Pick a random sample from the set of available arms
– Run UCB1 on sample, until some fraction of arms in the sample are lost
– Pro: Quicker convergence, more exploitation
– Con: Best arm in the sample may be worse than best arm overall
– Pick sample size to control this tradeoff
![Page 163: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/163.jpg)
Research © 2008 Yahoo!
Mortal Multi-armed Bandits
• Traditional bandit policies like UCB1 spend a large fraction of their initial pulls on exploration
– Hard-earned knowledge may be lost due to finite arm lifetimes
• Method 2 (Payoff threshold):
– New bandit policy: If the observed payoff of any arm is higher than a threshold, pull it till it expires
– Pro: Good arms, once found, are exploited quickly
– Con: While exploiting good arms, the best arm may be starving and may expire without being found
– Pick threshold to control this tradeoff
![Page 164: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/164.jpg)
Research © 2008 Yahoo!
Mortal Multi-armed Bandits
• Challenges:
– Selecting the critical sample size or threshold correctly, for arbitrary payoff distributions
– What if even the payoff distribution is unknown?
![Page 165: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/165.jpg)
Research © 2008 Yahoo!
Challenges and Open Problems
• Mortal multi-armed bandits
• What if the bandit policy has some information about the budget?
– The bandit policy can control which arms expire, and when
– “Handling Advertisements of Unknown Quality in Search Advertising” by Pandey+/NIPS/2006
• Combining budgets with extra knowledge of ad CTRs
– E.g., Using an ad taxonomy
• Using a bandit scheme to infer/correct an ad taxonomy
![Page 166: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/166.jpg)
Research © 2008 Yahoo!
Conclusions
![Page 167: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/167.jpg)
Research © 2008 Yahoo!
Conclusions
• We provided an introduction to Online Advertising
– Discussed the eco-system and various actors involved
– Discussed different flavors of online advertising• Sponsored Search, Content Match, Display Advertising
![Page 168: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/168.jpg)
Research © 2008 Yahoo!
Conclusions
Online Advertising
Revenue Models
Advertising Setting
Misc.
CPM CPC CPA
Display Content Match
Sponsored Search
Ad exchanges
![Page 169: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/169.jpg)
Research © 2008 Yahoo!
Conclusions
• Outlined associated statistical challenges
– Sponsored search, Content Match, Display
• We believe the following to be a technical roadmap
Offline Modeling Online ModelsTime series
Explore/Exploit
Multi-armed bandits
Regression, collaborative filtering, mixture of experts
Multi-resolution models
Selection bias Slate correlation
Noisy labels
![Page 170: Research © 2008 Yahoo! Statistical Challenges in Online Advertising Deepak Agarwal Deepayan Chakrabarti (Yahoo! Research)](https://reader030.vdocuments.us/reader030/viewer/2022032607/56649ec15503460f94bcd1b6/html5/thumbnails/170.jpg)
Research © 2008 Yahoo!
Conclusions
• Offline Modeling– By far the best studied so far
– Not a careful study of selection bias, slate correlations, noisy labels. Good opportunity here
– More emphasis on matrix structure, goal is to estimate interactions
• Explore/Exploit– Some work using multi-armed bandits; long way to go
• Time series model to capture temporal aspects– Little work
• Holistic approach that combines all components in a principled way