foursquare data quick talk @ bbq check in 3/21/12
DESCRIPTION
TRANSCRIPT
E
@benlee
foursquare explore +assorted data fun
Overview
foursquare data stats explore similar places interesting phrases venue transitions
2
15,000,000+ people
30,000,000+ places
1,500,000,000+ check-ins
Data
4
E
foursquare explore
Explore Flow
• Query parsing
• Retrieval
• Ranking
• Reason generation
Overview: RetrievalVenue Index
Candidate Venues
Social Interactions
Check-in History
Venue attributes
PersonalInteractions
Social Interactions
Venue attributes
PersonalInteractions
Global scores Social Scores PersonalScores
Top K venues
Global reasons
Social reasons
Personalreasons
Overview: Ranking / Reasons
Global attributes
• Time decayed popularity
• Time of day / day of week filtering
• Total interactions (# tips, todos etc)
Blue bottle
Social / Personal scores
• # Friends that have checkins
• Tips left by friends, etc
• Prior visits by user
• Similar places user has been in the past
Venue Similarity
Incredibly SparseMatrixU
sers
Venuesvi vj
for all i,jsim(vi, vj)
Venue Similarity w/ MapReduce
visited venues map
reduce
vi, vj
vi, vj
vi, vj
user
emit “all” pairs of visited venues for each user
Sum up each user’s score contribution to this pair of venues
score
score
key
key
score scorescore ...
...
final score
17
18
Reason generation
- “Dennis left a tip here”- “You and 4 friends have been here”- “This place is on your To-do list”
19
Improving explore
New signals– Tourist / Local– Everyday / Once-and-awhile– Expert rank
Retrieval improvment–Synonyms / query rewriting–Query category matching–Menu text
20
E
assorted data fun
Interesting phrases
• Hack day project on tip text
• Build a n-gram model, find “improbable” phrases relative to background corpus
• Inspired by Amazon’s statistically improbable phrases (SIPs) feature
barn cellar, famous pig, grey spider, old sheep, egg sac
69 Modern Classics Condensed via Amazon’s SIPs
stillsuit manufacturer, panoplia propheticus, gom jabbar, inkvine scar, ducal signet, factory crawler, poison snooper
69 Modern Classics Condensed via Amazon’s SIPs
drip coffees, best coffee, jack dorsey, no wifi, using square
Interesting Venue Phrases
Interesting Venue Phrases
bootie sf, hubba hubba revue, death guild
Interesting Venue Phrases
salted caramel, balsamic strawberry, roasted banana, honey lavender,
City level phrases
deep dish pizza, celery salt, hot dog
clam chowder, lobster roll, red sox
bread pudding, cafe au lait, bloody mary
breakfast tacos, mexican martinis, fried avocado
fried chicken, blue bottle, french toast
Phrases by time
Venue transitions
Users check in ~3 times a dayWhat can we learn from where they go next?
30
31
32
Thanks
foursquare.com/jobs
@benlee