foursquare data quick talk @ bbq check in 3/21/12

E

@benlee

foursquare explore +assorted data fun

Overview

foursquare data stats explore similar places interesting phrases venue transitions

2

15,000,000+ people

30,000,000+ places

1,500,000,000+ check-ins

Data

4

E

foursquare explore

Explore Flow

• Query parsing

• Retrieval

• Ranking

• Reason generation

Overview: RetrievalVenue Index

Candidate Venues

Social Interactions

Check-in History

Venue attributes

PersonalInteractions

Social Interactions

Venue attributes

PersonalInteractions

Global scores Social Scores PersonalScores

Top K venues

Global reasons

Social reasons

Personalreasons

Overview: Ranking / Reasons

Global attributes

• Time decayed popularity

• Time of day / day of week filtering

• Total interactions (# tips, todos etc)

Blue bottle

Social / Personal scores

• # Friends that have checkins

• Tips left by friends, etc

• Prior visits by user

• Similar places user has been in the past

Venue Similarity

Incredibly SparseMatrixU

sers

Venuesvi vj

for all i,jsim(vi, vj)

Venue Similarity w/ MapReduce

visited venues map

reduce

vi, vj

vi, vj

vi, vj

user

emit “all” pairs of visited venues for each user

Sum up each user’s score contribution to this pair of venues

score

score

key

key

score scorescore ...

...

final score

17

18

Reason generation

- “Dennis left a tip here”- “You and 4 friends have been here”- “This place is on your To-do list”

19

Improving explore

New signals– Tourist / Local– Everyday / Once-and-awhile– Expert rank

Retrieval improvment–Synonyms / query rewriting–Query category matching–Menu text

20

E

assorted data fun

Interesting phrases

• Hack day project on tip text

• Build a n-gram model, find “improbable” phrases relative to background corpus

• Inspired by Amazon’s statistically improbable phrases (SIPs) feature

barn cellar, famous pig, grey spider, old sheep, egg sac

69 Modern Classics Condensed via Amazon’s SIPs

stillsuit manufacturer, panoplia propheticus, gom jabbar, inkvine scar, ducal signet, factory crawler, poison snooper

69 Modern Classics Condensed via Amazon’s SIPs

drip coffees, best coffee, jack dorsey, no wifi, using square

Interesting Venue Phrases

Interesting Venue Phrases

bootie sf, hubba hubba revue, death guild

Interesting Venue Phrases

salted caramel, balsamic strawberry, roasted banana, honey lavender,

City level phrases

deep dish pizza, celery salt, hot dog

clam chowder, lobster roll, red sox

bread pudding, cafe au lait, bloody mary

breakfast tacos, mexican martinis, fried avocado

fried chicken, blue bottle, french toast

Phrases by time

Venue transitions

Users check in ~3 times a dayWhat can we learn from where they go next?

30

31

32

Thanks

foursquare.com/jobs

@benlee