![Page 1: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/1.jpg)
1yahooresearchberkeley.comRahul Nair
What’s in a place?Adventures with Location-Aware Media
Rahul Nair
Yahoo! Research Berkeley
![Page 2: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/2.jpg)
2yahooresearchberkeley.comRahul Nair
Flickr “geotagged”
20+ million images
Can we do better?
![Page 3: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/3.jpg)
3yahooresearchberkeley.comRahul Nair
Talk Outline
• Extracting information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
![Page 4: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/4.jpg)
4yahooresearchberkeley.comRahul Nair
Attraction Map of Paris
Stanley Milgram, 1976. Psychological Maps of Paris
![Page 5: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/5.jpg)
5yahooresearchberkeley.comRahul Nair
Attraction Map of Paris
Y!RB, 2007.
![Page 6: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/6.jpg)
6yahooresearchberkeley.comRahul Nair
Location-driven Modeling
• Derive meaningful data about map regions
• E.g., representative tags, photos
![Page 7: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/7.jpg)
7yahooresearchberkeley.comRahul Nair
Data Description
![Page 8: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/8.jpg)
8yahooresearchberkeley.comRahul Nair
Issues
• Sparse data set
• Photographer bias
– In location
– In tags
• Incorrect data
![Page 9: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/9.jpg)
9yahooresearchberkeley.comRahul Nair
Heuristics
• Number of photographs denotes the “importance” of a location
• Users will use a common subset of tags to describe objects/locations
• Concentrated tag usage indicates descriptiveness
![Page 10: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/10.jpg)
10yahooresearchberkeley.comRahul Nair
Algorithm
• Clustering: k-Means, get set of k clusters
• “Document” C is bag of all tags in cluster
• For each tag in C calculate:
– TF = |P(C,t)|
– IDF = |P(R)| / |P(R, t)|
– UF = |U(C,t)|/|U(C)|
![Page 11: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/11.jpg)
11yahooresearchberkeley.comRahul Nair
Scoring
• Score (t) = TF * IDF * UF
• Threshold values
– 30+ photographs
– Minimum 3 users
– Score > 1
• Final dataset: (tag, score, latitude, longitude)
![Page 12: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/12.jpg)
12yahooresearchberkeley.comRahul Nair
Talk Outline
• Mining information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
![Page 13: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/13.jpg)
13
DEMO
![Page 14: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/14.jpg)
14yahooresearchberkeley.comRahul Nair
Precomputation
• Divide the world into equal sized non-overlapping tiles
• Compute and store the tags for each tile
• Repeat for different zoom
levels
![Page 15: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/15.jpg)
15yahooresearchberkeley.comRahul Nair
Retrieval
• Find the tile level closest in size to the request area
• Select the tiles that fully cover the request area
• Return the tags that fall within the request area
![Page 16: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/16.jpg)
16yahooresearchberkeley.comRahul Nair
User Study
10 subjects
• 6 female, 4 male
• Ages 20-60
• Varying technical knowledge
• No geotagged photos of their own
![Page 17: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/17.jpg)
17yahooresearchberkeley.comRahul Nair
Experiment tasks
• Vacation recap
• San Francisco tour
• Explore a new city
![Page 18: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/18.jpg)
18yahooresearchberkeley.comRahul Nair
Recall
Reminded the subject about locations
• “It brings out memories”
• “Oh my God! This place has the best restaurants”
• “We wanted to see the Polynesian Cultural Center"
![Page 19: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/19.jpg)
19yahooresearchberkeley.comRahul Nair
Discovery
Participants discovered previously unknown locations and events
– “I’ve never heard of this festival”
– “There is car racing which I'd probably go see”
![Page 20: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/20.jpg)
20yahooresearchberkeley.comRahul Nair
Needle & Haystack
• Excellent visualization of the Haystack
• Hard to find specific information
– “Where was Culver City again?”
• No way to search
– “I guess what I’m looking for are bull fighting pictures”
![Page 21: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/21.jpg)
21yahooresearchberkeley.comRahul Nair
Other Responses
• Gets the “vibe” of a place
• Share with other people
• Tags did not always match the mental model of a location
• Wanted more tags
• Want more info about tags
![Page 22: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/22.jpg)
22yahooresearchberkeley.comRahul Nair
Conclusions
• Extract meaningful aggregate information from georeferenced data
• Allows users to explore locations in a new way
• Users like using the overview but also want the ability to search
![Page 23: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/23.jpg)
23yahooresearchberkeley.comRahul Nair
Future work
• Adding search capability
• Show photos in places with no tags
• Differentiate locations and events
• Apply to other types of georeferenced data
![Page 24: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/24.jpg)
24yahooresearchberkeley.comRahul Nair
Talk Outline
• Mining information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
![Page 25: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/25.jpg)
25yahooresearchberkeley.comRahul Nair
Current Mobile Experience
• Difficult to share (or even save!)
• Hard to find
– No context
– No semantic information
Current mobile experience?
![Page 26: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/26.jpg)
26yahooresearchberkeley.comRahul Nair
Current Mobile Experience
• Difficult to share (or even save!)
• Hard to find
– No context
– No semantic information
![Page 27: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/27.jpg)
27yahooresearchberkeley.comRahul Nair
ZoneTag Experience
• 2-click upload (same key!)
• Photo uploaded with location and time metadata
![Page 28: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/28.jpg)
28yahooresearchberkeley.comRahul Nair
Where does location come from?
• Bluetooth GPS (when available)
• User-contributed cell tower mapping
![Page 29: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/29.jpg)
29yahooresearchberkeley.comRahul Nair
ZoneTag Experience
• Tagging made easy
– Tag/annotate your photos from the phone
![Page 30: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/30.jpg)
30yahooresearchberkeley.comRahul Nair
Where do tags come from?
• Tags I used in this context (`home’)
• Tags my friends used in this context
• Tags other people used in this context (‘Ricoh’, ‘California Research Center’)
– E.g., TagMaps data
![Page 31: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/31.jpg)
31yahooresearchberkeley.comRahul Nair
Where do tags come from?
• Stuff around you:
– Yahoo! Local
– Upcoming.org
• Stuff from you (any RSS 2.0 feed):
– Calendar
– Favorite hangouts (Wayfaring, Plazes, Socialight)
![Page 32: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/32.jpg)
32yahooresearchberkeley.comRahul Nair
Example of Tagging
![Page 33: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/33.jpg)
34yahooresearchberkeley.comRahul Nair
Talk Outline
• Mining information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
![Page 34: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/34.jpg)
35yahooresearchberkeley.comRahul Nair
Flickr “geotagged”
20+ million images
Can we do mobile?
![Page 35: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/35.jpg)
36yahooresearchberkeley.comRahul Nair
Strengths of Mobile
• Personal
• Easy to access
• Networked
• Context aware
The ultimate photo wallet
![Page 36: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/36.jpg)
37yahooresearchberkeley.comRahul Nair
Design Goals
• Engagement and Discovery
– Spatial
– Social
• Allow customization
• Complete access to your photos
• Search and Filtering capability
![Page 37: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/37.jpg)
38yahooresearchberkeley.comRahul Nair
Channel Metaphor
• Each row is a single channel
• Navigate using 4 way
– Left & right to browse channel (infinite scroll)
– Up & down to change channels
![Page 38: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/38.jpg)
39yahooresearchberkeley.comRahul Nair
Detail view
• Enlarge photo
– Scroll through channel
• Photo details
• Add comments
• Add to favourites
![Page 39: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/39.jpg)
40yahooresearchberkeley.comRahul Nair
Nearby photos channel
• Photos from the users current location
• Local Highlights
• My Nearby
![Page 40: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/40.jpg)
41yahooresearchberkeley.comRahul Nair
Social Channels
• Contacts photos
– Expanded view
• Recent Activity
![Page 41: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/41.jpg)
42yahooresearchberkeley.comRahul Nair
My Stuff Channel
• My Photos
– By Location
– By Tag
• My Photo Wallet
• My Favourites
![Page 42: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/42.jpg)
43yahooresearchberkeley.comRahul Nair
Custom Channels
• Users can create channels to match their interests
– Tags
– Groups
– Location aware (dogs near me)
![Page 43: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/43.jpg)
44yahooresearchberkeley.comRahul Nair
Search and Filter
![Page 44: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/44.jpg)
45yahooresearchberkeley.comRahul Nair
Try it out
http://zurfer.research.yahoo.com
![Page 45: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/45.jpg)
46yahooresearchberkeley.comRahul Nair
Conclusions
• It is possible to extract information from georeferenced media
• Users like browsing the extracted data
• It can help users tag new media
• We hope it helps them browse on a mobile device
![Page 46: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.us/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/46.jpg)
47
Questions?
Rahul [email protected]
http://tagmaps.research.yahoo.comhttp://zonetag.research.yahoo.com http://zurfer.research.yahoo.com