geotag propagation in social networksgeotag propagation in … · user trust modeling in panoramio...
TRANSCRIPT
![Page 1: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/1.jpg)
1
Geotag Propagation in Social NetworksGeotag Propagation in Social NetworksBased on User Trust Model
Ivan Ivanov, Peter Vajda, Jong-Seok Lee,Lutz Goldmann, Touradj Ebrahimi
Multimedia Signal Processing GroupEcole Polytechnique Federale de Lausanne, Switzerland
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 2: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/2.jpg)
2
We introduce users in our system for geotagging in order to
Motivation
We introduce users in our system for geotagging in order to simulate a real social network
GPS coordinates to derive geographical annotation, which are not available for the majority of web images and photosare not available for the majority of web images and photos
A GPS sensor in a camera provides only the location of the photographer instead of that of the captured landmark
Sometimes GPS and Wi-Fi geotaggingdetermine wrong location due to noise
htt // l t t
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.placecast.net
![Page 3: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/3.jpg)
3
Tag short textual annotation (free form keyword) used to
Motivation
Tag – short textual annotation (free-form keyword) used to describe photo in order to provide meaningful information about it
User provided tags may sometimes be spam annotations User-provided tags may sometimes be spam annotationsgiven on purpose or wrong tags given by mistake
User can be “an algorithm”
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/scriptingnews/2229171225 http://code.google.com/p/spamcloud
![Page 4: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/4.jpg)
4
Consider user trust information derived from users’ tagging
Goal
Consider user trust information derived from users tagging behavior for the tag propagation
Build up an automatic tag propagation system in order to:D th t ti ti d Decrease the annotation time, and
Increase the accuracy of the system
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.costadevault.com/blog/2010/03/listening-to-strangers
![Page 5: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/5.jpg)
5Geotags
Tags• summer school• June• June• 2009• sea Geotags• Hagia Sophia
IPTC h i id d
Hagia Sophia• Blue Mosque• Istanbul• Turkey• geo:lat = 41.01667
IPTC schema is considered: City – name of the city Sublocation – area or name of the landmark
E l I t b l (H i S hi ) S F i (G ld G t B id )
g• geo:lon = 28.96667
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
Examples: Istanbul (Hagia Sophia), San Francisco (Golden Gate Bridge)
![Page 6: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/6.jpg)
6Trust and reputation systems
Email spam filtering
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.wpconfig.com/2010/07/04/tips-to-increase-your-pagerank
![Page 7: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/7.jpg)
7System overview
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 8: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/8.jpg)
8
Object duplicate detection module provides a matching score
Tag propagation
Object duplicate detection module provides a matching score matrix
Two application scenarios:Cl d t bl
, 1 , 1 i , jS i ,M j ,N
Closed set problem:
11 if
0 otherwise
i , j i , ji ,Mi , j
, S max SC
Open set problem:
0 otherwise ,
11 if
0 otherwise
i , j i , j i , ji ,Mi , j
ˆ, S max S S SO
,
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 9: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/9.jpg)
9
Three steps:
User trust modeling
Three steps: User evaluation – comparing the predicted tags to manually defined
ground truth (for a real photo sharing system, such as Panoramio, it is not necessary to collect ground truth data since user feedback can y greplace them):
1 if user tags image correctly0 otherwisei , j
, j iU
, Trust model creation – the percentage of the correctly tagged
images by particular user out of the overall number of tagged images M:
Mi jU
Tag propagation – only tags from users who are trusted ( ) are propagated to other photos in the dataset
1 i , jij
UT
Mj
ˆT T
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 10: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/10.jpg)
10User trust modeling in Panoramio
Tagged incorrectly?
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.panoramio.com/photo/23286122
![Page 11: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/11.jpg)
11
Idea:
User trust modeling in Panoramio
Idea:
true flag (correct tag)
false flag (wrong tag + suggest a correct tag)
Th i l t h th t t d The more misplacements a user has, the more untrusted he/she is
User trust ratio:
all images tagged by user
true flagsall associated flagsj
j
# T#
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 12: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/12.jpg)
12
Users can collaboratively eliminate a spammer:
Eliminate spammers in Panoramio
Users can collaboratively eliminate a spammer:
Bob
Eve
Bob
Alice
EAlice
0 9 T . 0 7 T . 0 5 T .
B b
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
EveAliceBob
![Page 13: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/13.jpg)
13
Users can collaboratively eliminate a spammer:
Eliminate spammers in Panoramio
Users can collaboratively eliminate a spammer:
Bob
Eve
Bob
Alice
EAlice
0 3 T . 0 7 T . 0 9 T .
B b
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
EveAliceBob
![Page 14: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/14.jpg)
14
Users can collaboratively eliminate a spammer:
Eliminate spammers in Panoramio
Users can collaboratively eliminate a spammer:
Bob
Eve
Bob
Alice
EAlice
0 7 T . 0 8 T . 0 4 T .
B b
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
EveAliceBob
![Page 15: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/15.jpg)
15Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
![Page 16: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/16.jpg)
16Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
![Page 17: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/17.jpg)
17Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
![Page 18: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/18.jpg)
18Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Alice
Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
![Page 19: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/19.jpg)
19Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Alice
Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
![Page 20: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/20.jpg)
20Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Alice
Bob Eve
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
![Page 21: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/21.jpg)
21Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Alice
Bob Eve
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
![Page 22: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/22.jpg)
22
Dataset
Experiments
Dataset 1320 images ‒ 22 cities, 3 sublocations for each city, 20 sample
images for each sublocation
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 23: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/23.jpg)
23
Dataset
Experiments
Dataset Large variety of views, distances and partial occlusions
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 24: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/24.jpg)
24
Methodology
Experiments
Methodology Extract a sub network from a large social network, in a way that every
user in this subsystem annotates every monument in the subset of the dataset
Upon this sub network, we build up an automatic propagation system in order to decrease the annotation time and increase the accuracy of the system44 bj t k d t t 66 h t h i d 44 subjects were asked to tag 66 photos, chosen in advance
If either one of geotags (name of the area or name of the monument depicted in the image) is correlated with the object in the image, we assume that the image is correctly taggedg y gg
Evaluation Recognition rate (accuracy):
RR TA
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
A
![Page 25: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/25.jpg)
25
The recognition rate for all landmarks
Results
The recognition rate for all landmarks
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 26: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/26.jpg)
26
The recognition rate across the different locations groups
Results
The recognition rate across the different locations groups (bars) and the recognition rate of all locations (dashed line)
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 27: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/27.jpg)
27
Trust ratios for the different users for a reduced (first 20
Results
Trust ratios for the different users for a reduced (first 20 images) and the complete set of training images
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 28: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/28.jpg)
28
The recognition rate of the geotag propagation system and
Results
The recognition rate of the geotag propagation system and the percentage of the propagated tags versus the threshold involved in user trust model
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 29: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/29.jpg)
29
The recognition rate of the geotag propagation system
Results
The recognition rate of the geotag propagation system versus the number of tags
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 30: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/30.jpg)
30
An efficient system for automatic geotag propagation by
Conclusion
An efficient system for automatic geotag propagation by associating locations with distinctive landmarks and using object duplicate detection for tag propagation
User trust information derived from users’ tagging behavior User trust information derived from users tagging behavior for the tag propagation
Only reliable geotags are propagated Increased accuracy of the tag
propagation and a decrease oftagging efforts
The proposed user trust model can begeneralized to photo sharing platformssuch as Panoramio
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
![Page 31: Geotag Propagation in Social NetworksGeotag Propagation in … · User trust modeling in Panoramio true flag (correct tag) false flag (wrong tag + suggest a correct tag) Th i l t](https://reader033.vdocuments.us/reader033/viewer/2022050202/5f558c299bb3d74003598738/html5/thumbnails/31.jpg)
31
Thank you for your attention!
Ivan Ivanov, PhD student, [email protected]@epfl.ch
Multimedia Signal Processing GroupSwiss Federal Institute of Technology