socialsensor at mediaeval placing task 2014
TRANSCRIPT
![Page 1: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/1.jpg)
SocialSensor at MediaEval Placing Task 2014
Giorgos Kordopatis-Zilos, Giorgos Orfanidis,
Symeon Papadopoulos and Yiannis Kompatsiaris
Information Technologies Institute, CERTH, Thessaloniki, Greece
![Page 2: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/2.jpg)
Summary of our participation
• Tag-based location estimation (3 runs)– Using the language model-based scheme of (A. Popescu, MediaEval ‘13)
as basis;– Extending it with the use of the Similarity Search method, an internal grid
technique and a Gaussian distribution model based on the spatial entropy of tags.
• Visual-based location estimation (2 runs)– Extract SURF+VLAD and CS-LBP+VLAD features.– Training of linear SVM for clustering the samples.
2
![Page 3: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/3.jpg)
Tag-based location estimation
3
Baseline Approach
Pre-Processing & Filtering
Grid Of Cells & Language Model
Assignment in Cells
![Page 4: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/4.jpg)
Baseline Approach (1/3)
4
Baseline Approach
Pre-Processing & Filtering
Grid Of Cells & Language Model
Assignment in Cells
• Remove all punctuation and symbols, e.g. “.%!&”
• Transform all characters to lower case
• Remove images with empty tags and title
![Page 5: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/5.jpg)
Baseline Approach (2/3)
5
Baseline Approach
Pre-Processing & Filtering
Grid Of Cells & Language Model
Assignment in Cells
![Page 6: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/6.jpg)
Baseline Approach (3/3)
6
Baseline Approach
Pre-Processing & Filtering
Grid Of Cells & Language Model
Assignment in Cells
![Page 7: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/7.jpg)
Extensions
7
Baseline Approach
Pre-Procession & Filtering
Grid Of Cells & Language Model
Assignment in Cells
Extensions
![Page 8: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/8.jpg)
8
Extensions(1/3) - Similarity Search
![Page 9: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/9.jpg)
9
Extensions(2/3) – Internal Grid
![Page 10: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/10.jpg)
10
Extensions(3/3) – Spatial Entropy
![Page 11: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/11.jpg)
11
Visual-based location estimation Model building• Extract two features, SURF+VLAD and CS-LBP+VLAD• Training of linear SVM in a predefined number of spatial clusters and subclusters
– 50 clusters and 50 subclusters corresponding to each cluster
Location Estimation• Decision of cluster membership:
– Estimation scores provided by the cluster weight vectors – Scores corresponding to the best subcluster weight score in
each cluster.•Variants for the final estimation:
– Assign each cluster with the coordinates of the most possible subcluster
– Similarity Search with 1000 samples of the selected subcluster
![Page 12: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/12.jpg)
Runs and Results
12
• Tag-based runs: Run 1, Run 4 and Run 5• Visual runs: Run 2 and Run 3• The full test set used for all runs except Run 3• For Run 3 a subset of 25,500 images was used
![Page 13: SocialSensor at MediaEval Placing Task 2014](https://reader035.vdocuments.us/reader035/viewer/2022070514/588145671a28abf65a8b6f71/html5/thumbnails/13.jpg)
The end
This work was supported by the SocialSensor FP7 project
13