![Page 1: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/1.jpg)
Synchronizing Multi-User Photo Galleries with MRF
Emanuele Sansone, Giulia Boato Minh-Son Dao DISI, University of Trento UIT-HCM Trento, Italy HCMC,
Viet Nam [email protected], [email protected] [email protected]
![Page 2: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/2.jpg)
Synchronization and Clustering
Multiple Galleries
Feature Extraction
Offset Estimation
Synchronized Galleries
![Page 3: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/3.jpg)
Stereo Matching I
Stereo camera
Acquired Images
![Page 4: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/4.jpg)
Stereo Matching II
![Page 5: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/5.jpg)
Observed node = pixel intensity
Latent node = possible disparity for a given node
Potentials associated to each link in the network
Inference algorithm
Disparity
Stereo Matching: Model
![Page 6: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/6.jpg)
• Pixels Photos• Pixel Intensitities Image features (Timestamp, GPS, SURF, color
histograms)• States: possible diparities States: possible offsets
Stereo Matching vs. Photo Galleries Synchronization
![Page 7: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/7.jpg)
Synchronization: States
![Page 8: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/8.jpg)
Synchronization: Potentials
![Page 9: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/9.jpg)
Synchronization: Potentials
![Page 10: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/10.jpg)
Synchronization: Potentials
![Page 11: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/11.jpg)
Synchronization: Potentials
![Page 12: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/12.jpg)
Synchronization: Potentials
![Page 13: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/13.jpg)
Synchronization: Potentials
![Page 14: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/14.jpg)
Synchronization: Final Model
![Page 15: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/15.jpg)
Synchronization: Inference
![Page 16: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/16.jpg)
Clustering
• K-means algorithm• 3 runs
![Page 17: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/17.jpg)
Clustering
• K-means algorithm• 3 runs
Color Structure Descriptors(CSD - 64 values) + Local Binary Patterns 3x3 (LBP - 18 values each block)
![Page 18: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/18.jpg)
Clustering
• K-means algorithm• 3 runs
Color Structure Descriptors(CSD - 64 values) + Local Binary Patterns 3x3 (LBP - 18 values each block)
PCA Color Structure Descriptors (6 values)
![Page 19: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/19.jpg)
Clustering
• K-means algorithm• 3 runs
Color Structure Descriptors(CSD - 64 values) + Local Binary Patterns 3x3 (LBP - 18 values each block)
PCA Color Structure Descriptors (6 values)
PCA Color Structure Descriptors (6 values) + TIME
![Page 20: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/20.jpg)
Results: Synchronization
Dataset Precision Accuracy
Vancouver 0.35 0.86
London 0.25 0.89
![Page 21: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/21.jpg)
Results: Synchronization
Dataset Precision Accuracy
Vancouver 0.35 0.86
London 0.25 0.89
• Good performance in terms of accuracy
![Page 22: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/22.jpg)
Results: Synchronization
Dataset Precision Accuracy
Vancouver 0.35 0.86
London 0.25 0.89
• Good performance in terms of accuracy• Only 1/4 galleries correctly synchronized
![Page 23: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/23.jpg)
Results: ClusteringDataset Run Features Jaccard Index
Vancouver
1 CSD-64+LBP3x3 0.1673
2 CSD-6 0.1382
3 CSD-6+TIME 0.1315
London
1 CSD-64+LBP3x3 0.1287
2 CSD-6 0.0742
3 CSD-6+TIME 0.0885
![Page 24: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/24.jpg)
Results: ClusteringDataset Run Features Jaccard Index
Vancouver
1 CSD-64+LBP3x3 0.1673
2 CSD-6 0.1382
3 CSD-6+TIME 0.1315
London
1 CSD-64+LBP3x3 0.1287
2 CSD-6 0.0742
3 CSD-6+TIME 0.0885
• Performance are decreased by using only color descriptors
![Page 25: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/25.jpg)
Results: ClusteringDataset Run Features Jaccard Index
Vancouver
1 CSD-64+LBP3x3 0.1673
2 CSD-6 0.1382
3 CSD-6+TIME 0.1315
London
1 CSD-64+LBP3x3 0.1287
2 CSD-6 0.0742
3 CSD-6+TIME 0.0885
• Performance are decreased by using only color descriptors• In the Vancouver dataset time doesn’t help in clustering
![Page 26: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/26.jpg)
Results: ClusteringDataset Run Features Jaccard Index
Vancouver
1 CSD-64+LBP3x3 0.1673
2 CSD-6 0.1382
3 CSD-6+TIME 0.1315
London
1 CSD-64+LBP3x3 0.1287
2 CSD-6 0.0742
3 CSD-6+TIME 0.0885
• Performance are decreased by using only color descriptors• In the Vancouver dataset time doesn’t help in clustering• In the London dataset time becomes a more reliable feature
![Page 27: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/27.jpg)
Lessons Learned
Synchronization:Only 1/4 galleries correctly synchronized
![Page 28: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/28.jpg)
Lessons Learned
Synchronization:Only 1/4 galleries correctly synchronized• new features (e.g. text tags, social network data)
![Page 29: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/29.jpg)
Lessons Learned
Synchronization:Only 1/4 galleries correctly synchronized• new features (e.g. text tags, social network data)• “no association” between a pair of images
![Page 30: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/30.jpg)
Lessons Learned
Synchronization:Only 1/4 galleries correctly synchronized• new features (e.g. text tags, social network data)• “no association” between a pair of images
Clustering:Time is important!
![Page 31: Synchronizing Multi-User Photo Galleries with MRF](https://reader033.vdocuments.us/reader033/viewer/2022052401/55c13cacbb61eb7f1f8b4576/html5/thumbnails/31.jpg)
Thank you