virtual contact sheets - carnegie mellon school of ..../worichte/talks/pdl... · virtual contact...
TRANSCRIPT
![Page 1: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/1.jpg)
Virtual Contact Sheets Wolfgang Richter
Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes, Lin Zhong*, Mahadev Satyanarayanan
PARALLEL DATA LABORATORY Carnegie Mellon University
*Rice University
![Page 2: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/2.jpg)
Crowdsourcing the News [1] “When every cell phone can record
video and take pictures, everyone is a potential news source.”
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 2
Who killed Ian Tomlinson?
![Page 3: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/3.jpg)
CNN: Caught Red Handed [2]
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 3
I’m a sneaky thief.
![Page 4: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/4.jpg)
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 4
Problem
Cloud
Recent, not uploaded
Older, uploaded
![Page 5: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/5.jpg)
Fundamental Mobile Challenges • Real-time sync not possible:
• Energy • Infinite battery life desired
• Bandwidth • Increasingly metered and charged
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 5
How far can we degrade the objects in a search before it loses meaning?
![Page 6: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/6.jpg)
Just-in-Time Search 1. Check virtual contact sheet
• Collection of fidelity-reduced images in the cloud
2. Pull full fidelity image on hit • Search full fidelity; present full fidelity results
to searcher
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 6
![Page 7: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/7.jpg)
Cloud
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 7
Near Real-Time Sync (1) Search hit (2) Retrieve
![Page 8: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/8.jpg)
Contact Sheet Analogy
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 8
![Page 9: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/9.jpg)
Evaluating Feasibility [3]
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 9
Face Detection, RGB Histogram, Brightness Filter
Benchmark
Using Varying Resolution and JPEG Quality Factor
![Page 10: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/10.jpg)
Energy Savings
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 10
0
1000
2000
3000
4000
5000
6000
7000
Full 640x480 320x240 160x120 80x60 40x30 20x20
En
erg
y C
on
sum
ptio
n (
mJ)
Fidelity
Energy
10 images = 5 seconds call time (Droid)
![Page 11: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/11.jpg)
Bandwidth Savings
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 11
0
50
100
150
200
250
300
350
400
Full 640x480 320x240 160x120 80x60 40x30 20x20
Bandw
idth
(K
B)
Fidelity
Bandwidth
Orders of magnitude savings in bandwidth.
![Page 12: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/12.jpg)
What about Search Accuracy?
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 12
0
0.2
0.4
0.6
0.8
1
640x480 320x240 160x120 80x60
Filt
er
Acc
ura
cy
Fidelity
0.650
0.424
10095755025
No one-size-fits-all sweet spot.
Face Detection
![Page 13: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/13.jpg)
Conclusion & Future Work • Searches retain meaning
• No one-size-fits-all, but • Orders of magnitude bandwidth savings • Significant energy savings
• More search primitives? • Perceptual hashing • Face recognition • Texture Filters
• Better fidelity reduction? • Keep faces at high fidelity
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 13
![Page 14: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/14.jpg)
Further Reading [1] TED. http://www.ted.com/talks/paul_lewis_crowdsourcing_the_news.html.
[2] CNN. http://www.cnn.com/2010/CRIME/08/24/new.jersey.theft.photo/index.html?hpt=C1.
[3] SimpleCV. http://simplecv.org/.
Jan Harkes, Mahadev Satyanarayanan, Ardalan Amiri Sani, and Benjamin Gilbert. A Contact Sheet Approach to Searching Untagged Images on Smartphones. CMU Technical Report CMU-CS-11-132.
Ardalan Amiri Sani, Wolfgang Richter, Xuan Bao, Trevor Narayan, Mahadev Satyanarayanan, Lin Zhong, and Romit Roy Choudhury. Opportunistic Content Search of Smartphones. Rice University Technical Report TR0627-2011.
Xuan Bao, Trevor Narayan, Ardalan Amiri Sani, Wolfgang Richter, Romit Roy Choudhury, Lin Zhong, and Mahadev Satyanarayanan. The Case for Context-Aware Compression. HotMobile 2011.
Jason Flinn and Mahadev Satyanarayanan. Energy-aware Adaptation for Mobile Applications. SOSP 1999.
Y. Nimmagadda, K. Kumar, and Y. H. Lu. Energy-efficient image compression in mobile devices for wireless transmission. ICME 2009.
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 14
![Page 15: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/15.jpg)
Full Fidelity
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 15 12,000+ Flickr dataset
Average photo size: 374 KB
![Page 16: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/16.jpg)
Reduce Resolution
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 16 12,000+ Flickr dataset
Average photo size: 221 KB
![Page 17: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/17.jpg)
JPEG Quality Factor
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 17 12,000+ Flickr dataset
Average photo size: 6 KB
![Page 18: Virtual Contact Sheets - Carnegie Mellon School of ..../worichte/talks/pdl... · Virtual Contact Sheets Wolfgang Richter Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes,](https://reader034.vdocuments.us/reader034/viewer/2022042806/5f6c98f6110444796e051b04/html5/thumbnails/18.jpg)
Bandwidth Savings Table
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 18
Resolution 100 95 75 50 25 5
640x480 221 KB 114 KB 47 KB 31 KB 19 KB 6 KB
320x240 61 KB 33 KB 14 KB 10 KB 6 KB 2 KB
160x120 17 KB 10 KB 5 KB 3 KB 2 KB 826 B
80x60 5 KB 3 KB 2 KB 1 KB 892 B 460 B
40x30 2 KB 1 KB 725 B 588 B 484 B 345 B
20x20 1000 B 727 B 501 B 436 B 385 B 316 B