![Page 1: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/1.jpg)
Francesco Pittaluga and Sanjeev J. KoppalUniversity of Florida
Electrical and Computer Engineering
Privacy Preserving Optics for Miniature Vision Sensors
![Page 2: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/2.jpg)
The next wave of small devices
Shoham et al. 07, Wood 08, Enikov et al. 09, Agrihouse 15
Microrobots
Medical devices Remote sensor nodes
![Page 3: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/3.jpg)
In the future, there will be trillions of networked miniature cameras.
![Page 4: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/4.jpg)
Privacy in the Face of Trillions of Eyes
Some groups are particularly vulnerable
![Page 5: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/5.jpg)
Our ideas
1) Pre-Capture Privacy
– Privacy before capture
2) Miniaturizing Algorithms
- High performance
- Smallest mass and volume
We show mobile scale prototypes
![Page 6: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/6.jpg)
Motivating Example
Group of People
We want to:
1) Track/Photograph everyone
2) Prevent face recognition
Quantitative: accurate people tracking and
low recognition rate
![Page 7: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/7.jpg)
Block Diagram
Group of People
Light
1. Tracking peoplewith privacy
Modulating Optics CameraLight
2. Photographing people with privacy
![Page 8: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/8.jpg)
Block Diagram
Group of People
Light
1. Tracking peoplewith privacy
Modulating Optics CameraLight
2. Photographing people with privacy
![Page 9: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/9.jpg)
Ray Diagram
Group of People
Display
Beam splitter
Displayed light
Camera
Light
![Page 10: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/10.jpg)
Ray Diagram
Group of People
Display
Beam splitter
Displayed light
Camera
Light
Displayed light+
Scene radiance
![Page 11: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/11.jpg)
Pre-Capture White-Out
Group of People
Some overexposed pixels
Beam splitter
Light
ImageCaptured
![Page 12: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/12.jpg)
Optical K-Anonymity
Group of People
Optically superimpose face
Beam splitter
Light
Sweeney 2002
![Page 13: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/13.jpg)
Optical K-anonymity
Scene Displayed Image Sensor output
![Page 14: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/14.jpg)
Image Formation Model
Weight sum of k-1 images
Image pixelScene point radiance
Pixel-radiance map Weight
Camera-displaytransform
Optical path split ratio
![Page 15: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/15.jpg)
Miniaturization (Orthographic)
![Page 16: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/16.jpg)
Miniaturization (Orthographic)
M_min (min size for res.)
Camera
Display
Beam splitter
![Page 17: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/17.jpg)
Miniaturization (Orthographic)
M_min (min size for res.)
Camera
Display
Beam splitter
lmask
lbeam
![Page 18: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/18.jpg)
Volume occupied
Camera
Display
Beam splitter
![Page 19: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/19.jpg)
Miniaturization by translation
Camera
Display
Beam splitter
![Page 20: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/20.jpg)
Miniaturization (Perspective)
Please see details in the paper
![Page 21: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/21.jpg)
Block Diagram
Group of People
Light
1. Tracking peoplewith privacy
Modulating Optics CameraLight
2. Photographing people with privacy
![Page 22: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/22.jpg)
Block Diagram
Group of People
Light
1. Tracking peoplewith privacy
Modulating Optics CameraLight
2. Photographing people with privacy
![Page 23: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/23.jpg)
Our key ideas
• Defocus in time-of-flight (TOF) and thermal domains preserve utility and provide privacy
• Multiple defocus apertures allow privacy and utility even in visible domains
Not effective in RGB for small blur Neustaedter 2006
![Page 24: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/24.jpg)
Privacy Vision Sensor 1 of 3
Group of People
Light
1. Tracking peoplewith privacy
Modulating Optics CameraLight
2. Photographing people with privacy
![Page 25: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/25.jpg)
Defocused time-of-flight camera
Conventional usage
With Defocus
![Page 26: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/26.jpg)
Defocused time-of-flight camera
Conventional usage
With Defocus
![Page 27: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/27.jpg)
Privacy Vision Sensor 2 of 3
Group of People
Light
1. Tracking peoplewith privacy
Modulating Optics CameraLight
2. Photographing people with privacy
![Page 28: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/28.jpg)
Defocused thermal camera
FLIR One with defocus lens
![Page 29: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/29.jpg)
Miniaturization of defocus
u
d
![Page 30: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/30.jpg)
Miniaturization of defocus
Camera with defocus
u
d
![Page 31: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/31.jpg)
Miniaturization of defocus
Camera with defocus
![Page 32: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/32.jpg)
Miniaturization of defocus
Camera with defocus
Angular support
Viewing direction
Givendesired support
Giventolerance
Koppal 2013
![Page 33: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/33.jpg)
Miniaturization of defocus
Given Output
u , dOur
PAMI 2013
Given
Defocus parameterand angular res.
Biggest featureto anonymize
![Page 34: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/34.jpg)
Miniaturization of defocus
Given Output
u , dOur
PAMI 2013
GivenOutput
![Page 35: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/35.jpg)
Miniaturization of defocus
Camera with defocus
![Page 36: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/36.jpg)
Miniaturization of defocus
Camera with defocus
![Page 37: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/37.jpg)
Privacy Vision Sensor 3 of 3
Group of People
Light
1. Tracking peoplewith privacy
Modulating Optics CameraLight
2. Photographing people with privacy
![Page 38: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/38.jpg)
Scale space analysis
Gaussian pyramid
Lindeberg 1998
![Page 39: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/39.jpg)
Scale space analysis with optical defocus
Optical apertures
Scene
Worry about privacy here!
![Page 40: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/40.jpg)
Scale space detection with optical defocus
A classifier is trained on the blobs
![Page 41: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/41.jpg)
Optical Array Miniaturization
![Page 42: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/42.jpg)
Optical Array Miniaturization
Optical elements withmass/volume/FOV
Physical device has size limits
![Page 43: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/43.jpg)
Optical Array Miniaturization
Optical elements withmass/volume/FOV
Physical device has size limits
![Page 44: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/44.jpg)
Optical Array Miniaturization
• Has two parts
– Selection
– Packing
We focus on selection
Korf et al. 2010
![Page 45: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/45.jpg)
Knapsack Problem
These could get addedinto the design
![Page 46: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/46.jpg)
Optical Knapsack Problem
![Page 47: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/47.jpg)
Optical Knapsack Problem
No magic: Pseudo-polynomial approximation
Angulardiscretization
![Page 48: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/48.jpg)
Summary
K-anonymity Camera Defocus in thermaland TOF preserves privacy
Optical scale space analysis Optical knapsack solution
![Page 49: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/49.jpg)
Acquired Image
Future Work: Privacy in Image Formation
Geometry
Lighting
Materials
Camera
Image
Formation
![Page 50: Privacy Preserving Optics for Miniature Vision Sensors](https://reader030.vdocuments.us/reader030/viewer/2022012718/61b1107af086dd73050f5db5/html5/thumbnails/50.jpg)
Demo at CVPR 2015
Sensor Setup Pre-Capture White-Out