privacy protection of multimedia information - vis...
TRANSCRIPT
![Page 1: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/1.jpg)
1
Privacy Protection of Multimedia
Information
Sen-ching Samson Cheung 張先正
張先正
張先正
張先正
Center for Visualization & Virtual Environments
Department of Electrical & Computer Engineering
University of Kentucky
http://www.vis.uky.edu/mialab
![Page 2: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/2.jpg)
2
�Smart video
surveillance
�Biometric signals
�Mobile-m
edia
processing
�RFID tracking
Multimedia privacy concerns
![Page 3: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/3.jpg)
3
Challenges from Multimedia
�What to protect?
�Identify selective semantic objects for
protection
�How to protect it?
�Reliable protection without sacrificing
perceptual utility, processing speed and
bandwidth
�How to control it?
�Flexible control and secure authentication of
privacy data
![Page 4: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/4.jpg)
4
Privacy Protected Video Surveillance
Ob
ject
S
egm
enta
tio
nan
d T
rack
ing
Ob
fusc
atio
n
Pri
vacy
Dat
a P
rese
rvat
ion
Su
rvei
llan
ceV
ideo
Dat
abas
e
Su
bje
ct
Iden
tifi
cati
on
Mo
du
le
Sec
ure
Cam
era
Sys
tem
Pri
vacy
Dat
a M
anag
emen
t S
yste
m
![Page 5: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/5.jpg)
5
Summary of contributions
Tasks
Our contributions
CameraPlacement
Optimal visual sensor design for localizing
subjects
Object Segmentation
and Tracking
Visible-light and therm
al camera fusion for
better background modeling
Obfuscation
Object-basedVideo In-painting
Privacy Data
Preservation
Rate-distortion optimal data hiding
Anonymous Subject
Identification
Homomorphicencryption based biometric
access control
Applications
VIBE: Video Interface
Behavioral Evaluation
![Page 6: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/6.jpg)
6
Visual Tagging for Subject Identification
Visual Tagging: use of visual features to locate objects
�Pure visual, no special hardware
�Do not need subject cooperation
�Self and other occlusions
![Page 7: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/7.jpg)
7
Multi-camera Localization
Use epipolar lines
from different
cameras to localize
occluded objects
![Page 8: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/8.jpg)
8
Camera Network Planning
�Multi-camera localization needs each
object to be visible by at least two
cameras
�For a surveillance environment:
�How many cameras?
�Where should we put those cameras?
�What is the expected performance?
�Need a proper model to capture all known
and unknown parameters
![Page 9: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/9.jpg)
9
Optimal Sensor Planning
�Art Gallery or Illumination Problems
�K-guarding problem -at most n-1 cameras for any
planar n-sided polygon with no holes [Belleville et al. 94]
�Optimal solution: NP-complete problem !?
�Continuous-domain approach
�Hill-clim
bing [Bodor et al 07], simulated annealing [Mittal,
Davis 08], evolutionary approach [Dunn, Olague, Lutton 06]
�Restrictive modeling, computational intensive, local
minima
�Proposed Discrete-domain approach
�Integer-programming
�Resource constraint problem
![Page 10: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/10.jpg)
10
Statistical Visibility Model
Random Parameters, P: model F(P)
�position of a tag
�orientation of a tag
�mutual occlusion
Fixed Parameters, K: user-specified
�room topology
�cameras’ intrinsic parameters
�dimensions (lengths) of a tag
Design Parameters, C: controllable
�number of cameras
�position of each camera
�orientation of each camera
General framework:
�Computable Visibility
function I
(P|K
,C) for a tag
at P
�Performance of
C:
�Optimization:
∫)
()
,|
(P
dFC
KP
I
∫)
()
,|
(m
axP
dFC
KP
IC
![Page 11: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/11.jpg)
11
Visibility Function I(P|K,C)
�Environment K
�Camera C=(x
C,yC,φ,ρ)
�Tag P =(x
P,yP,zP,θ,λ)
�I(P|K,C) = 1 if the tag image is large enough
θ
(xP,yP,zP)
(xC,yC)
(φ,ρ)
λ
![Page 12: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/12.jpg)
12
Variable Definitions
�Discretize both tag and camera space into
lattice points
�Output Variable: camera placement
�Measurement Variable: Tag visibility
xTag at Pivisible at 2 or more cameras
Otherwise
Camera present at Ci
Otherwise
![Page 13: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/13.jpg)
13
A camera cannot see two direction at the same time
Binary Integer Program
�Cost function
�Constraints:
1.
2.
3.
ibm
axExpected volume visible by 2 or more cameras
At each tag grid P
i, define xibased on the visibility function
Camera constraint
Linear cost & constraints
-Solved using lpsolve, c-plex
-NP-hard (greedy search)
∑=P
N jj
jx1ρ
()
∑=
<+
−C
N ij
ci
ji
xN
CK
PI
b1
11
),
|(
∑=
≥−
CN i
ji
ji
xC
KP
Ib
10
2)
,|
(
∑=
≤C
N ii
mb
1
1y)
(x,
at
All
≤∑ ib
ib
![Page 14: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/14.jpg)
14
Simulated Perform
ance
Comparison with other schemes:
Use of traffic modeling:
![Page 15: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/15.jpg)
15
Experimental Results
Optimal
Visibility = 0.53
Uniform
Visibility = 0.38
Zhao, J., S.-C. Cheung and T. Nguyen. 2008. Optimal Camera
Network Configurations for Visual Tagging. In IEEE Journal
on Selected Topics in Signal Processing, Volume 2,
Number 4, August 2008, pp. 464-479.
![Page 16: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/16.jpg)
16
Summary of contributions
Tasks
Our contributions
CameraPlacement
Optimal visual sensor design for localizing
subjects
Object Segmentation
and Tracking
Visible-light and therm
al camera fusion for
better background modeling
Obfuscation
Object-basedVideo In-painting
Privacy Data
Preservation
Rate-distortion optimal data hiding
Anonymous Subject
Identification
Homomorphicencryption based biometric
access control
Applications
VIBE: Video Interface
Behavioral Evaluation
![Page 17: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/17.jpg)
Background Subtraction
�Shadows and highlights
�Illumination changes
�Non-static background
�Color similarity
![Page 18: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/18.jpg)
Fusion of thermal and visible-light
�Therm
al Im
aging (PV320 digital therm
al camera)
�Uncooled focal plane array of ferroelectric
sensors (-20 –500oC)
�Challenges
�Registration
�Data Fusion
�Existing Approaches:
�Optical Fusion [Volfson06], [Wu08]
�Im
age W
arping [Davis05], [Kumar06][Han07]
![Page 19: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/19.jpg)
Cameras Registration and Blob alignment
1.Calibration to obtain fundamental and rectification
matrices
2.Estimate a homography for each foreground blob based
on disparity -assume each person is of constant depth
(a) Infrared (b) visible light
![Page 20: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/20.jpg)
Blob Extraction and Data Fusion
�Individual tracker
tracks object in each
camera view
�Combined tracker
estimates
homographies
�Second tier adjusts
parameters and
updates the states
using fused data
Zhao, J. and S.-C. Cheung. 2009. Human Segmentation by
Fusing Visible-light and Therm
al Im
aginary. Submitted to the
Ninth IEEE International Workshop on Visual
Surveillance (VS2009).
![Page 21: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/21.jpg)
Results versus image warping
![Page 22: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/22.jpg)
Fusion Results
![Page 23: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/23.jpg)
Video Results
23
![Page 24: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/24.jpg)
24
Summary of contributions
Tasks
Our contributions
CameraPlacement
Optimal visual sensor design for localizing
subjects
Object Segmentation
and Tracking
Visible-light and therm
al camera fusion for
better background modeling
Obfuscation
Object-basedVideo In-painting
Privacy Data
Preservation
Rate-distortion optimal data hiding
Anonymous Subject
Identification
Homomorphicencryption based biometric
access control
Applications
VIBE: Video Interface
Behavioral Evaluation
![Page 25: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/25.jpg)
25
Video Obfuscation
Original
Pixelation/
Blurring
Black
Out
In-painted
![Page 26: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/26.jpg)
26
Challenges of Video Inpainting
![Page 27: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/27.jpg)
27
Dynamic Object In-painting
�Basic idea: Using object template extracted form
other time instant to complete a conceptually
consistent sequence.
�Steps:
1. Similarity based on optimal alignment
2. Motion continuity
3. Positioning of templates
?
![Page 28: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/28.jpg)
28
Motion Continuity
??
![Page 29: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/29.jpg)
29
Object-based Video In-painting
�Better motion in-painting by better registration and task
separation
�Capable to in-paint partially and completely occluded
objects
�Im
proved computational performance (Matlab)
Num
ber
of fr
ames
with
com
plet
e oc
clus
ion
Num
ber
of fr
ames
with
par
tial o
cclu
sion
![Page 30: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/30.jpg)
30
Public-domain Sequences
![Page 31: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/31.jpg)
31
Multi-people sequence
![Page 32: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/32.jpg)
32
Complex Sequences
![Page 33: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/33.jpg)
33
Summary of contributions
Tasks
Our contributions
CameraPlacement
Optimal visual sensor design for localizing
subjects
Object Segmentation
and Tracking
Visible-light and therm
al camera fusion for
better background modeling
Obfuscation
Object-basedVideo In-painting
Privacy Data
Preservation
Rate-distortion optimal data hiding
Anonymous Subject
Identification
Homomorphicencryption based biometric
access control
Applications
VIBE: Video Interface
Behavioral Evaluation
![Page 34: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/34.jpg)
34
Keeping sensitive inform
ation
Medium
Method
Pro
Con
Separate
File
Encryption +
Cryptographic
signature
�Standard Technology
�Storage efficiency
�Pervious to attacker
�Difficult to distribute with
the modified video
�Separate authentication for
modified video
Meta-data
Encryption +
Cryptographic
signature
�Standard Technology
�Storage efficiency
�Less pervious to attacker
�Depend on format
Data
hiding
Encrypted
watermark
�Im
pervious to attacker
�Inseparable from data
�Joint authentication
�May need more storage
�May affect visual quality
![Page 35: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/35.jpg)
35
Privacy Data Preservation
![Page 36: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/36.jpg)
36
Data Hiding
�Data hiding/Stenography/W
atermarking
�Active research in the past fifteen years
�Typical applications include authentication, copy
detection, monitoring
�Challenges in our application:
�Picture-in-picture: large embedding capacity
�Compatibility with existing compression scheme
�Minimal visual distortion
![Page 37: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/37.jpg)
37
Optimal Data Hiding
Psy
cho-
visu
alM
odel
ing
Blo
ck-b
ased
Rat
e-D
isto
rtio
nC
alcu
latio
n
Dis
cret
eO
ptim
izat
ion
Sol
ve c
onst
rain
ed o
ptim
izat
ion
020
4060
8010
012
014
016
018
020
00
0.2
0.4
0.6
0.81
1.2
1.4
Rat
e
Distortion
020
4060
8010
012
014
016
018
020
00
0.2
0.4
0.6
0.81
1.2
1.4
Rat
e
Distortion
Com
bine
d ra
te-
dist
ortio
n co
st C
(x) #
embe
dded
bits
![Page 38: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/38.jpg)
38
R-D framework
�Target cost function:
�Ri= Increase in Bandwidth of Block i
�Di= Perceptual Distortion in Block i
�δ= Relative Weight
�Greedy embedding of P data bits in Block i:
�Lagrangian optimization: determine the optimal Piand λto
embed the target number of data bits:
![Page 39: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/39.jpg)
39
Proposed Data Hiding
Mot
ion
Com
pens
atio
nD
CT
Ent
ropy
Cod
ing
•D
CT
Dom
ain
•F
requ
ency
, co
ntra
st a
nd
lum
inan
ce m
aski
ng [W
atso
n]
H.2
63H
.263
Enc
rypt
ed f
oreg
roun
dvi
deo
bit-
stre
am
DC
TP
erce
ptua
lM
ask
Par
ityE
mbe
ddin
g
R-D
O
ptim
izat
ion
Pos
ition
s of
th
e “o
ptim
al’
DC
T c
oeff
for
embe
ddin
g
DC
T(i,
j) =
wat
erm
ark_
bit+
2*ro
und(
DC
T(i,
j)/2)
Priv
acy
prot
ecte
dvi
deo
Last
dec
oded
fram
e
J. Paruchuri & S.-C. Cheung “Rate-
Distortion Optimized Data Hiding
for Privacy Protection” submitted to
ISCAS 2008
![Page 40: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/40.jpg)
Block embedding strategies compared
40
![Page 41: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/41.jpg)
Bit Allocation strategies compared
41
![Page 42: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/42.jpg)
Overall results
42
![Page 43: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/43.jpg)
43
Examples 1/2
119kbps
No data
Distortion
637 kbps
81 kbps data
Rate &
Distortion
562 kbps
81 kbps data
Rate only
370 kbps
81 kbps data
![Page 44: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/44.jpg)
44
Examples 2/2
406.3kbps
No data
Distortion
743 kbps
81 kbps data
Rate &
Distortion
678 kbps
81 kbps data
Rate only
610 kbps
81 kbps data
![Page 45: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/45.jpg)
45
Summary of contributions
Tasks
Our contributions
CameraPlacement
Optimal visual sensor design for localizing
subjects
Object Segmentation
and Tracking
Visible-light and therm
al camera fusion for
better background modeling
Obfuscation
Object-basedVideo In-painting
Privacy Data
Preservation
Rate-distortion optimal data hiding
Anonymous Subject
Identification
Homomorphicencryption based biometric
access control
Applications
VIBE: Video Interface
Behavioral Evaluation
![Page 46: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/46.jpg)
46
�What you have:
�RFID [Wickramasuriya04], Hard hat
[Schiff07], Colored Marker [Zhou09]
�Tags are vulnerable or easy to
forge
Subject Identification
�Who you are:
�Biometric like fingerprint, iris, face
and gait
�Worse privacy as system can
associate video to identity
How do you perform
biometric
verification anonymously?
How do you perform
biometric
verification anonymously?
![Page 47: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/47.jpg)
47
�Anonymous Biometric Access Control [Luo09] [Yi09]
�A secure multi-party protocol that guarantees:
�Bob knows if ∃y∈DBwhere y matches q but does
not know which y.
�Alice knows the result but knows nothing about DB
ABAC
ABAC
Alice and her biometric q
Bob & his DB
![Page 48: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/48.jpg)
ABAC Protocol
48
Query
Encryption
Distance
Computation
Distance Bit
Extraction
Threshold
Comparison
Multiplication
(Accumulation)
Enc p
k(y+r)
with rand r
Decryption
Hash(y+r)
Hash(r)
Equal?
Enc p
k(y) where y=0 (success)
or function of matching
Alice can’t
cheat!
![Page 49: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/49.jpg)
Using ABAC in Surveillance
49
![Page 50: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/50.jpg)
50
Problem: ABAC is not scalable
�Use Paillier homomorphic encryption and
interactive protocols to implement
similarity search [Luo09], [Yi09]
�Match one 9600-bit iris code [Masek03]:
�Initialization (one-time)
290 ms
�Hamming distance
98 ms
�Bit extraction & compare
4120 ms
�For a DB with 10,000 iris codes:
�~11.5 hours and 120MB data exchanged
![Page 51: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/51.jpg)
�Tradeoff complexity with privacy by providing
extra information to Bob (server):
�Idea behind kAQ:
�Alice tells Bob the cell C to which her query belongs
�Bob runs ABAC within C only ⇒
achieves k-anonymity
�Design of the cell or quantization structure:
1.
Must minimize leakage of Alice’s privacy
2.
Must ensure the correctness of similarity search
k-Anonymous Quantization
51
Bob knows a subset (cell) C that contains at least k
xi∈DBsuch that all q with d(q,xi)<εare in C.
![Page 52: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/52.jpg)
Minimize privacy leakage
�Which structure leaks less information?
�Hypothesis: Quantization 1 as the entries within
are “maximally dissimilar”
52
Biometric signals in gallery
Four cells
Quantization 1
Quantization 2
Probe q
![Page 53: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/53.jpg)
�Demonstrated in fingerprints [Jain02] and palmprints
[Kong06]
X := distance between iris-codes from twins
Y := distance between iris-codes from non-twins
Null hypothesis
H0:
µ X= µ
Y
Alternative hypothesis
H1:
µ X< µ
Y
�Data: 1118 iris from 100 pairs of twins [CASIA05]
�Distribution-free W
ilcoxonRank-Sum Test produces
a one-side P-value of 6.17 x 10-75
⇒Strongly favors the adoption of the alternative
hypothesis
Kinship ⇒
Similar biometrics ?
53
![Page 54: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/54.jpg)
Given a kAQ Γ, its privacy-protecting capability
can be measured by this utility function:
To design a good kAQ:
�Maximizing utility is not enough!
�To ensure the correctness of kAQ, we need to
ensure that if y matches x in C, y itself must also
be in C ⇒
Neighborhood Structure
Utility of kAQ
54
2
,)
,(
min∑
∩∈
Γ∈
CD
Bx
xj
ij
iC
xx
d
![Page 55: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/55.jpg)
�Typical choice of Neighborhoods:
�Bounding Balls or Boxes
�Good Neighborhood:
�Should capture the variability of all biometrics
�Should minimize the overlap of neighborhoods
between different individuals
Neighborhoods
55
Mary’s Mom’s
Mary’s
C1
C2
![Page 56: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/56.jpg)
�Data-driven design –based on capturing
many biometrics from each person
�1-2 patterns for testing, rest for training –
correct recognition depends on generalization
�Neighborhood candidates:
Different Neighborhoods
56
εε
ε
Actual
Bounding
Box
Actual
bounding
Ball
Maximum
radius
Average+1stdev
radius
![Page 57: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/57.jpg)
Greedy kAQ algorithm
1.
Embed hamming space into low-dimensional
Euclidean space
2.
Uniform quantization to form bins
3.
# of cells, N = floor(|DB|/k)
4.
Randomly assign a neighborhood to each cell
5.
Select the neighborhood-cell pair that maximize the
gain in utility
6.
Repeat step 5 until all neighborhoods are exhausted
7.
If a cell has less than k neighborhoods, N := N-1 and
back to step 4.
Note: overlapping neighborhoods in different cells will
cause multiple cells to be used in later stage.
57
![Page 58: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/58.jpg)
Results
For a database 10,000 entries
�full encrypted processing takes ~ 12 hours to run.
�k-anonymous quantization with k=50 takes 650 seconds
58
privacy
S. Yee, Y. Lou, J. Zhao and S.-C. Cheung. 2009. Anonymous Biometric Access Control. Accepted to
EURASIP Journal on Information Security
![Page 59: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/59.jpg)
59
The Experiment Result of Different
Neighborhood Structures
ε-ball with a statistical radius is the best choice.
![Page 60: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/60.jpg)
60
Summary of contributions
Tasks
Our contributions
CameraPlacement
Optimal visual sensor design for localizing
subjects
Object Segmentation
and Tracking
Visible-light and therm
al camera fusion for
better background modeling
Obfuscation
Object-basedVideo In-painting
Privacy Data
Preservation
Rate-distortion optimal data hiding
Anonymous Subject
Identification
Homomorphicencryption based biometric
access control
Applications
VIBE: Video Interface
Behavioral Evaluation
![Page 61: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/61.jpg)
Tantrums: disruptive behaviors in children
�80% of children ages 1 to 4, with 20% of 2-year olds and
10% of 4-year-olds have daily tantrums.
�Prolonged, frequent, and age-inappropriate tantrums
�may indicate underlying mental illness
�may predict later antisocial behavior
�are at a higher risk of abuse
�affect proper functioning of families and schools
�demoralize caregivers as a reflection of poor parenting
61
![Page 62: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/62.jpg)
Existing Approaches
�Need careful assessments on
�Events that trigger and ameliorate tantrums
�Exact tantrum behaviors
�Consequence
�Clinicians and behavior therapists rely on
�Real-time observation during clinic’s visits
�Caregivers’ account of Events
�Limitations
�Many children do not engage in disruptive behaviors in the
clinician’s office due to absence of social triggers or
expectation of acceptable behaviors in the clinical setting.
�Caregivers’ account is often biased, incomplete and
selective
62
![Page 63: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/63.jpg)
VIBE: Video Interface Behavioral Evaluation
�What is it?
�A networked video monitoring system to
catalogue children’s disruptive behavior
�Advantages
�Unobtrusive recording of behaviors and social
interaction in the child’s natural environments
(school, home or car)
�Privacy enhancement technologies to filter
sensitive contents
�Event Recognition technologies to allow clinicians
in rapidly identify relevant episodes
63
![Page 64: Privacy Protection of Multimedia Information - Vis …cheung/courses/ee639_fa10/lecture/privacy.pdfCalibration to obtain fundamental and rectification matrices 2. ... copy detection,](https://reader031.vdocuments.us/reader031/viewer/2022022601/5b47ac5b7f8b9a824f8c1018/html5/thumbnails/64.jpg)
Current Status
�Two-year study in Lexington and Beijing to
�Aim 1: To examine attitudes toward VIBE among caregivers of
children with disruptive behaviors in different cultures
�Aim 2: To identify factors associated with negative attitudes
toward VIBE and determ
ine whether these constitute a barrier
�Aim 3: To evaluate and compare the clinician-reported value
about children’s disruptive behaviors collected using retrospective
written accounts, caregiver-recorded video and VIBE.
�Partners
�Dr. Neelkamal Soares, Pediatrics, University of Kentucky
�Dr. Brea Perry, Sociology, University of Kentucky
�Dr. Xiaoyi Yu, Peking University
64