1

Privacy Protected Video Surveillance

Sen-ching Samson Cheung

Center for Visualization & Virtual EnvironmentsDepartment of Electrical & Computer Engineering University of Kentucky

http://www.vis.uky.edu/mialab

2

Acknowledgements Graduate Students

Vijay Venkatesh Mahalingam Jian Zhao Jithendra K. Paruchuri

Research Support Department of Homeland Security Oak Rridge Associated Universities

3

What is privacy?

To develop human excellence without interference [Aristotle’s Politics 350 B.C.]

Control over information about oneself [Warren and Brandeis 1890]

…the right most valued by all civilized men — the right to be let alone.

- U.S. Supreme Court Justice Louis Brandeis, 1890

4

Today’s privacy concerns Electronic Voting E-commerce Medical Records Financial Records Cyber Activities

5

Smart video surveillance

Biometric theft Multimedia

processing RFID tracking

Tomorrow’s privacy concerns

6

Privacy Protection Technology

Technologies that aim at protecting personal privacy without compromising the “legitimate” use of information.

Main PPT include the followings: Encryption Tools Platform for Privacy Preferences (P3P) Automated Privacy Audit Anonymizer Privacy Protected Data mining

Is privacy protection of multimedia any different?

7

Challenges from Multimedia

What to protect? Identify semantic objects for protection

How to protect it? Reliable protection without losing

perceptual utility How to control it?

Flexible control and secure authentication of privacy data

8

Talk Overview Video Surveillance

Subject Identification Optimal Camera Network Video Obfuscation Privacy Data Management

Portable AV devices Evaluation of audio privacy protections

Secure Distributed Processing

9

Overview

Object Identification and Tacking

Obfuscation

Secure Data Hiding

SurveillanceVideo

Database

Subject IdentificationModule

SecureCameraSystem

Privacy Data Management System

10

Talk Overview Video Surveillance

Subject Identification Optimal Camera Network Video Obfuscation Privacy Data Management

11

Video ObfuscationOriginal

Pixelation/Blurring

BlackOut

In-painted

12

Challenges of Video Inpainting

13

Dynamic Object In-painting Basic idea: Using object template extracted

form other time instant to complete a conceptually consistent sequence.

Steps:1. Similarity based on optimal alignment

2. Motion continuity3. Positioning of templates

?

14

Motion Continuity

? ?

15

Object-based Video In-painting

Better motion in-painting by better registration and task separation

Capable to in-paint partially and completely occluded objects

Improved computational performance (Matlab)

Number of frames with complete occlusion

Number of frames with partial occlusion

16

Public-domain Sequences

17

Complex Sequences

18

Talk Overview Video Surveillance

Subject Identification Optimal Camera Network Video Obfuscation Privacy Data Management

19

Privacy Data ManagementSubject A

Subject B

Subject C

Producer

Client A

Client B

Client C

Privacy Protection

Key question: How does a client know which subject to ask?

20

Three-agent architecture

Mediator AgentPKM,SKM

Subject AgentPKU,SKU

Client AgentPKC,SKC

RSA(TOC; PKm)

Step 2

RSA(K; PKU)TOC: U

Step 3

RSA(K; PKU)

Step 4

RSA(K; PKC)Step 5

RSA(K;PKC) RSA(K;PKC)

Step 6

Step 7

AES(Vu; K)

Step 1

RSA(K; PKU)U

RSA(TOC; PKm)

AES(Vu; K)

21

Keeping sensitive information

Medium Method Pro ConSeparateFile

Encryption + Cryptographic signature

Standard Technology Storage efficiency

Pervious to attacker Difficult to distribute with the modified video Separate authentication for modified video

Meta-data

Encryption + Cryptographic signature

Standard Technology Storage efficiency

Less pervious to attacker Depend on format

Data hiding

Encrypted watermark

Impervious to attacker Inseparable from data Joint authentication

May need more storage May affect visual quality

22

Data Hiding for Privacy Data Preservation

23

Data Hiding Data hiding/Stenography/Watermarking

Active research in the past fifteen years Typical applications include authentication,

copy detection, monitoring Challenges in our application:

Picture-in-picture: large embedding capacity Compatibility with existing compression

scheme Minimal visual distortion

24

Optimal Data Hiding

Psycho-visualModeling

Block-basedRate-Distortion

Calculation

DiscreteOptimization

Solve constrained optimization

C

CjitjiCjiD w )0,0(),(),(),(

0 20 40 60 80 100 120 140 160 180 2000

0.2

0.4

0.6

0.8

1

1.2

1.4

Rate

Dis

torti

on

Combined rate-distortion cost C(x)

# embedded bits

25

Proposed Data Hiding

Motion Compensation

DCT EntropyCoding

• DCT Domain• Frequency, contrast and luminance masking [Watson]

H.263 H.263

Encrypted foregroundvideo bit-stream

DCT PerceptualMask

ParityEmbedding

R-D Optimization

Positions of the “optimal’DCT coeff forembedding

DCT(i,j) = watermark_bit+2*round(DCT(i,j)/2)Privacy

protectedvideo

Last decodedframe

J. Paruchuri & S.-C. Cheung “Rate-Distortion Optimized Data Hiding for Privacy Protection” submitted to ISCAS 2008

26

R-D framework Target cost function:

Ri = Increase in Bandwidth of Block i Di = Perceptual Distortion in Block i δ = Relative Weight

Greedy embedding of P data bits in Block i:

Lagrangian optimization: determine the optimal Pi and λ to embed the target number of data bits:

i

iiP PCi

min

)( iii PfC

i i iii DRC )1(:

27

Examples 1/2

119kbpsNo data

Distortion 637 kbps81 kbps data

Rate & Distortion562 kbps81 kbps data

Rate only370 kbps81 kbps data

28

Examples 2/2

406.3kbpsNo data

Distortion743 kbps81 kbps data

Rate & Distortion678 kbps81 kbps data

Rate only610 kbps81 kbps data

29

Conclusions Privacy Protecting Video Surveillance

Visual Tagging for subject identification Optimized camera network for visual tagging In-painting for video obfuscation Privacy Data Management R-D optimized watermarking

Current Research Video In-painting in crowded environment Performance Evaluation for PPT Secure Reversible Modification Audio Privacy Protection Signal processing in encrypted domain