dmitry gorodnichy and tony mungham

23
Automated video surveillance: challenges and solutions. ACE Surveillance (Annotated Critical Evidence) case study. Dmitry Gorodnichy and Tony Mungham Laboratory & Scientific Services Directorate Canada Border Services Agency www.videorecognition.com/ACE

Upload: sirius

Post on 23-Jan-2016

41 views

Category:

Documents


0 download

DESCRIPTION

Automated video surveillance: challenges and solutions. ACE Surveillance (Annotated Critical Evidence) case study. Dmitry Gorodnichy and Tony Mungham Laboratory & Scientific Services Directorate Canada Border Services Agency www.videorecognition.com/ACE. Outline. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Dmitry Gorodnichy and Tony Mungham

Automated video surveillance: challenges and solutions.

ACE Surveillance (Annotated Critical Evidence) case study.

Dmitry Gorodnichy and Tony Mungham

Laboratory & Scientific Services DirectorateCanada Border Services Agency

www.videorecognition.com/ACE

Page 2: Dmitry Gorodnichy and Tony Mungham

2

Outline

Problems with status-quo Video Surveillance Real-time and archival problems Operational considerations

Next generation solution - Video Analytics based “Motion detection” myth and problem “Object detection” as example of real intelligence

ACE Surveillance – first fully-functional object- detection-based prototype Year long tests with different levels of complexity What that means for future of Video Surveillance

Conclusions

Page 3: Dmitry Gorodnichy and Tony Mungham

3

Role of Video Technology (VT)

In the context of enhancing security, Video Technology (VT) is one of the most demanded technologies of the 21st century It is publicly acceptable It provides rich in content data

Multi-million funding in Canada and worldwide: CBSA Port Runner project invested 10s of Millions in

CCTV upgrades Transport Canada opens $35M of funding towards

procurement of CCTV

Page 4: Dmitry Gorodnichy and Tony Mungham

4

VT at CBSA

CBSA is a major user of CCTV systems at POEs Most major CCTV installations start to leverage VT Current task: to lead applied R&D to push VT to help

CBSA apply S&T innovative approaches to border management: Event detection and notification to provide effective response

to events Traffic trends analysis to assist with border management Video storage management to manage the cost of storage

and meet obligations under the privacy act Data integration/fusion of contextualised video information

Page 5: Dmitry Gorodnichy and Tony Mungham

5

Problem with status quo use of CCTV surveillance

Modes of operation:

1. Active - personnel watch video at all times

2. Passive - in conjunction with other duties

3. Archival - for post-event analysis

Current systems and protocols are not efficient in either mode!

Problem in real-time modes: an event may easily pass unnoticed . due to false or simultaneous alarms, lack of time needed to rewind and analyse all video

streams.

Page 6: Dmitry Gorodnichy and Tony Mungham

6

Problems in Archival mode:

Due to temporal nature of data:

1. Storage space consumption problem• Typical assignment:

2-16 cameras, 7 or 30 days of recording, 2-10 Mb / min.1.5 GB per day per camera / 20 - 700 GB total !

2. Data management and retrieval problem• London bombing video backtracking experience:

“Manual browsing of millions of hours of digitized video from thousands of cameras proved impossible within time-sensed period”

[by the Scotland Yard trying to back-track the suspects]

Page 7: Dmitry Gorodnichy and Tony Mungham

7

Operational considerations

Lots of CCTV infrastructure: Many local initiatives, not coordinated

Most video technology decisions are influenced by vendors - short-term solutions

Over 30 different video systems within the same dept. (at RCMP)

A national program with proper benchmark-based planning and evaluation of VT is required Leveraging advances recently made in S&T Technical standards for capturing /saving video data. Policies in when, where and how VT should be used.

Page 8: Dmitry Gorodnichy and Tony Mungham

8

Video Technology today

Video Analytics (Video Recognition)

Analog

20th century

21st century

First video recording

Digital

Wireless, Network Connected (IP)

Page 9: Dmitry Gorodnichy and Tony Mungham

9

Next generation Video Technology

Is Video Analytics basedalso identified as:

Video Recognition, Intelligent Video,Smart Video / Smart CameraVideo Analysis & Content ExtractionPerceptual Vision

is not much about capturing better data (better lenses, grabbers, coders, transmitters)

but about understanding captured data (better theory)

Page 10: Dmitry Gorodnichy and Tony Mungham

10

Status-quo “video intelligence”

Transport Canada CCTV Reference Manual for Security Application .

Australian Government National code of practice for CCTV applications in urban transport

USA Government :recommended security Guidelines for Airport Planning, Design and Construction.

…. refer to “Motion-based” capture as Intelligent Surveillance Technology, and make their recommendations based on thereon.

Page 11: Dmitry Gorodnichy and Tony Mungham

11

“Motion-detection” is not intelligent!

Term “Motion-based” is coined to make people believe that video recognition is happening, which is not!

It’s actually illumination-change-based, as it uses simple point brightness comparison:

Which often happens not because of motion!Changing light / weather (esp. in 24/7 monitoring)Against sun/light, out of focus, blurred, thru glassReflections, diffraction, optical interferences Image transmission, compression losses

Page 12: Dmitry Gorodnichy and Tony Mungham

12

“Object-detection” is intelligent …

… but few can do it, since necessary advances in video recognition theory became possible only recently (>2002).

In 2002 National Research Council of Canada (NRC) starts developing Video Recognition Systems to leverage its scientific Video Recognition expertise for the industry.

In 2005, it develops ACE Surveillance:an object-detection-based Automated surveillanCE prototype capable of automatically extracting Annotated Critical Evidence from live video.

NRC becomes also the organizer of the first Canadian academic workshops dedicated to Video Processing for Security (since 2004)

Page 13: Dmitry Gorodnichy and Tony Mungham

13

What is ACE Surveillance?

A Windows software that performs real-time video analytics by integrating best object detection and tracking algorithms.

Replaces video clips with annotated still images: Compresses 1 Gb of video into 2 Mb of easy to browse and

analyze still images ACE Surveillance output:

A 7-hour activity from day to night (17:00 - 24:00) is summarized into 2 minutes (600Kb) of Annotated Critical Evidence snapshots.

Note illumination changes! - Watch tree shadows and sun light.

Page 14: Dmitry Gorodnichy and Tony Mungham

14

ACE Surveillance architecture

Works with ordinary USB cameras or CCTV cameras with USB video converters.

Adds on top of existing infrastructure using an ordinary desktop computer.

Video clips (Tb)...

Archival mode of operation

Real-time mode of operation

Last captured CES

ALARM!

C.E.S. (Gb)

ACE Capture ACE Browser

Page 15: Dmitry Gorodnichy and Tony Mungham

15

Adds on top of existing infrastructure

Page 16: Dmitry Gorodnichy and Tony Mungham

16

Status quo “Motion-based” capture(Courtesy: NRC-IIT Video Recognition Systems project)

1. Many captured snapshots are useless: either noise or redundant

2. Without visual annotation, motion information is lost.

3. Hourly distribution of snapshots is not useful

Page 17: Dmitry Gorodnichy and Tony Mungham

17

ACE Surveillance “Object-based” capture(Courtesy: NRC Video Recognition Systems project)

1. Each captured shot is useful.

2. Object location and velocity shown augmentent.

3. Hourly distribution of shots is indicative of what happened in each hour, provides good summarization of activities over long period of time.

Page 18: Dmitry Gorodnichy and Tony Mungham

18

ACE Surveillance testing benchmarks

Tested in different levels of complexity: lighting conditions, object motion patterns, camera location environmental constraints.

most difficult - outdoors in unconstrained environments with little or no object motion consistency (as around a private house in a regular neighbourhood).

most easy - in controlled indoor environment where minimal direct sunlight is present and where all objects are of approximately the same size and exhibit similar motion pattern (as at access gate inside the business building).

Page 19: Dmitry Gorodnichy and Tony Mungham

VT within CBSA 19

Outdoor, wireless, eye-level Outdoor, webcam, overview Indoor with sunlight, CCTV Indoor w/o sunlight, CCTV

Page 20: Dmitry Gorodnichy and Tony Mungham

20

Enables efficient detection of abnormal activities Delivery EntryBack Door Entry

On

wee

k-da

yO

n w

eek-

end

More than usual

Page 21: Dmitry Gorodnichy and Tony Mungham

21

ACE Surveillance results

In real-time mode: alarm sounds & last captured evidence (time-stamped) is shown.

In archival mode: “Zoom on the evidence” browsing of captured evidences – zoom on a day, on hour, then on event - point and click (for high res as needed)

Made Commissioners much more aware of activities.

Page 22: Dmitry Gorodnichy and Tony Mungham

22

Conclusions

Affordable automated (intelligent) video surveillance (AVS) is possible! To replace traditional DVR OR to supplement them: DVR for 1 month + AVS for 1 year

However: Requires extra training from security officers. Requires new protocols to handle automatically extracted

evidence. - From forensic prospective, data that are not original and have

been processed by a computer can not be considered as evidence. Requires new privacy policies.

- Surveillance data are normally not kept for a long period of time (<1 month), due to their size. AVS allows to store on local machine many months (even years) of evidence data.

Page 23: Dmitry Gorodnichy and Tony Mungham

23

ACE surveillance case study outcome

ACE Surveillance (which is developed by a research lab) provides a reference standard against which can be measured solutions coming from industry.

It deals with common misconceptions related to deploying intelligent video surveillance systems (IVS): “motion detection” myth vs real object detection and tracking. The “one-fit-all” myth. - Extra video analytics expertise is required to

set and operate IVS. better video data (better resolution or compression) do not imply

better video intelligence. - ACE Surveillance is shown to work with regular TV quality data (320 by 240 pixels).

However better quality of video image is needed for forensic purposes as evidence

Due to closing of the project by NRC, CBSA takes lead on it.