us dto vace phase iii meeting washington dc october 2006
DESCRIPTION
US DTO VACE Phase III Meeting Washington DC October 2006. IIT-NRC Video Recognition Systems program http://iit-iti.nrc-cnrc.gc.ca http://iit-iti.nrc-cnrc.gc.ca/about-sujet/cv-vi_e.html http://synapse.vit.iit.nrc.ca (www.perceptual-vision.com) Leader: Dr. Dmitry Gorodnichy. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/1.jpg)
IIT-NRC Video Recognition Systems program
http://iit-iti.nrc-cnrc.gc.cahttp://iit-iti.nrc-cnrc.gc.ca/about-sujet/cv-vi_e.html
http://synapse.vit.iit.nrc.ca (www.perceptual-vision.com)
Leader: Dr. Dmitry Gorodnichy
US DTO VACE
Phase III Meeting
Washington DC October 2006
![Page 2: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/2.jpg)
NRC-IIT Video Recognition Systems program
+
Canadian national interests in deploying video technologies
Leader: Dr. Dmitry Gorodnichy
US DTO VACE
Phase III Meeting
Washington DC 1/XI/2006
![Page 3: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/3.jpg)
3. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Outline
Part 1: - Who we are: NRC IIT CVG VRS
- VRS projects and results
Part 2:- Canadian National/Regional interests in Video Technology– Government (OGD) contacts and programs– Universities, Industry references
(in particular wrt national security)
![Page 4: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/4.jpg)
4. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
National Research Council of Canada (NRC-CNRC, or NRC)• Over 30 institutes
• Across Canada, Divided in clusters
• Logo: “From Discovery to Innovation…”
• Mandate: “Make science work for Canada”(i.e. provide solutions for OGD and industry)
• Funding: government & revenue
![Page 5: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/5.jpg)
5. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Social values
driver
impact
NRCUniversitiesOGDsIndustry
Technology/Services
Knowledge/ Discovery
Industry
IIT
NRC’s role & place
ENG
IMTI
IMI
IOT
IAR
PBI
IBDIMB
BRI
HIAIBS INMS
NINT
IIT
ICPET
IMS
IRC
IFCI
SIMS
LS PS
GHIINH
IIT
![Page 6: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/6.jpg)
6. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
CBSA, RCMP DRDC, DND/CF
CSIS, PPTC, CIC, PCOPSERC, TC, CATSA
CPRC
- VRS program
Flight Facility
papers,conferencesCVPR, CRV
Social values
driver
impact
NRCUniversitiesOGDsIndustry
wrt. Video Technologies
Technology/Services
Knowledge/ Discovery
Industry
IIT
Other clients:HealthMedia
Education
NRC’s role & place
…
![Page 7: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/7.jpg)
7. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A closer look
![Page 8: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/8.jpg)
8. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A closer look
![Page 9: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/9.jpg)
9. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A closer look
FF
IIT
![Page 10: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/10.jpg)
10. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Institute for Information Technology (IIT-ITI, or IIT)
Revenue & Forecast
$0
$200,000
$400,000
$600,000
$800,000
$1,000,000
$1,200,000
$1,400,000
$1,600,000
$1,800,000
$2,000,000
FY 03-04 FY 04-05 FY 05-06 (Forecast)
Fee forServiceRevenue
CollaborativeRevenue
LicensingRevenue
Level of Activityby Site
Fredericton
Gatineau
Moncton
Ottawa
Saint JohnSydney
Total: $18,976M
NRC-IIT generates most of its revenues through licensing its enabling technologies
• Funding: Govern + Revenues
• Location: Ottawa + Atlantic (e-IT)
Make smth that works (and sales)!
![Page 11: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/11.jpg)
11. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
IIT: Areas of Specialty
•3D Imaging, Modeling and Visualization•Data and Text Mining •Computational Video
•Natural Language Processing •Information Security, Privacy and Trust•Human Computer Interaction
•Intelligent Internet Applications•Software Engineering
![Page 12: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/12.jpg)
12. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
IIT: Some technologies
• From Space Shuttle (Canadarm & Canadarm2), to Hollywood, to Fine Arts, to Security applications
![Page 13: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/13.jpg)
13. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Video Recognition Systems(VRS)
• Part of Computational Video Group (formed in 2001)
• Our Mandate: Develop Video Recognition technologies for Canadian companies and OGD’s
• Formerly called Perceptual Vision Technology
• Higher mission:
Computer Vision allows computers to see.Perceptual Vision allows computers to understand
what they see.™
Test-bed and Criteria: If you are able to recognize him / it,
why computer can’t ?..
![Page 14: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/14.jpg)
14. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
What we do
• Research, coding… • Working with companies: consulting, making prototypes• Working with OGD: consulting, joint proposals
• “Outreach & influence”: raising awareness about Video Recognition Technology – as opposed to Video Technology– public (as you see surveill. cam in the hall or watch TV), – scientists (from diff. dept: CS,Phys,NeuroBio,Psych)– government (as they allocate money)
• Organizing conferences – CRV VideoRec’07
What we don’t do (vs. Universities): teaching (except specialized lectures)
![Page 15: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/15.jpg)
15. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A bit of history
• First projects: Canadarm2• Then: Nouse™, Surveilance
![Page 16: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/16.jpg)
16. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
VRS now • 4 permanent Dr.’s, • 3-10 contract / guest /
coop / student workers• 2 Tech.Support
• Collaborate with Ottawa U. (VIVA lab)
• Works closely with OGD that require Video Tech.
• Lead: D. Gorodnichy
• Collaborators: M. Fiala:- Markers Tracking algorithms - Panoramic Surveillance systems
G.Roth:- Feature detection, - Projective Vision algorithms
![Page 17: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/17.jpg)
17. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Application directions
Domain 1: Security, Surveillance and Monitoring.
Domain 2: Visually-enabled computer-human interaction (inc. hand-free user interfaces)
Domain 3: Visually-enhanced communication and Intelligent video processing (inc. annotation of video)
• Canada Borders Services Agency (CBSA)• Canadian Police Research Center (CPRC, RCMP)
• Ottawa Health Center
• U of Ottawa Music Dept.• Several Canadian/Ontario private companies
(And partners)
![Page 18: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/18.jpg)
18. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Research & Results Technologies developed
1. Neuro-associative memorization/ recognition Face Recognition from (low-res) Video (Better than correlogram) Object
learning
2. (Multiple) object detection and tracking Automated tele-operating, …
3. Nose (convex-shape) tracking Hands-free interfaces
4. Critical Evidence Snapshot extraction Intelligent Surveillance
5. Fingers, hands detection tracking Piano playing annotation
![Page 19: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/19.jpg)
19. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
1. Neuro-associative memorization/ recognition Face Recognition from (low-res) Video (Better than correlogram) Object
learning
2. (Multiple) object detection and tracking Automated tele-operating, …
3. Nose (convex-shape) tracking Hands-free interfaces
4. Critical Evidence Snapshot extraction Intelligent Surveillance
5. Fingers, hands detection tracking Piano playing annotation
Research & Results Technologies developed
(X,Y)
name
![Page 20: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/20.jpg)
20. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
1. Face Recognition in VideoGoal:
Recognize faces from regular video quality:
Bad news: low-resolution (TV:320x260) data i.o.d = 12 pixels !Good news: face detectors (Viola,CMU,MIT,Pitts) can detect such faces !• So, real problem is: Different recognition modality!• So, for solution: make use of temporal domain! (“Super-resolution” comment)
• Applications: (not for identification (yet), but …) for user recognition, TV annotation, soft biometrics, multiple-people tracking, multiple-camera tracking, back-tracking
![Page 21: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/21.jpg)
21. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
12 pixels between the eyes should be sufficient
Nominal face resolution for video-based recognition
1) Efficient visual attention mechanisms (motion etc)
2) Decision based on accumulating results over t.
3) Three main principles of neuro-processing
1. non-linear processing,
2. massively distributed collective decision making
3. synaptic plasticity.
Allows:
a) to accumulate learning data in time by adjusting synapses,
b) to associate a visual stimulus to a semantic meaning based on the computed synaptic values
Keys principles we adhere to (from human vision recognition system)
(X,Y)
“Paul”
![Page 22: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/22.jpg)
22. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Face Recognition in Video
Solution: - 12 pixel i.o.d. face model +
accumulation over time while tracking (both in training and recognition)
- combination of neuro-biological and computer vision approaches
Tested: on TV shows, on computer user recognition
![Page 23: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/23.jpg)
23. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
See also:
Image and Vision Computing Journal, Special Issue on “Face Processing in Video Sequences”.
(Vol. 24, No. 6). June 1, 2006. – Six-page Editorial…
– On what makes video processing special– On nominal (optimal for video) face resolution– On promising directions and future trends– On video test-sets: Why not to use “Gone with the wind”?
Also• CVPR’04 Workshop on Face Processing in Video (FPiV’04), Washington
(www.vision-interface.net/fpiv04 )• CRV’05 Workshop on Face Processing in Video (FPiV’05), Victoria
(www.vision-interface.net/fpiv05 )
![Page 24: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/24.jpg)
24. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Goals: - make surveillance
data manageable, - be suitable for
long-range monitoring, - be suitable for
low quality video (which they often are), - be affordable (time-wise, space-wise)Challenge: foreground is not easy to detect/track in low-quality
data: thru glass, blurred, against sun (24h), etc..Solution:- Automated extraction of A.C.E. (Annotated Critical Evidence) - Integrates recent advances in video detection and tracking
2. A.C.E. Surveillance
![Page 25: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/25.jpg)
25. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
A.C.E. Surveillance• Is currently used to monitor
several premises 24/7 (long-term)
• Fast/Succinctsummarization
obtained
(in terms of ACE snapshots)
• Little space
• Browsable by time
by objectby similarity*
![Page 26: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/26.jpg)
26. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
From 22:00 to 7:00with web-cam
See also:CRV’06 Workshop on Video Processing for Security (VP4S-06), Quebec.
(www.computer-vision.org/4security )
![Page 27: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/27.jpg)
27. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
3. Automated Tele-operator
• Replaces manual video operator: automatically switches cameras
• Code taken from surveillance monitoring application(another test-bed comment on Test Data)
![Page 28: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/28.jpg)
28. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
4. Nouse™ (Nose as Mouse)
• Goal: To enable hands-free control
• Problem: sub-pixel precision, robustness and convenience is needed.
+ For truly hand-free user face recognition is required.
x y , z PUI
monitor
binary eventON
OFF
recognition /memorization
Unknown User!
![Page 29: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/29.jpg)
29. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
5. Pianist playing annotationWhich hand/finger played a note ?
• Problem: Detect/Track hands and fingers. Multi-object, deformable object tracking is hard.
• Unbiased test-bed for hands tracking – very different from sign language + pianist hands (flexible and fast)
![Page 30: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/30.jpg)
30. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Image Search for Security Apps.(Gerhard Roth)
• UAV takes large number of images–Returns to same environment at a later time
–Find images matching ones previously taken
• City has a host of cameras at intersections–Given a single view of a vehicle
–Want to find this vehicle at other intersections
• Take many views of an object or vehicle–Many images of vehicle at different viewpoints
–Want to find this vehicle in surveillance video
![Page 31: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/31.jpg)
31. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
•bi-tonal (only black and white)•4 corners: for 6-DOF camera/pattern pose determination•Digital Methods: Error Correction, CRC-16 Checksum
ARTag Fiducial Marker System (Mark Fiala)
![Page 32: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/32.jpg)
32. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
•Augmented Reality
•General Purpose Tracker
•Robot Navigation and control
Marker Detection Applications(Fiala, Roth)
![Page 33: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/33.jpg)
33. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Part 2:
Interests in Video Technology from OGD
Receptor capacity for Video Technology: users and developers
National/Regional security programs and intentions
![Page 34: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/34.jpg)
34. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Ottawa sector and national(by interest as customers vs. R&D)
* - expressed interest in participation in external evaluation for DTO VACE technologies
CSIS“CIA”
Also: PPTC, PSERC
* CF / DND(Forces)
Classified only
Industry - CRC (Communication
Research Center )
DRDC – NI“DARPA”
(Biometrics)
* RCMP(Police)
* DRDC – AIS (UAV)
DRE S/V…
* CBSA - LSSD
Border Services
* NRC – FRL(Flight
Facilities)
* NRC-IITCVG-VRS program
Capacity/Readiness for R&D
Capacity/Readiness for testing/using as customer
![Page 35: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/35.jpg)
35. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Companies to know
• Point Grey Research (Vancouver, UBC) – firewire cams & video processing
• i3DVR – “Intelligent” DVR
• Deep Development Corp / Gatekeeper Systems “High Resolution for Tough Environments” – DVR (won US Force contract for Boeing Military planes)
• March Networks – DVRs
![Page 36: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/36.jpg)
36. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Universities to know
• UBC (Vancouver)
• Ottawa U, VIVA Lab works closely with NRC
• York U (Toronto)
• …(see PC for CRV VideoRec’07 workshop)
(also www.vision-interface.net – for complete list)
![Page 37: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/37.jpg)
37. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
17 slides omitted for online version
![Page 38: US DTO VACE Phase III Meeting Washington DC October 2006](https://reader034.vdocuments.us/reader034/viewer/2022051419/568159e6550346895dc73037/html5/thumbnails/38.jpg)
38. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA
Summary
• How can we contribute to VACE:– Science-wise: Reviewing, Testing
– Client-wise: Testing
– Liaisoning with OGD and national/regional/municipal surveillance programs
– Consulting, Tips, Feedback, Cheering-up :)
– Inviting (Making) everyone here to participate in:
CRV Workshop on Video Processing and Recognition(VideoRec’07)
Montreal, Canada, May 28-30