![Page 1: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/1.jpg)
Computer Vision
EE837, CS867, CE803
![Page 2: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/2.jpg)
Introduction
Lecture 01 Computer Vision
![Page 3: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/3.jpg)
• Basic linear Algebra, probability, calculus - Required
• Basic data structures/programming knowledge - Required
• Working knowledge of MATLAB - Required
• Knowledge and understanding of basic image processing - Preferable
Prerequisites
![Page 4: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/4.jpg)
• Class slides, research papers, tutorials and supplemental material
• Linda G. Shapiro and George Stockman, “Computer Vision”, Upper Saddle River, NJ: Prentice Hall, 2001.
• David A. Forsyth and Jean Ponce, “Computer Vision A Modern Approach”, 2nd edition, Prentice Hall, Inc., 2003.
• Richard Szeliski , “Computer Vision: Algorithms and Applications”, Springer; 2011 edition, 2011. Made available online by the author: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf
Extra
List of CV books http://homepages.inf.ed.ac.uk/rbf/CVonline/books.htm
Text and Reading
![Page 5: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/5.jpg)
• Camera geometry and basic transformations
• Camera calibration and camera- parameters estimation
• Sources, shadows, shading and shape from shading
• Feature Extraction
• Texture Synthesis
• Template Matching and Image Registration
• Segmentation
• Vision based Tracking
• Multiple view geometry
Broader course topics
![Page 6: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/6.jpg)
Homework – Mostly programming assignments: 15%
Midterm/Hourly: 15%
Surprise quizzes/attendance: 10%
Final Project: 30%
Final exam: 30%
Grading policy
![Page 7: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/7.jpg)
• Homework/programming assignments:– Reports should be type-written– Code and program output are required
• Final Project:– Brain storm on project ideas– Project highlights – 10 minute each group– Individual or a group of max two with individual roles clearly defined– Type-written report upto 10 pages in CVPR format, with additional pages for commented codes as appendix. – Project presentation
• Late Policy:– No credit for late submissions
Grading policy
![Page 8: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/8.jpg)
• Plagiarism is strictly prohibited• Cite the source• Negative marking will be done, where found
Grading policy
![Page 9: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/9.jpg)
• Dr George Stockman Professor Emeritus, Michigan State University
• Dr Mubarak Shah Professor, University of Central Florida
• The Robotics Institute Carnegie Mellon University
Material citations
![Page 10: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/10.jpg)
Any queries?
By appointment only:
Cdr Dr Hammad
PG 111
Preferably Tuesday and Wednesday11AM –Noon
Email: [email protected]
![Page 11: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/11.jpg)
Lets start !!!
![Page 12: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/12.jpg)
What is an image?
What we see
What a computer sees
![Page 13: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/13.jpg)
What is an image?
What we see
What a computer sees
![Page 14: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/14.jpg)
Where is the Sun?
![Page 15: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/15.jpg)
Image Processing
Fourier TransformSampling, Convolution
Image enhancement Feature detection
![Page 16: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/16.jpg)
What is Computer Vision?
• Inverse Optics
• Intelligent interpretation of Imagery
• Building a Visual CortexPart of the cerebral cortex responsible for processing visual
information
• No matter what your definition is…
Vision is complex…….but is FUN !!!
![Page 17: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/17.jpg)
Difference between CV and IP
• Image processing: Process the output of sensors.Computer vision: Relates the output of the sensors to real world.
• Image processing: The output is a transformed image.Computer vision: The output is usually a decision.
• Image Processing: Signal processing.Computer Vision: Artificial Intelligence .
• Defect detection or automatic driving relates to ?Enhancing an image relates to ?
![Page 18: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/18.jpg)
Lighting
Scene
Camera
Computer
Scene Interpretation
Components of a Computer Vision System
Image acquisition
![Page 19: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/19.jpg)
Video clip
![Page 20: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/20.jpg)
Sequence of images
16 images in succession that shows motion
![Page 21: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/21.jpg)
Shape from shading
• Shade deceives human visual system• Changes the 3D shape• Gradual variation of the shading gives 3D information
![Page 22: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/22.jpg)
(1,0,1) (-1,1,1) (-1,-1,1)
Shape from texture
![Page 23: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/23.jpg)
3D from Shading
Shape from Shading
![Page 24: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/24.jpg)
Shape from texture
• Same shape (circles) repeated, forms texture• Circles become ellipses at some places• Gives 3D cue• Texture can be used to recover 3D
![Page 25: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/25.jpg)
Shape from motion
• Cannot understand just from dots that what it is• Humans have this capability to understand motion
![Page 26: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/26.jpg)
Shape from motion
![Page 27: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/27.jpg)
Optical flow
• Color wheel• Completing pixel wise motion
Sequence Raw optical flow
![Page 28: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/28.jpg)
Optical Flow
![Page 29: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/29.jpg)
Microsoft photosynth
• Panorama stitching• Can capture in amazing resolution and full 3D.• For anyone with a D-SLR (Single Lens Reflex) or a point-and-shoot camera.• https://photosynth.net/preview/about/
![Page 30: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/30.jpg)
Video clip and mosaic
• Stitching images together
![Page 31: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/31.jpg)
Applications of Computer Vision
• Face Recognition
• Object Recognition
• Video Surveillance and Monitoring
• Object detection, tracking and behavior analysis
• Remote Sensing: UAVs
• Robotics
• Computer Graphics
• And more ………….
![Page 32: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/32.jpg)
Face Recognition
• Principle Components Analysis (PCA)• Fisher Linear Discriminant (FLD)
![Page 33: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/33.jpg)
Face recognition
![Page 34: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/34.jpg)
Facial expression
Surprised Smiling
![Page 35: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/35.jpg)
Detecting driver alertness
![Page 36: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/36.jpg)
Human detection
• Left – UAV image• Bounding boxes• Will learn basic techniques on how we can track these moving objects
![Page 37: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/37.jpg)
Video surveillance and monitoring
• Automated surveillance systems – Detection and tracking
Object detection Object tracking Object classification
Activity recognition
![Page 38: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/38.jpg)
Airport surveillance
![Page 39: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/39.jpg)
Aerial imagery - UAVs
• Drones Military use
Instead of drones many want to brand the technology as "Unmanned Aerial Systems" (UAS) in preference over "drones.“
• Aerial surveying of crops• Acrobatic aerial footage in filmmaking• Search and rescue operations• Inspecting power lines and pipelines• Counting wildlife• Delivering medical supplies to inaccessible regions
![Page 40: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/40.jpg)
Aerial imagery
Object tracking
Kernel tracking +blob tracking +
occlusion
Motion detection
Frame differencing + background modeling +
object segmentation
Camera motion compensation
Feature based +gradient
Event detection and tracking
![Page 41: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/41.jpg)
Aerial imagery – Registration results
![Page 42: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/42.jpg)
Aerial imagery – Detection results
![Page 43: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/43.jpg)
Aerial imagery – Tracking results
![Page 44: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/44.jpg)
Wide area surveillance
![Page 45: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/45.jpg)
Wide area surveillance
![Page 46: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/46.jpg)
Tracking results
![Page 47: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/47.jpg)
Unmanned Ground Vehicle
• Comes under Robot vision
• Google Self driving car
• The system combines information from Google Street View with artificial intelligence software that combines input from:
• Video cameras inside the car• Identifying pedestrians and moving obstacles
• LIDAR sensor on top of the vehicle• For 3D map
• Radar sensors on the front of the vehicle• Position of distant objects
• Position sensor attached to one of the rear wheels• Locate the car's position on the map.
![Page 48: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/48.jpg)
Unmanned Ground Vehicle
• Defense Advanced Research Projects Agency (DARPA) urban challenge
![Page 49: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/49.jpg)
Human activity recognition
• Involves
• Events• Actions• Activities
• Different datasets available for analysis
![Page 50: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/50.jpg)
Human activity recognition - datasets
• Weizmann action dataset• 10 actions• 09 actors per action
• KTH Data Set• 06 categories• 25 actors• 04 instances• 600 clips
![Page 51: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/51.jpg)
Human activity recognition - datasets
• UCF Sports dataset• 9 actions• 142 videos
• IMAX multi-view dataset
Bench swing Dive Swing Run
Kick Lift Ride Golf swing Skate
![Page 52: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/52.jpg)
Human activity recognition - datasets• UCF 50
![Page 53: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/53.jpg)
Stereo• Regular camera lose 3D information• Microsoft Kinect sensor – game changer• Gives direct 3D information + RGB image• 50,000 different gestures – Challenge is that can you identify all/some of these
3D depth sensors
RGB Camera
IR LED Emitter
Array of microphones
Tilt motor
![Page 54: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/54.jpg)
Binocular Stereo
![Page 55: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/55.jpg)
Stereo• Regular camera lose 3D information
![Page 56: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/56.jpg)
Range Scanning and Structured Light
![Page 57: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/57.jpg)
High density crowded scenes
• Tracking required for:
• Crowd management• Public space design• Virtual environments• Visual surveillance• Intelligent environments• And more !!!
![Page 58: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/58.jpg)
High density crowded scenes• Can we do tracking in this kind of crowd?
Political Rallies Religious Festivals Marathons High Density Moving Objects
![Page 59: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/59.jpg)
High density crowded scenes• Can we do tracking in this kind of crowd?
• Average chip size 14 x 22 pixels• 492 Frames• Selected 199 athletes for tracking• Successfully tracked 143 athletes
![Page 60: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/60.jpg)
High density crowded scenes• Can we do tracking in this kind of crowd?
![Page 61: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/61.jpg)
High density crowded scenes• Can we do tracking in this kind of crowd?
• Average chip size 14 x 17 pixels• 453 Frames• Selected 50 athletes for tracking
![Page 62: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/62.jpg)
High density crowded scenes• Can we do tracking in this kind of crowd?
![Page 63: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/63.jpg)
Behaviors in crowded scenes• Can we identify the behavior of the crowd?
![Page 64: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/64.jpg)
Image localization
Location in terms of Longitude (40.4419) Latitude (40.4419)
Input Output
• Image compared with database of images
![Page 65: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/65.jpg)
Geospatial trajectory extraction• Sequence of images compared with database
![Page 66: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/66.jpg)
Computer graphics• CV used for movies like Harry Potter, Avatar, Matrix etc
![Page 67: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/67.jpg)
Layer based image composition
• Segmentation method• Green Chroma key screen• Green and blue differ the most in hue from skin colors
Virtual studiohttp://en.wikipedia.org/wiki/Chroma_key
![Page 68: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/68.jpg)
Layer based video composition• Segmentation method
![Page 69: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/69.jpg)
Layer based video composition• Segmentation method
![Page 70: EE837, CS867, CE803 Computer Vision - umartalhaumartalha.weebly.com/uploads/8/3/8/2/8382263/01_lecture_cv_nust_… · EE837, CS867, CE803. Introduction Lecture 01 Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081601/60f734a4a827c53bed434fa5/html5/thumbnails/70.jpg)
Industrial robots vs low skilled workers