eecs 274 computer vision introduction. what is computer vision? terminator 2
TRANSCRIPT
![Page 1: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/1.jpg)
EECS 274 Computer Vision
Introduction
![Page 2: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/2.jpg)
What is computer vision?
Terminator 2
![Page 3: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/3.jpg)
Every picture tells a story
• Goal of computer vision is to write computer programs that can interpret images
![Page 4: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/4.jpg)
Can computers match (or beat) human vision?
• Yes and no (but mostly no!)– humans are much better at “hard” things– computers can be better at “easy” things
![Page 6: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/6.jpg)
Why is computer vision difficult?• Inverse problem• Ill-posed• High-dimensional data• Noise• Variation
![Page 7: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/7.jpg)
Earth viewers (3D modeling)
Image from Microsoft’s Virtual Earth(see also: Google Earth)
![Page 8: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/8.jpg)
Google streetview
![Page 9: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/9.jpg)
Photosynth
http://labs.live.com/photosynth/http://www.youtube.com/watch?v=p16frKJLVi0
by Noah Snavely, Steve Seitz, and Rick Szeliski
![Page 10: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/10.jpg)
Optical character recognition
Digit recognition, AT&T labshttp://www.research.att.com/~yann/
Technology to convert scanned docs to text• If you have a scanner, it probably came with OCR software
License plate readershttp://en.wikipedia.org/wiki/Automatic_number_plate_recognition
![Page 11: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/11.jpg)
Face detection
• Many new digital cameras now detect faces– Canon, Sony, Fuji, …
![Page 12: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/12.jpg)
Smile detection
Sony Cyber-shot® T70 Digital Still Camera
![Page 13: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/13.jpg)
Object recognition (in supermarkets)
LaneHawk by EvolutionRobotics“A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk,you are assured to get paid for it… “
![Page 14: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/14.jpg)
Face recognition
Who is she?
![Page 15: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/15.jpg)
Vision-based biometrics
“How the Afghan Girl was Identified by Her Iris Patterns” Read the story
![Page 16: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/16.jpg)
Login without a password…
Fingerprint scanners on many new laptops,
other devices
Face recognition systems now beginning to appear more widely
http://www.sensiblevision.com/
![Page 17: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/17.jpg)
Object recognition (in mobile phones)
• This is becoming real:– Microsoft Research– Point & Find, Nokia, NTT Docomo
![Page 18: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/18.jpg)
The Matrix movies, ESC Entertainment, XYZRGB, NRC
Special effects: shape capture
Bullet time:http://www.youtube.com/watch?v=J5ryLMZTO5M
![Page 19: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/19.jpg)
Pirates of the Carribean, Industrial Light and MagicClick here for interactive demo
Special effects: motion capture
![Page 20: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/20.jpg)
Sports
Sportvision first down lineNice explanation on www.howstuffworks.com
http://www.youtube.com/watch?v=UyPU2l9rdvo
![Page 21: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/21.jpg)
Smart cars
• Mobileye– Vision systems currently in high-end BMW, GM, Volvo models – By 2010: 70% of car manufacturers.– Video demo
![Page 22: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/22.jpg)
Vision-based interaction (and games)
Nintendo Wii has camera-based IRtracking built in. See Lee’s work atCMU on clever tricks on using it tocreate a multi-touch display!
Digimask: put your face on a 3D avatar.
“Game turns moviegoers into Human Joysticks”, CNETCamera tracking a crowd, based on this work.
![Page 23: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/23.jpg)
Vision-based HCI
• Reatrix: http://www.youtube.com/watch?v=QzsQKULMbiU
![Page 24: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/24.jpg)
Gaming
• Sony Eyetoy • Microsoft Natal
http://www.youtube.com/watch?v=AOXohr4XE-4&feature=related
http://www.youtube.com/watch?v=1BRSfCuLYHc
![Page 25: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/25.jpg)
Motion capture
• Marker-based motion capture– http://www.youtube.com/watch?v=V0yT8mwg9nc
• Organic motion• http://www.organicmotion.com/
![Page 26: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/26.jpg)
Looking at people• Hand gesture• Head pose• Expression • Identity
http://www.youtube.com/watch?v=NwVBzx0LMNQ
![Page 27: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/27.jpg)
Vision in space
Vision systems (JPL) used for several tasks• Panorama stitching• 3D terrain modeling• Obstacle detection, position tracking• For more, read “Computer Vision on Mars” by Matthies et al.
NASA'S Mars Exploration Rover Spirit captured this westward view from atop a low plateau where Spirit spent the closing months of 2007.
![Page 28: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/28.jpg)
Gigapan
• http://www.gigapan.org/index.php• HP TouchSmart with Gigapn demo at
Chicago O’Hare airport
![Page 29: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/29.jpg)
Robotics
http://www.robocup.org/NASA’s Mars Spirit Roverhttp://en.wikipedia.org/wiki/Spirit_rover
![Page 30: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/30.jpg)
Medical imaging
Image guided surgeryGrimson et al., MIT
3D imagingMRI, CT
![Page 31: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/31.jpg)
Digital comestics
![Page 32: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/32.jpg)
Inpainting
Bertalmio et al. SIGGRAPH 00
![Page 33: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/33.jpg)
Debluring
Fergus et al. SIGGRAPH 06
![Page 34: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/34.jpg)
Digital photo albums
• Picasa, Flickr, Photobucket, etc.• Categorization• Tagging• Search
![Page 35: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/35.jpg)
Computational photography
• Image acquisition• Hardware/software• Optics• Shuttle speed• Novel sensors• Multiple camera• Multiple shots• Multi flash• Applications: high dynamic range imaging, super
resolution, photomontage, panorama moasicing, debluring, light field, camera projector system…
![Page 36: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/36.jpg)
Image and video search
• Google• YouTubes• Microsoft• Yahoo
![Page 37: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/37.jpg)
Current state of the art• You just saw examples of current systems.
– Many of these are less than 5 years old• This is a very active research area, and rapidly
changing
– Many new applications in the next 5 years• To learn more about vision applications and
companies
– David Lowe maintains an excellent overview of vision companies• http://www.cs.ubc.ca/spider/lowe/vision.html
• Confluence of vision, graphics, learning, sensing and signal processing
![Page 38: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/38.jpg)
Software and hardware
• Algorithms: processing images and videos
• Camera: acquiring images/videos • Embedded system
![Page 39: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/39.jpg)
Topics
• Image formation: camera model, camera calibration, radiometry, color, shading
• Early vision: stereopsis, structure from motion, illumination, reflectance, shape from X, texture
• Mid-level vision: segmentation, grouping, Kalman filter, particle filter, shape representation
• High-level vision: correspondence, matching, object detection, object recognition, visual tracking
• Recent topics: image and video retrieval, internet vision
![Page 40: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/40.jpg)
Related topics
![Page 41: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/41.jpg)
Textbooks and references• Textbook
– Computer Vision: A Modern Approach, David Forsyth and Jean Ponce– Computer Vision: Algorithms and Applications (draft), Richard Szeliski
• Reference for background study: – Introductory Techniques for 3-D Computer Vision, Emanuele Trucco and
Alessandro Verri– Multiple View Geometry in Computer Vision, Richard Hartley and Andrew
Zisserman– An Invitation to 3-D Vision by Yi Ma, Stefano Soatto, and Jana Kosecka– Robot Vision, Berthold Horn– Learning OpenCV: Computer Vision with OpenCV Library, Gary Bradski and
Adrian Kaehler
• Reading assignments will be from the text and additional material that will be handed out or made available on the web page
• All lecture slides will be available on the course website http://faculty.ucmerced.edu/mhyang/course/cse274/index.htm
![Page 42: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/42.jpg)
Grading
• Based on projects• No midterm or final• 20% Homework• 40% Programming assignments• 40% Term project
![Page 43: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/43.jpg)
Project 1: features
![Page 44: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/44.jpg)
Project 2: Lucas-Kande Tracker
http://www.youtube.com/watch?v=yoQ8pSXrl4g
![Page 45: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/45.jpg)
Project 3: object detection
![Page 46: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/46.jpg)
Term Project
• Open-ended project of your choosing• Oral presentation
– Midterm presentation– Final presentation and demo
• Publish your results
![Page 47: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/47.jpg)
General Comments
• Prerequisites—these are essential!
– Data structures– A good working knowledge of MATLAB, C,
and C++ programming– Linear algebra – Vector calculus
• Course does not assume prior imaging experience
– computer vision, image processing, graphics, etc.
![Page 48: EECS 274 Computer Vision Introduction. What is computer vision? Terminator 2](https://reader035.vdocuments.us/reader035/viewer/2022062804/56649dda5503460f94ad0db0/html5/thumbnails/48.jpg)
Acknowledgements
• Slides– David Forsyth and Jean Ponce– Richard Szleski and Steve Seitz