![Page 1: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/1.jpg)
![Page 2: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/2.jpg)
![Page 3: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/3.jpg)
Have you ever used computer vision?
How? Where?
Think-Pair-Share
![Page 4: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/4.jpg)
Jitendra Malik, UC Berkeley
Three ‘R’s of Computer Vision
![Page 5: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/5.jpg)
Jitendra Malik, UC Berkeley
Three ‘R’s of Computer Vision
“The classic problems of computational vision:
reconstruction
recognition
(re)organization.”
![Page 6: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/6.jpg)
Have you ever used computer vision?
How? Where?
Reconstruction? Recognition? (Re)organization?
Think-Pair-Share
![Page 7: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/7.jpg)
• Laptop: Biometrics auto-login (face recognition, 3D), OCR
• Smartphones: QR codes, computational photography (Android Lens Blur, iPhone Portrait Mode), panorama construction (Google Photo Spheres), face detection, expression detection (smile), Snapchat filters (face tracking), Google Tango (3D reconstruction),
• Web: Image search, Google photos (face recognition, object recognition, scene recognition, geolocalization from vision), Facebook (image captioning), Google maps aerial imaging (image stitching), YouTube (content categorization)
• VR/AR: outside-in tracking (HTC VIVE), inside out tracking (simultaneous localization and mapping, HoloLens)
• Xbox: Kinect, full body tracking of skeleton, gesture recognition
• Medical imaging: CAT / MRI reconstruction, assisted diagnosis, automatic pathology, connectomics, endoscopic surgery
• Industry: vision-based robotics (marker-based), machine-assisted router (jig), automated post, ANPR (number plates), surveillance, drones
• Transportation: assisted driving (everything), face tracking/iris dilation for drunkeness, drowsiness
• Media: Visual effects for film, TV (reconstruction), virtual sports replay (reconstruction), semantics-based auto edits (reconstruction, recognition)
![Page 8: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/8.jpg)
Optical character recognition (OCR)
Digit recognition, AT&T labs
http://www.research.att.com/~yann/
Technology to convert scanned docs to text• If you have a scanner, it probably came with OCR software
License plate readershttp://en.wikipedia.org/wiki/Automatic_number_plate_recognition
JH
![Page 9: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/9.jpg)
Face detection
• Almost all digital cameras detect faces
• Snapchat face filters
![Page 10: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/10.jpg)
![Page 11: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/11.jpg)
![Page 12: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/12.jpg)
![Page 13: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/13.jpg)
Smile detection
Sony Cyber-shot® T70 Digital Still Camera JH
![Page 14: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/14.jpg)
Object recognition (in supermarkets)
LaneHawk by EvolutionRobotics
“A smart camera is flush-mounted in the checkout lane, continuously
watching for items. When an item is detected and recognized, the
cashier verifies the quantity of items that were found under the basket,
and continues to close the transaction. The item can remain under the
basket, and with LaneHawk,you are assured to get paid for it… “
JH
![Page 15: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/15.jpg)
Vision-based biometrics
“How the Afghan Girl was Identified by Her Iris Patterns” Read the story
wikipedia
JH
![Page 16: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/16.jpg)
Login without a password…
![Page 17: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/17.jpg)
Login without a password…
![Page 18: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/18.jpg)
Login without a password…
![Page 19: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/19.jpg)
Object recognition (in mobile phones)
Google GogglesWikipedia: “In a May 2014 update to Google Mobile for iOS, the Google
Goggles feature was removed due to being ‘of no clear use to too many people.’“ : (
![Page 20: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/20.jpg)
3D from images
Building Rome in a Day: Agarwal et al. 2009
![Page 21: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/21.jpg)
Human shape capture
![Page 22: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/22.jpg)
Human shape capture
![Page 23: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/23.jpg)
Human shape capture
![Page 24: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/24.jpg)
Human shape capture
![Page 25: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/25.jpg)
Star Wars: Rogue One – Peter Cushing / Admiral Tarkin
Special effects: shape capture
![Page 26: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/26.jpg)
Special effects: shape capture
![Page 27: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/27.jpg)
Special effects: motion capture
![Page 28: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/28.jpg)
Interactive Games: Kinect
• Object Recognition: http://www.youtube.com/watch?feature=iv&v=fQ59dXOo63o
• Mario: http://www.youtube.com/watch?v=8CTJL5lUjHg
• 3D: http://www.youtube.com/watch?v=7QrnwoO1-8A
• Robot: http://www.youtube.com/watch?v=w8BmgtMKFbY
JH
![Page 29: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/29.jpg)
Sports
Sportvision first down line
Nice explanation on www.howstuffworks.com
http://www.sportvision.com/video.html
JH
![Page 30: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/30.jpg)
Medical imaging
Image guided surgery
Grimson et al., MIT3D imaging
MRI, CT
JH
![Page 31: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/31.jpg)
AutoCars - Uber bought CMU’s lab
![Page 32: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/32.jpg)
![Page 33: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/33.jpg)
Industrial robots
Vision-guided robots position nut runners on wheels
JH
![Page 34: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/34.jpg)
Vision in space
Vision systems (JPL) used for several tasks• Panorama stitching
• 3D terrain modeling
• Obstacle detection, position tracking
• For more, read “Computer Vision on Mars” by Matthies et al.
NASA'S Mars Exploration Rover Spirit captured this westward view from atop
a low plateau where Spirit spent the closing months of 2007.
JH
![Page 35: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/35.jpg)
Mobile robots
http://www.robocup.org/NASA’s Mars Spirit Rover
http://en.wikipedia.org/wiki/Spirit_rover
Saxena et al. 2008
STAIR at StanfordJH
![Page 36: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/36.jpg)
Augmented Reality and Virtual Reality
MS HoloLens, Oculus, Magic Leap,
![Page 37: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/37.jpg)
Jitendra Malik, UC Berkeley
Three ‘R’s of Computer Vision
“[Further progress in] the classic problems of computational vision:
reconstruction
recognition
(re)organization
[requires us to study the interaction among these processes].”
![Page 38: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/38.jpg)
State of the art today?
With enough training data, computer vision nearly
matches human vision at most recognition tasks
Deep learning has been an enormous disruption to
the field. Many techniques are being “deepified”.
JH
![Page 39: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/39.jpg)
![Page 40: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/40.jpg)
![Page 41: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/41.jpg)
![Page 42: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/42.jpg)
![Page 43: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/43.jpg)
![Page 44: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/44.jpg)
![Page 45: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/45.jpg)
![Page 46: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/46.jpg)
![Page 47: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/47.jpg)
Computer Vision and Nearby Fields
Derogatory summary of computer vision:
“Machine learning applied to visual data.”
• Computer Graphics: Models to Images
• Computer Vision: Images to Models
JH
![Page 48: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/48.jpg)
Scope of CSCI 1430
Computer Vision Robotics
Neuroscience
Graphics
Computational Photography
Machine Learning
Medical Imaging
Human Computer Interaction
Optics
Image ProcessingGeometric Reasoning
RecognitionDeep Learning
JH
![Page 49: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/49.jpg)
COURSE STAFF
![Page 50: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/50.jpg)
I work in here.
![Page 51: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/51.jpg)
![Page 52: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/52.jpg)
![Page 53: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/53.jpg)
![Page 54: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/54.jpg)
![Page 55: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/55.jpg)
Instructor: James Tompkin
Max Planck InstituteGermany
University College LondonUK
![Page 56: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/56.jpg)
[email protected]: CIT 547
Render pixels?Capture pixels?Interact with pixels?
Watch my research overview video!
I am probably interested.
![Page 57: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/57.jpg)
TA introduction
Eric XiaoDaniel NurieliEleanor TursmanMartin Zhu
Jackson (Jack) Gibbons
![Page 58: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/58.jpg)
Prerequisites
• Linear algebra, basic calculus, and probability
• Programming, data structures
CS 143 – James Hays
• Continuing his course – many materials, the courseworks, from him + previous staff – serious thanks!
• If you see ‘JH’ on a slide, then it is one of his.
![Page 59: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/59.jpg)
Contact
• Course runs quiet hours – 9pm to 9am.– No contact! We will ignore you.
• Piazza first– TAs are not on there all the time; set hours.
• [email protected] second
• TA office hours: Moon Lab 227, Tues-Thurs 7-9pm
• James: Tues 1-2pm
![Page 61: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/61.jpg)
Textbook
![Page 62: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/62.jpg)
Projects / Grading
• 100% programming projects (6 total)
Project 0 (do this immediately)• Install MATLAB• Go Help, go through ‘Getting Started with
MATLAB’ tutorial. Familiarize.• Go through second tutorial (image operands)• Come to us in office hours (next week) if you have
trouble.
![Page 63: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/63.jpg)
Proj 1: Image Filtering and Hybrid Images
• Implement image filtering to separate high and low frequencies.
• Combine high frequencies and low frequencies from different images to create a scale-dependent image.
JH
![Page 64: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/64.jpg)
Proj 2: Local Feature Matching
• Implement interest point detector, SIFT-like local feature descriptor, and simple matching algorithm.
JH
![Page 65: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/65.jpg)
Proj 3: Multi-view Geometry
• Recover camera calibration from feature point matches.
• Foundation for almost all measurement in computer vision.
JH
![Page 66: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/66.jpg)
Proj 4: Scene Recognition with Bag of Words
• Quantize local features into a “vocabulary”, describe images as histograms of “visual words”, train classifiers to recognize scenes based on these histograms.
JH
![Page 67: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/67.jpg)
Proj 5: Object Detection with a Sliding Window
• Train a face detector based on positive examples and “mined” hard negatives, detect faces at multiple scales and suppress duplicate detections.
JH
![Page 68: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/68.jpg)
Proj 6: Convolutional Neural Net
• Proj 4 again, but state of the art.
JH
![Page 69: Have you ever used computer vision? - Brown …cs.brown.edu/courses/cs143/2017_Spring/lectures_Spring...Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8dae968d07d976b17b34ac/html5/thumbnails/69.jpg)
Any questions?!
• Friday: Light and Color