cs376 computer vision lecture 18: introduction to visual ...huangqx/cs376_lecture_18.pdf · cs376...
TRANSCRIPT
![Page 1: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/1.jpg)
CS376 Computer Vision Lecture 18: Introduction to Visual
Recognition
Qixing Huang
April 3th 2019
![Page 2: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/2.jpg)
Topics covered/to be covered
• Features & filters
• Grouping & fitting
• Multiple views
• Recognition
![Page 3: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/3.jpg)
Features and filters
Transforming and describing images; Textures, colors, edges
Building blocks for neural networks
![Page 4: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/4.jpg)
Grouping & fitting
Clustering, Segmentation, fitting; what parts belong together?
Shi et al.
![Page 5: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/5.jpg)
Multiple Views
Invariant features, matching Epipolar geometry Structure-from-motion, stereo
![Page 6: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/6.jpg)
Recognition and learning
Data representation (vectorized) -> machine learning techniques
![Page 7: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/7.jpg)
Dataset
Representation
ML Algorithm
![Page 8: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/8.jpg)
Progress charted by datasets
![Page 9: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/9.jpg)
Progress charted by datasets
![Page 10: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/10.jpg)
Progress charted by datasets
![Page 11: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/11.jpg)
Progress charted by datasets
![Page 12: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/12.jpg)
Data Representations
![Page 13: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/13.jpg)
Deformable-Part-Model (Felzenszwalb et al. 10)
Pictorial Structures (Fischler et al. 73)
1973 2010
![Page 14: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/14.jpg)
Deformable-Part-Model (Felzenszwalb et al. 10)
Pictorial Structures (Fischler et al. 73)
1973 2010
![Page 15: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/15.jpg)
SIFT (Lowe 04)
1973 2010 2004
![Page 16: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/16.jpg)
HOG (Dalal and Triggs 05) GIST (Oliva and Torralba 01)
1973 2010 2004 2001 2005
![Page 17: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/17.jpg)
AlexNet (Krizhevsky et al. 12)
1973 2010 2004 2001 2005 2012
![Page 18: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/18.jpg)
VGG19 (Simonyan and Zisserman 14)
1973 2010 2004 2001 2005 2012 2014
![Page 19: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/19.jpg)
ResNet (He et al. 16)
1973 2010 2004 2001 2005 2012 2014 2016
![Page 20: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/20.jpg)
PointNet (R. Qi and Su et al. 17)
1973 2010 2004 2001 2005 2012 2014 2016 2017
![Page 21: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/21.jpg)
Machine Learning Algorithms
![Page 22: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/22.jpg)
Normalized Cut (Shi and Malik 97)
1997
![Page 23: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/23.jpg)
Graph Cut (Boykov et al. 99)
1997 1999
![Page 24: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/24.jpg)
AdaBoosting for face detection (Viola and Jones 04)
TextonBoost for segmentation (Shotton et al. 06)
1997 1999 2004 2006
![Page 25: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/25.jpg)
Support Vector Machine in Deformable Part Model (Felzenszwalb et al. 10)
1997 1999 2004 2006 2010
![Page 26: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/26.jpg)
Back-propagation in neural network training/implementation (Rumelhart et al. 86, LeCun et al. 98, Abadi et al. 16)
1997 1999 2004 2006 2010 2012
![Page 27: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/27.jpg)
Adam: A method for stochastic optimization (Kingma and Ba 14)
1997 1999 2004 2006 2010 2012 2014
![Page 28: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/28.jpg)
Topics to be Covered
![Page 29: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/29.jpg)
Machine Learning Basics
• Unsupervised Learning
– K-means
– K-nearest
– Graph cut (Mincut, Normalized Cut)
• Supervised Learning
– SVM
– Random forests
– Boosting
![Page 30: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/30.jpg)
Machine Learning Basics
• Convert data in their vectorized forms
What we have leaned in class?
![Page 31: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/31.jpg)
Deep Learning Basics
• Convolution layers/Fully connection layers/Max pooling/RELU
• Stochastic gradient descent/Dropout/ADAM
![Page 32: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/32.jpg)
Image Classification
• K-nearest neighbor classification
• SVM classification
• Boosting
• AlexNet
![Page 33: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/33.jpg)
Semantic Segmentation
• Texton boosting [Shotton et al. 07]
– MRF Formulation
• Fully connected neural networks
– Conv + Deconv [Noh et al. 15]
![Page 34: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/34.jpg)
Object Detection
• Deformable part model [Felzenszwalb et al. 10]
• Region CNN and variants [Girshick et al. 14]
• Regression-based techniques [Law and Deng 18]
![Page 35: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/35.jpg)
Other Topics
• Human pose estimation
• Monocular reconstruction
• 3D understanding
![Page 36: CS376 Computer Vision Lecture 18: Introduction to Visual ...huangqx/CS376_Lecture_18.pdf · CS376 Computer Vision Lecture 18: Introduction to Visual Recognition Qixing Huang thApril](https://reader031.vdocuments.us/reader031/viewer/2022041102/5edc2b90ad6a402d6666b8e3/html5/thumbnails/36.jpg)
Announcement
• Last lecture is the final exam
• Last assignment is due later