towards open world recognition abhijit bendale, terrance boult university of colorado of colorado...
TRANSCRIPT
![Page 1: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/1.jpg)
Towards Open World Recognition
Abhijit Bendale, Terrance BoultUniversity of Colorado of Colorado Springs
Poster no 85
![Page 2: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/2.jpg)
2
Multi-Class Classification System
W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI 2013
![Page 3: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/3.jpg)
3
Out in the Real-WorldDetect New Category
Pool Table
Bowling Pin
Boxing glove
Calculator
Chess board
?
?
?
?
![Page 4: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/4.jpg)
4
Open World Recognition
• World with Knowns (K) &Unknowns Unknowns (UU)
Detect as Unknown
• NU: Novel Unknowns
Label Data• LU: Labeled
Unknowns
Incremental Class Learning
• K: Known
Scale
![Page 5: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/5.jpg)
5
Open Set Learning
Incremental Learning
Scalable Learning
Ristin+ (CVPR’14)
Yeh+ (CVPR’08)
Li+ CVPR’07)
Mensink+(PAMI’13)
Related Work
Jain+ (ECCV’14)
Scheirer+ PAMI’13)
Scheirer+(PAMI’14)
Deng+ (NIPS’11)
Marszalek+(ECCV’08)
Liu+ (CVPR’13)
Open World Recognition
![Page 6: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/6.jpg)
6
Open Space in Classification
??
?
?
?
?
? ? ??
??
??
Open Set Recognition Empirical risk
functionRegularization constant
Open space risk
W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI 2013
Closed SpaceOpen Space
![Page 7: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/7.jpg)
7
NCM – Metric Learning
NCM Classifier with Metric Learning
T Mensink, J Verbeek, F Perronin, G Csurka “Distance based Image Classification: Generalizing to New Classes at Near Zero Cost” IEEE TPAMI 2013M Ristin, M Guillaumin, J Gall, L Van Gool “Incremental Learning of NCM Forests for Large-Scale Image Classification” CVPR 2014
![Page 8: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/8.jpg)
8
Compact Abating Probability (CAP) Models
W Scheirer, L Jain, T Boult “Probability Models for Open Set Recognition” IEEE TPAMI 2014
Class Mean
![Page 9: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/9.jpg)
9
Theorem 1: Open Space Risk for Model Combination
![Page 10: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/10.jpg)
10
Theorem 2: Open Space Risk for Transformed Spaces
![Page 11: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/11.jpg)
11
Opening an Existing Algorithm:Nearest Non-Outlier (NNO) Algorithm
W = Linear Transformation (weight matrix from metric learning)
Standard gamma functionIn volume of m-D ball Class mean for class i
τ is threshold for open world
Probability
![Page 12: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/12.jpg)
12
Open World Evaluation
Training phase
Testing phase
Parameter Learning Phase Incremental Learning Phase
Closed Set TestingUnknown Categories
Known Categories
Open Set Testing
? ?
![Page 13: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/13.jpg)
13
Training for Open World
NCM - ML NNO
• Parameter Learning with initial set of categories
• Estimation of τ for open set learning to balance open space risk• Optimize for Known vs Unknown Errors• Incrementally add new categories
![Page 14: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/14.jpg)
14
ExperimentsDatasets
• ILSVRC’10: 1.2M training images, 1000 classes• ILSVRC’12: 1.2M training images, 1000 classes
Features• Dense SIFT features, Quantized into 1000 Bag of Visual Words• Publically available features• LBP, HOG, Dense SIFT (for ILSVRC’12)
Algorithms• Nearest Class Mean - ML Classifier (NCM) [Mensink etal PAMI 2013]• Nearest Non-Outlier Algorithm (NNO) [This Paper]• 1vSet [Scheirer etal PAMI 2013]• Linear SVM [Liblinear, Fan etal JMLR 2008]
![Page 15: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/15.jpg)
50 Initial Categories
Increasing # of unknown categories during testing i.e. increasing openness of problem
Incrementally addingcategories during training
Closed Settesting
![Page 16: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/16.jpg)
16
200 Initial Categories
Increasing # of unknown categories during testing i.e. increasing openness of problem
Incrementally addingcategories during training
Closed Settesting
500 known + 500 unknown categories
![Page 17: Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649efc5503460f94c0f782/html5/thumbnails/17.jpg)
17
• Formalized Open World Recognition and showed how to “Open” an existing algorithm.
• NNO allows construction of scalable systems that can be updated incrementally
Conclusion & Future Work
See us at Poster no 85 …!!!!
• Exploring sophisticated novelty detectors, open world detection, “opening” other baseline algorithms etc.
• Open World Deep Learning methods• Happy to Collaborate…!!!