binary classification for application in intelligent 3d …...•final set: 2,589 images spread...
TRANSCRIPT
![Page 1: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/1.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Binary Classification of Images forApplications in Intelligent 3D
ScanningBranislav Vezilid, Dušan Gajid, Dinu Dragan, Veljko Petrovid, Srđan Mihid,
Zoran Anišid, Vladimir Puhalac
11th International Symposium on Intelligent Distributed Computing, Belgrade, Serbia, October 11-13, 2017
1
![Page 2: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/2.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Outline
1. Introduction
2. Methodologya) Overview of the Research Approach
b) Data Set
c) Feature Extraction
d) Data Visualization (t-SNE)
e) Model Set-up
f) Evaluation
3. Results
4. Conclusion
2
![Page 3: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/3.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Introduction
• 3D scanning process
3
![Page 4: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/4.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Introduction
MISFIRED IMAGE NORMAL IMAGE
4
![Page 5: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/5.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Introduction
• Our goal is to create a system that is able to detect images with darker areas
• We differentiate two classes: Normal images
Misfired images (as we like to call them)
• Our approach: Manually extract features from images
Use them as input to selected machine learning algorithms
5
![Page 6: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/6.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Overview of the Research Approach
6
Model
Model set-up
Standardize data
Set up training and model parameters
Training
Data set
Feature extraction
Convert image to HSV and extract value channel
Crop left and right side of image
Calculate histograms
Evaluation
![Page 7: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/7.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Data Set
• Data set consists of 26,027 images with resolution of 5184 x 3456 in JPEG format
• All images are resized to resolution 400 x 266
• Only 2,729 out of 26,027 images (10.49%) are marked as misfired
• Balanced data set is produced by randomly sampling normal images from remaining data
• Balanced data set = equal number of samples per class
• Additional manual removal of outliers
• FINAL SET: 2,589 images spread across both classes
7
![Page 8: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/8.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Feature Extraction
8
Convert image to HSV and extract value channel
Calculate histograms
Crop left and right side of image
![Page 9: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/9.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Feature Extraction
9
Convert image to HSV and extract value channel
Crop left and right side of image
Calculate histograms
Color Grayscale
![Page 10: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/10.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Feature Extraction
10
Convert image to HSV and extract value channel
Crop left and right side of image
Calculate histograms
Color Grayscale
Analysed 4 different crop ratios:1. 5%2. 12.5%3. 25%4. 40%
![Page 11: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/11.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Feature Extraction
11
Convert image to HSV and extract value channel
Crop left and right side of image
Calculate histograms
![Page 12: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/12.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Feature Extraction
MISFIRED IMAGE NORMAL IMAGE
12
![Page 13: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/13.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
t-SNE (t-distributed Stochastic Neighbor Embedding)
• Method for visualizing data set and distribution of classes among different features
• Cropped parts of image at different widths: 5%, 12.5%, 25%, 40%
13
![Page 14: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/14.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Model Set-up
•4 different machine learning methods:
k-Nearest Neighbors
Support Vector Machines
Random Forest
Artificial Neural Networks (Multi-layer perceptron)
14
![Page 15: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/15.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Model Set-up
15
Standardize data TrainingSet up training and model parameters
![Page 16: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/16.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Model Set-up
• Zero-centered by feature
• Scale by feature variance
16
Standardize data TrainingSet up training and model parameters
![Page 17: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/17.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Model Set-up
• k-Nearest Neighbors Parameters: k-number of neighbors
• Support Vector Machines Parameters: kernel, C, gamma
• Random Forest Parameters: criteria, number of trees, max depth, minimal samples for split
• Artificial Neural Network Parameters: network architecture, optimizer and learning rate, loss function
17
Standardize data TrainingSet up training and model parameters
![Page 18: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/18.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Model Set-up
• k-Nearest Neighbors Parameters: k-number of neighbors
• Support Vector Machines Parameters: kernel, C, gamma
• Random Forest Parameters: criteria, number of trees, max depth, minimal samples for split
• Artificial Neural Network Parameters: network architecture, optimizer and learning rate, loss function
18
Standardize data TrainingSet up training and model parameters
![Page 19: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/19.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Evaluation
19
![Page 20: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/20.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Results for k-Nearest Neighbors
• Parameter optimization was done using 11-fold cross validation
• Parameters: k – number of neighbors
Euclidean distance
Edge[%] 5 12.5 25 40
k –
ne
igh
bo
rs
593.69
(+/- 1.41)93.65
(+/- 1.29)92.71
(+/- 2.15)91.53
(+/- 1.95)
1093.17
(+/- 1.96)93.40
(+/- 2.02)92.73
(+/- 2.53)91.73
(+/- 1.69)
1593.52
(+/- 1.10)93.76
(+/- 1.75)92.71
(+/- 1.66)91.68
(+/- 2.20)
3592.91
(+/- 2.12)93.18
(+/- 2.14)92.76
(+/- 1.79)91.07
(+/- 2.07)
20
![Page 21: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/21.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Results for Support Vector Machines
• Parameter optimization was done using 11-fold cross validation
• Best parameters: kernel function: rbf
C = 10
Gamma = 0.001
Edge [%] 5 12.5 25 40
Accuracy [%]94.33
(+/- 1.47)94.42
(+/- 1.48)94.23
(+/- 1.76)93.53
(+/- 1.61)
21
![Page 22: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/22.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Results for Random Forest
• Parameter optimization was done using 11-fold cross validation
• Parameters: criteria, number of trees, max depth, min samples for split
Edge[%] 5 12.5 25 40
Criteria gini gini entropy gini
Number of trees
64 512 64 512
Max depth 512 512 512 60
Min samples for split
5 2 2 2
Accuracy [%]94.55
(+/- 1.38)94.38
(+/- 1.40)93.71
(+/- 1.82)93.51
(+/- 0.95)
22
![Page 23: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/23.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Results for ANN (Artificial Neural Network)
• Architecture same as classification block in well-known networks (VGG16, ResNet50, etc.)
• Dropout is increased to 0.75, from regular 0.5
• ReLU activation function in hidden layers
• Softmax function in output layer
Network architecture
23
![Page 24: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/24.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Results for ANN (Artificial Neural Network)
• SGD (Stohastic Gradient Descent) learning rate = 0.0001
Momentum = 0.9
Nesterov = True
Reduce learning rate by factor of 0.1, if value loss haven’t been improved for 2 consecutive epochs
• Training 100 epochs (~1.5h)
• Loss Categorical cross entropy
Edge [%] 5 12.5 25 40
Accuracy [%] 94.47 94.59 94.36 93.04
24
![Page 25: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/25.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Results for ANN (Artificial Neural Network)
• Training process
Edges: • 5% •12.5% •25% •40%
25
![Page 26: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/26.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Final Results
Comparison of proposed methods on test data
Edge [%] 5 12.5 25 40
RF 95.14 94.76 94.96 93.60
ANN94.47(-0.67)
94.59(-0.17)
94.36(-0.60)
93.04(-0.56)
SVM94.75(-0.39)
94.59(-0.17)
94.23(-0.73)
93.40(-0.20)
k-NN94.58(-0.56)
94.56(-0.20)
94.31(-0.65)
93.40(-0.20)
26
![Page 27: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/27.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Conclusion
• Proposed a method for detecting misfired images
• Based on histograms as global descriptors and machine learning algorithms
• All 4 considered algorithms showed very good and comparable performance
• Random Forest showed best performance with 95.14% accuracy on edge area set to 5%
• Possible improvements:• Add additional feature descriptors• Clear and expand the data set
• Future work: Final goal is development of complete quality control system for 3D scanning System is expected to detect: misfiring, empty scene, blur Reconstruction of missing 3D model body parts
28
![Page 28: Binary classification for application in intelligent 3D …...•FINAL SET: 2,589 images spread across both classes 7 13.10.2017. IDC 2017 Belgrade Binary Classification of Images](https://reader033.vdocuments.us/reader033/viewer/2022050315/5f7783c666b93510973f97de/html5/thumbnails/28.jpg)
13.10.2017. IDC 2017 Belgrade
Binary Classification of Images for Applications in Intelligent 3D Scanning
Binary Classification of Images forApplications in Intelligent 3D
ScanningBranislav Vezilid, Dušan Gajid, Dinu Dragan, Veljko Petrovid, Srđan Mihid,
Zoran Anišid, Vladimir Puhalac
11th International Symposium on Intelligent Distributed Computing, Belgrade, Serbia, October 11-13, 2017
29