© 2018 Kiwisoft, all rights reserved

Aurélien Géron

May 23

rd

, 2018

Strata Data Conference, London

Deep Computer Vision for Manufacturing

© 2018 Kiwisoft, all rights reserved

© 2018 Kiwisoft, all rights reserved

● Machine Vision

● Introduction to Convolutional Neural Networks

● A Tour of Deep Computer Vision

● Challenges in Manufacturing

Agenda

© 2018 Kiwisoft, all rights reserved

Machine Vision

© 2018 Kiwisoft, all rights reserved

Machine Vision > Vision System

© 2018 Kiwisoft, all rights reserved

Machine Vision > 3D-Laser

© 2018 Kiwisoft, all rights reserved

● Sorting

● Inspection

● Analytics

● Robot Guidance

● Security

● More!

Machine Vision > Applications

© 2018 Kiwisoft, all rights reserved

● Sorting

● Inspection

● Analytics

● Robot Guidance

● Security

● More!

Machine Vision > Applications

© 2018 Kiwisoft, all rights reserved

● Sorting

● Inspection

● Analytics

● Robot Guidance

● Security

● More!

Machine Vision > Applications

© 2018 Kiwisoft, all rights reserved

● Sorting

● Inspection

● Analytics

● Robot Guidance

● Security

● More!

Machine Vision > Applications

© 2018 Kiwisoft, all rights reserved

● Sorting

● Inspection

● Analytics

● Robot Guidance

● Security

● More!

Machine Vision > Applications

© 2018 Kiwisoft, all rights reserved

● Sorting

● Inspection

● Analytics

● Robot Guidance

● Security

● More!

Machine Vision > Applications

© 2018 Kiwisoft, all rights reserved

Machine Vision > Classical Algorithms

© 2018 Kiwisoft, all rights reserved

Machine Vision > Edge Detection

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters-1

-2

-1

0

0

0

1

2

1

Convolution (cross-correlation)

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters > Zero Padding-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters > Zero Padding-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters > Zero Padding-1

-2

-1

0

0

0

1

2

1

© 2018 Kiwisoft, all rights reserved

Machine Vision > Filters

© 2018 Kiwisoft, all rights reserved

Machine Vision > Blob Discovery

© 2018 Kiwisoft, all rights reserved

Machine Vision > Blob Discovery

© 2018 Kiwisoft, all rights reserved

Machine Vision > Blob Discovery

© 2018 Kiwisoft, all rights reserved

Machine Vision > Blob Discovery

© 2018 Kiwisoft, all rights reserved

Introduction to ConvNets

© 2018 Kiwisoft, all rights reserved

Visual Cortex (V2)

CNNs > Visual Cortex

© 2018 Kiwisoft, all rights reserved

CNNs > Visual Cortex > V1, V2...

© 2018 Kiwisoft, all rights reserved

CNNs > Visual Cortex > Receptive Fields

© 2018 Kiwisoft, all rights reserved

CNNs > Convolutional Layers

© 2018 Kiwisoft, all rights reserved

CNNs > Receptive Field Size & Padding

© 2018 Kiwisoft, all rights reserved

CNNs > Stride

© 2018 Kiwisoft, all rights reserved

CNNs > Color Channels

© 2018 Kiwisoft, all rights reserved

CNNs > Color Channels

© 2018 Kiwisoft, all rights reserved

CNNs > Satellite ImagesDiffusion MRI

© 2018 Kiwisoft, all rights reserved

CNNs > Medical ImagesDiffusion MRI

© 2018 Kiwisoft, all rights reserved

CNNs > Convolutional Layers

© 2018 Kiwisoft, all rights reserved

CNNs > Pooling Layers

© 2018 Kiwisoft, all rights reserved

CNNs > ConvNet Architecture

© 2018 Kiwisoft, all rights reserved

CNNs > ConvNet Architecture

Tree

© 2018 Kiwisoft, all rights reserved

CNNs > ConvNet Architecture

Tree

Temple

© 2018 Kiwisoft, all rights reserved

CNNs > ConvNet Architecture

Tree

Temple

© 2018 Kiwisoft, all rights reserved

A Tour of Deep Computer Vision

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Classification

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > ClassificationSheep

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > ResNet Architecture

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Classification with LocalizationSheep

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > OverFeatFully Connected

4 units

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Object Detection

Sheep 1

Sheep 2 Sheep 3

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

Sheep 1

Sheep 2 Sheep 3

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > YOLO

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Semantic Segmentation

Sheep

Grass

Road

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Fully Convolution Network

Source: “Fully Convolutional Networks for Semantic Segmentation”, by Jonathan Long, Evan Shelhamer, Trevor Darrell, https://arxiv.org/abs/1411.4038 (Figure 1).

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Instance SegmentationGrass

Road

Sheep1 Sheep

2

Sheep3

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Masked R-CNN

Source: “Mask R-CNN”, by Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick, https://arxiv.org/abs/1703.06870 (Figure 1).

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Masked R-CNN

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Anomaly Detection

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Anomaly Detection

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Anomaly Detection

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Anomaly Detection

One Class SVM

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Anomaly Detection

One Class SVM

© 2018 Kiwisoft, all rights reserved

Deep Computer Vision > Anomaly Detection

One Class SVM

© 2018 Kiwisoft, all rights reserved

Challenges

© 2018 Kiwisoft, all rights reserved

Challenges > Pushback

© 2018 Kiwisoft, all rights reserved

Challenges > Pushback

© 2018 Kiwisoft, all rights reserved

Challenges > Hardware

© 2018 Kiwisoft, all rights reserved

Challenges > Framing the Problem

© 2018 Kiwisoft, all rights reserved

Challenges > Framing the Problem

© 2018 Kiwisoft, all rights reserved

Challenges > Framing the Problem

© 2018 Kiwisoft, all rights reserved

Challenges > Framing the Problem

© 2018 Kiwisoft, all rights reserved

Challenges > Framing the Problem

© 2018 Kiwisoft, all rights reserved

Challenges > Training Set

© 2018 Kiwisoft, all rights reserved

Challenges > Training Set

© 2018 Kiwisoft, all rights reserved

Challenges > Model Rot

Precision

Time

© 2018 Kiwisoft, all rights reserved

Challenges > High Volume, Low Latency

© 2018 Kiwisoft, all rights reserved

Challenges > Model Interpretation

© 2018 Kiwisoft, all rights reserved

Challenges > Model Interpretation

© 2018 Kiwisoft, all rights reserved

Challenges > Model Interpretation

© 2018 Kiwisoft, all rights reserved

© 2018 Kiwisoft, all rights reserved

Questions?

© 2018 Kiwisoft, all rights reserved

Media Credits

© 2018 Kiwisoft, all rights reserved

● Slide 1: public domain● Slide 2: https://youtu.be/qrzQ_AB1DZk● Slide 5: http://www.hte.net/solutions/machine-vision/smart-vision-cameras (reproduced with the kind authorization of HTE)● Slide 6: https://en.wikipedia.org/wiki/Glossary_of_machine_vision#/media/File:3D-Laserscanner_on_tripod.jpg (CC BY-SA 3.0, by user Dr.

Schorsch) ● Slide 6: https://en.wikipedia.org/wiki/Glossary_of_machine_vision#/media/File:Engine_movingparts.jpg● (CC BY-SA 3.0, by user Wapcaplet)● Slides 7, 8, 9: https://upload.wikimedia.org/wikipedia/commons/9/9e/Stemilt%27s_SweeTango_croo.jpg (CC-BY-2.0, by Stemilt Growers)● Slide 10: https://en.wikipedia.org/wiki/Industrial_robot#/media/File:Automation_of_foundry_with_robot.jpg (public domain)● Slides 11, 12: https://pixabay.com/en/metal-press-industrial-equipment-853694/ (CC0 Creative Commons)● Slide 13: Source: https://en.wikipedia.org/wiki/File:Valve_original_%281%29.PNG (CC BY-SA 3.0, by Simpsons contributor)● Slide 14: https://en.wikipedia.org/wiki/Canny_edge_detector#/media/File:Valve_monochrome_canny_(6).PNG (CC BY-SA 3.0, by Simpsons

contributor)● Slides 15-34 and 36-39: https://en.wikipedia.org/wiki/Connected-component_labeling#/media/File:

Two-pass_connected_component_labeling.svg (CC BY-SA 4.0, assumed by user Wereon)● Slide 41: https://commons.wikimedia.org/wiki/Brodmann_areas (CC BY-SA 2.1 jp, by Database Center for Life Science(DBCLS))● Slide 42: https://commons.wikimedia.org/wiki/File:Brain_circuits_for_visually_guided_saccades.jpg (CC-BY-4.0, by user Robert H. Wurtz)● Slide 49: https://www.flickr.com/photos/gsfc/4591847715 (public domain)● Slide 50: https://en.wikipedia.org/wiki/Medical_image_computing#/media/File:DiffusionMRI_glyphs.png (CC BY-SA 3.0, by user Tucania)● Slide 84: https://youtu.be/MPU2HistivI● Slide 89: https://youtu.be/OOT3UIXZztE● Slide 90: https://pixabay.com/en/wolf-in-sheep-s-clothing-wolf-sheep-2577813/ (CC0 Creative Commons)● Slide 97: https://en.wikipedia.org/wiki/Luddite#/media/File:FrameBreaking-1812.jpg (public domain)● Slide 98: https://pixabay.com/en/atm-withdraw-cash-map-ec-card-1524870/ (CC0 Creative Commons)● Slide 99: https://commons.wikimedia.org/wiki/File:Inspection_system.jpg (CC BY-SA 4.0, by user Yingz)● Slide 108: https://commons.wikimedia.org/wiki/File:Millville,_New_Jersey_-_Glass_bottles._(Numerous_bottles_in_foreground_and_bottles

_on_a_conveyor_belt.)_-_NARA_-_518673.jpg (public domain)

Credits