machine learning and deep learning in...
TRANSCRIPT
![Page 1: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/1.jpg)
Machine Learning and Deep Learning in Manufacturing
03/14/2017
Prof. Seungchul Lee
1
![Page 2: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/2.jpg)
Introduction
• Since 2013 July: UNIST
– School of Mechanical Engineering
• 2010, Ph.D. from the University of Michigan, Ann Arbor
– S. M. Wu Manufacturing Research Center
– The Center of Intelligent Maintenance Systems (IMS)
• 2007, M.S. from the University of Michigan, Ann Arbor
• 2005, B.S. of Electrical Engineering from Seoul National University
• 2001, B.S. of Mechanical Engineering from Seoul National University
2
![Page 3: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/3.jpg)
Robot Playing Piano
3By iSystems
![Page 4: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/4.jpg)
How to Make Machine Intelligent ?
4
Artificial Intelligence
![Page 5: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/5.jpg)
Machine Learning for Machine Intelligence
5
Classification
Regression
Clustering Dim reduction
![Page 6: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/6.jpg)
Deep Learning for Machine Intelligence
6
CNN
RNN
![Page 7: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/7.jpg)
Outline
• Machine Learning– Supervised Learning
– Unsupervised Learning
• Deep Learning: CNN
• Dimension Reduction– Latent Space
• Deep Learning: RNN
• (Model-augmented) Bayesian Machine Learning
7
![Page 8: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/8.jpg)
Framework of Machine Learning
8
Sensor Data
Data Window
Features
Decision
Data acquisition and pre-processing
Windowing
Feature extraction
Model building and Classification (Inference)
![Page 9: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/9.jpg)
Implementation of Machine Learning
9
![Page 10: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/10.jpg)
Weather Station
![Page 11: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/11.jpg)
Weather Station: Visualization
temperature
humidity
brightness
![Page 12: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/12.jpg)
Data Science: (Unexpected) Hidden Information
Jul 31 10:02
Grad. Student came to lab.
Jul 30 23:26
Grad. Students
went home
Jul 31 05:27
Sunrise
Brightness Data
![Page 13: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/13.jpg)
Framework of Machine Learning
13
Sensor Data
Data Window
Features
Decision
Data acquisition and pre-processing
Windowing
Feature extraction
Model building and Classification (Inference)
![Page 14: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/14.jpg)
Outline
• Machine Learning– Supervised Learning
– Unsupervised Learning
• Deep Learning: CNN
• Dimension Reduction– Latent Space
• Deep Learning: RNN
• (Model-augmented) Bayesian Machine Learning
14
![Page 15: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/15.jpg)
Supervised Learning
• Data with labels
• Classification
• Example– Rotating machinery (power plant)
– Data-driven diagnostics
– Training data set
15
chair desk
…
![Page 16: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/16.jpg)
Web-based Monitoring Dashboard
16
Probability of Classification
normal
unbalance
misalignment
![Page 17: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/17.jpg)
Outline
• Machine Learning– Supervised Learning
– Unsupervised Learning
• Deep Learning: CNN
• Dimension Reduction– Latent Space
• Deep Learning: RNN
• (Model-augmented) Bayesian Machine Learning
17
![Page 18: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/18.jpg)
Unsupervised Learning
• No labels
• Representation
• Abstraction
• Clustering
18
![Page 19: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/19.jpg)
Serov Motor
• System configuration– Arduino
– Servo motor
• Load (anomaly) generation– Anomaly is induced through manual press
19
![Page 20: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/20.jpg)
Demo for Unsupervised Learning
20
Th
reshold
![Page 21: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/21.jpg)
Outline
• Machine Learning– Supervised Learning
– Unsupervised Learning
• Deep Learning: CNN
• Dimension Reduction– Latent Space
• Deep Learning: RNN
• (Model-augmented) Bayesian Machine Learning
21
![Page 22: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/22.jpg)
(Traditional) Machine Learning
22
![Page 23: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/23.jpg)
Neural Network
23
![Page 24: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/24.jpg)
Neural Network
24
![Page 25: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/25.jpg)
Deep Neural Network
25
![Page 26: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/26.jpg)
Convolutional Neural Networks (CNN)
• 이미지분류에서높은성능을보인 CNN 기법을진동신호기반결함진단에사용제안
26
Training Data Feature Extraction Classification
DiagnosticsDeep Learning
![Page 27: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/27.jpg)
CNN on STFT Image
• Time series as input in PHM
• 기본 CNN구조를활용하기위하여신호를이미지화 (STFT spectrogram)
27
STFT image
signal
![Page 28: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/28.jpg)
Outline
• Machine Learning– Supervised Learning
– Unsupervised Learning
• Deep Learning: CNN
• Dimension Reduction– Latent Space
• Deep Learning: RNN
• (Model-augmented) Bayesian Machine Learning
28
![Page 29: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/29.jpg)
Dimension Reduction
• Principal Component Analysis (PCA)– Dim reduction without losing too much information
29
u1
u2
1
1
2
xu
x
![Page 30: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/30.jpg)
Dimension Reduction
• PCA in time signals
30
![Page 31: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/31.jpg)
31
Provide compressed representations
Deep Learning: Autoencoder
![Page 32: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/32.jpg)
Outline
• Machine Learning– Supervised Learning
– Unsupervised Learning
• Deep Learning: CNN
• Dimension Reduction– Latent Space
• Deep Learning: RNN
• (Model-augmented) Bayesian Machine Learning
32
![Page 33: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/33.jpg)
Input Latent Variable Classification
• Latent Variable = Hidden State
• Hidden state is not directly visible, but
• Decision (target) depends on hidden state
• Time sequential data Recurrent Neural Network
33
Hidden
State h1
h2 h3 h4 h5 h6
![Page 34: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/34.jpg)
Deep Learning: RNN for Classification
• Deep learning structure for sequential data– Recurrent Neural Network (RNN)
– Information to be passed
34
diagnosis
W
…
…
…
State 3W
…
…
…
State 4
4o
Window 1
…
…
…
State 1W
Window 2
…
…
…
State 2Hidden
State
Input
Window 3 Window 4
![Page 35: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/35.jpg)
Any Problems So for ?
35
![Page 36: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/36.jpg)
Outline
• Machine Learning– Supervised Learning
– Unsupervised Learning
• Deep Learning: CNN
• Dimension Reduction– Latent Space
• Deep Learning: RNN
• (Model-augmented) Bayesian Machine Learning
36
![Page 37: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/37.jpg)
Motivation: Robotics Application
37
![Page 38: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/38.jpg)
Bayesian Estimation
• Dynamics (not considered yet)
• Intuition behind Bayesian Inference (Kalman filter)– Sequential measurements
– Noise
– True state?
– 예측가능
38
![Page 39: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/39.jpg)
Bayesian Inference: Model as A Prior
• Machine learning: Generative
• Assumption– Latent variables are independent
• Pattern matching problem– Snapshot data
– No sequential (historical) consideration
• May not fully utilize all information (Variable dynamics)
39
X
Y
X
Y
X
Y
Latent Variable
Observation
Measurement
![Page 40: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/40.jpg)
Tool Wear Estimation
• Hidden Markov Model
40
![Page 41: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/41.jpg)
Demo for Model-based FDI
• Residual
• Kalman filter
41
![Page 42: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/42.jpg)
Forecasting
• Prognostics
• The ultimate goal of PHM
• If models are good (?)– will provide more accurate RUL prediction
42
( ) ( ) ( )
( ) ( )
x t Ax t Bu t
y t Cx t
0.9 0.1
0 0.8
0 0
0 0 0
A
0 100 200 300 400 500 6001
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2
C/1
Ca
pa
city (
Ah
r)
Cycles
B0005
Monte-Carlo Simulation based on PEMs model
0 100 200 300 400 500 6001
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2
C/1
Ca
pa
city (
Ah
r)
Cycles
B0005
Monte-Carlo Simulation based on PEMs model
![Page 43: Machine Learning and Deep Learning in Manufacturingisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018-12-28 · Machine Learning and Deep Learning in Manufacturing](https://reader035.vdocuments.us/reader035/viewer/2022081513/5f07ec0c7e708231d41f6e4b/html5/thumbnails/43.jpg)
Data Science in Manufacturing
• 양품/불량검사– 현장전문가또는생산기술연구소엔지니어가경험과 domain
knowledge 를이용해측정신호로 rule 기반검사
• 설비상태– Long-term data
• Product data
• Manufacturing Service
43