data science in industry - applying machine learning to real-world challenges
TRANSCRIPT
![Page 1: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/1.jpg)
Data Science in Industry Applying Machine Learning to
Real-world Challenges
![Page 3: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/3.jpg)
obtained Ph.D. indata mining and machine learning
![Page 4: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/4.jpg)
worked in both academia and industry
![Page 5: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/5.jpg)
![Page 6: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/6.jpg)
Not just a researcher,but a coder & hacker
![Page 7: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/7.jpg)
What is data science?
![Page 8: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/8.jpg)
data is everywhere...
![Page 9: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/9.jpg)
data science helps
extract knowledge from data...
![Page 10: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/10.jpg)
Data scientists investigate complex data problems
![Page 11: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/11.jpg)
find and interpret rich data sources
![Page 12: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/12.jpg)
Visualize the data
![Page 13: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/13.jpg)
get insights from data
![Page 14: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/14.jpg)
from insights….
![Page 15: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/15.jpg)
Questions?
![Page 16: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/16.jpg)
![Page 17: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/17.jpg)
![Page 18: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/18.jpg)
Now is the fun part...
![Page 19: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/19.jpg)
Data Science techniques!
![Page 20: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/20.jpg)
Data science 101
● regression● classification● clustering● ranking (not covered in this lecture)● recommendation (not covered in this lecture)
![Page 21: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/21.jpg)
Regression
![Page 22: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/22.jpg)
What is regression?
![Page 23: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/23.jpg)
A bit formal definition….
models a functional relationship between
an input variable x and
a response variable y
![Page 24: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/24.jpg)
x
y
![Page 25: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/25.jpg)
![Page 26: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/26.jpg)
![Page 27: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/27.jpg)
find the equation
![Page 28: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/28.jpg)
What else can regression do?
![Page 29: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/29.jpg)
Predicting who may change jobs!
![Page 31: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/31.jpg)
x
y
Recap - regression
![Page 32: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/32.jpg)
classification
![Page 33: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/33.jpg)
identify to which of a set of categories a new data point belongs
![Page 34: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/34.jpg)
Spam or Not spam?
![Page 35: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/35.jpg)
Credit approve or not?
![Page 36: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/36.jpg)
Optical character recognition
![Page 37: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/37.jpg)
Document classification
![Page 38: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/38.jpg)
SVM
![Page 39: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/39.jpg)
Decision tree
![Page 40: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/40.jpg)
Use classification to...
find who you are in social networks
![Page 41: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/41.jpg)
![Page 42: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/42.jpg)
classification
![Page 43: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/43.jpg)
classification
![Page 44: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/44.jpg)
Missing data
Outdated data
Non-standard data
![Page 45: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/45.jpg)
Why we want to classify?
![Page 46: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/46.jpg)
Understanding users’ social roles is crucial to many
social network applications
![Page 47: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/47.jpg)
including advertising targeting,
marketing, personalization,
recommendation, etc.
![Page 48: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/48.jpg)
Finding out who you really are...
![Page 49: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/49.jpg)
![Page 50: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/50.jpg)
manually labeling is time-consuming
and error prone
![Page 51: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/51.jpg)
Human learning
Machine learning
![Page 52: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/52.jpg)
![Page 53: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/53.jpg)
SVM
![Page 54: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/54.jpg)
Decision tree
![Page 55: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/55.jpg)
How accurate can we get?
![Page 56: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/56.jpg)
![Page 57: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/57.jpg)
Can we further improve?
![Page 58: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/58.jpg)
Clustering
![Page 59: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/59.jpg)
grouping a set of data points
![Page 60: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/60.jpg)
data points in the same group ( cluster) are more similar to each other
than to those in other groups (clusters)
![Page 61: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/61.jpg)
![Page 62: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/62.jpg)
k-means clustering algorithm
![Page 63: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/63.jpg)
k clusters
![Page 64: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/64.jpg)
k = 3
![Page 65: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/65.jpg)
step 1:randomly select k points
as centroids
![Page 66: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/66.jpg)
3 random centroids
![Page 67: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/67.jpg)
step 2:assign every data point to
the nearest centroid
![Page 68: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/68.jpg)
![Page 69: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/69.jpg)
step 3:calculate mean of each cluster
as the new centroid
![Page 70: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/70.jpg)
![Page 71: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/71.jpg)
repeatassign clusters based on
the new centroids
![Page 72: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/72.jpg)
![Page 73: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/73.jpg)
How to use clustering to solve big data problem?
![Page 74: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/74.jpg)
Machine data is massive
![Page 75: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/75.jpg)
![Page 76: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/76.jpg)
1 Tb/day is normal
![Page 77: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/77.jpg)
no one has time to read all data...
![Page 78: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/78.jpg)
Clustering comes to rescue!
![Page 79: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/79.jpg)
![Page 80: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/80.jpg)
clustering algorithm summarizesbig data to a few groups
![Page 81: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/81.jpg)
each group representsa number of similar data points
![Page 82: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/82.jpg)
investigating data pointsone by one
![Page 83: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/83.jpg)
just investigating the clusters!
![Page 84: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/84.jpg)
Things to considerin practice...
![Page 85: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/85.jpg)
scalability
![Page 86: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/86.jpg)
velocity
![Page 87: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/87.jpg)
variety
![Page 88: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/88.jpg)
real-time
![Page 89: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/89.jpg)
What’s next?
![Page 90: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/90.jpg)
Recap
● regression
● classification
● clustering
![Page 91: Data Science in Industry - Applying Machine Learning to Real-world Challenges](https://reader030.vdocuments.us/reader030/viewer/2022032420/55a60dd21a28abd0418b45ae/html5/thumbnails/91.jpg)
This presentation was initially created for a guest lecture at Utah State University for teaching and education purposes.
Thanks!