cs325artificialintelligence ch18a–introtomachinelearning · overview data-driven...
TRANSCRIPT
![Page 1: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/1.jpg)
CS325 Artificial IntelligenceCh 18a – Intro to Machine Learning
Cengiz Günay
Spring 2013
Günay Ch 18a – Intro to Machine Learning
![Page 2: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/2.jpg)
Overview
Data-drivenLots of data (financial, internet, biology, etc.)?But also problems too difficult reflexive reasoning
Machine learning is inspired by the brain and neurons
Types of machine learning:supervised (this and next class)unsupervised (next week)reinforcement (later)
Günay Ch 18a – Intro to Machine Learning
![Page 3: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/3.jpg)
Overview
Data-drivenLots of data (financial, internet, biology, etc.)?But also problems too difficult reflexive reasoning
Machine learning is inspired by the brain and neurons
Types of machine learning:supervised (this and next class)unsupervised (next week)reinforcement (later)
Günay Ch 18a – Intro to Machine Learning
![Page 4: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/4.jpg)
Overview
Data-drivenLots of data (financial, internet, biology, etc.)?But also problems too difficult reflexive reasoning
Machine learning is inspired by the brain and neurons
Types of machine learning:supervised (this and next class)unsupervised (next week)reinforcement (later)
Günay Ch 18a – Intro to Machine Learning
![Page 5: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/5.jpg)
Overview
Data-drivenLots of data (financial, internet, biology, etc.)?But also problems too difficult reflexive reasoning
Machine learning is inspired by the brain and neurons
Types of machine learning:supervised (this and next class)unsupervised (next week)reinforcement (later)
Günay Ch 18a – Intro to Machine Learning
![Page 6: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/6.jpg)
Who uses Machine Learning (ML)?
Compared to Bayes Nets:
Bayes Nets require full knowledgeML is data-driven. How about sampling Bayes Nets?
Companies use ML for?
Product recommendations: Amazon, Netflix (had an MLcontest)Typing: Swype keyboard, learning word suggestionsPattern recognition: handwriting, OCR, audioWeb mining: Google page rank algorithm
Günay Ch 18a – Intro to Machine Learning
![Page 7: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/7.jpg)
Who uses Machine Learning (ML)?
Compared to Bayes Nets:
Bayes Nets require full knowledgeML is data-driven. How about sampling Bayes Nets?
Companies use ML for?
Product recommendations: Amazon, Netflix (had an MLcontest)Typing: Swype keyboard, learning word suggestionsPattern recognition: handwriting, OCR, audioWeb mining: Google page rank algorithm
Günay Ch 18a – Intro to Machine Learning
![Page 8: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/8.jpg)
Who uses Machine Learning (ML)?
Compared to Bayes Nets:
Bayes Nets require full knowledgeML is data-driven. How about sampling Bayes Nets?
Companies use ML for?
Product recommendations: Amazon, Netflix (had an MLcontest)Typing: Swype keyboard, learning word suggestionsPattern recognition: handwriting, OCR, audioWeb mining: Google page rank algorithm
Günay Ch 18a – Intro to Machine Learning
![Page 9: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/9.jpg)
![Page 10: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/10.jpg)
![Page 11: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/11.jpg)
Stanford’s Stanley
Günay Ch 18a – Intro to Machine Learning
![Page 12: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/12.jpg)
ML Taxonomy
Learn what?
parameters, structure, hidden conceptsHow? from labels (supervised), from nature of data
(unsupervised), from environmental feedback(reinforcement)
Why? future prediction (stock market), diagnosis (BayesNets), summarize (Google search), classify (digitrecognition)
Types: passive/active, online/offlineBased on output: classification vs. regressionBased on behavior: generative vs. discriminative
Günay Ch 18a – Intro to Machine Learning
![Page 13: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/13.jpg)
ML Taxonomy
Learn what? parameters, structure, hidden concepts
How? from labels (supervised), from nature of data(unsupervised), from environmental feedback(reinforcement)
Why? future prediction (stock market), diagnosis (BayesNets), summarize (Google search), classify (digitrecognition)
Types: passive/active, online/offlineBased on output: classification vs. regressionBased on behavior: generative vs. discriminative
Günay Ch 18a – Intro to Machine Learning
![Page 14: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/14.jpg)
ML Taxonomy
Learn what? parameters, structure, hidden conceptsHow?
from labels (supervised), from nature of data(unsupervised), from environmental feedback(reinforcement)
Why? future prediction (stock market), diagnosis (BayesNets), summarize (Google search), classify (digitrecognition)
Types: passive/active, online/offlineBased on output: classification vs. regressionBased on behavior: generative vs. discriminative
Günay Ch 18a – Intro to Machine Learning
![Page 15: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/15.jpg)
ML Taxonomy
Learn what? parameters, structure, hidden conceptsHow? from labels (supervised), from nature of data
(unsupervised), from environmental feedback(reinforcement)
Why? future prediction (stock market), diagnosis (BayesNets), summarize (Google search), classify (digitrecognition)
Types: passive/active, online/offlineBased on output: classification vs. regressionBased on behavior: generative vs. discriminative
Günay Ch 18a – Intro to Machine Learning
![Page 16: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/16.jpg)
ML Taxonomy
Learn what? parameters, structure, hidden conceptsHow? from labels (supervised), from nature of data
(unsupervised), from environmental feedback(reinforcement)
Why?
future prediction (stock market), diagnosis (BayesNets), summarize (Google search), classify (digitrecognition)
Types: passive/active, online/offlineBased on output: classification vs. regressionBased on behavior: generative vs. discriminative
Günay Ch 18a – Intro to Machine Learning
![Page 17: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/17.jpg)
ML Taxonomy
Learn what? parameters, structure, hidden conceptsHow? from labels (supervised), from nature of data
(unsupervised), from environmental feedback(reinforcement)
Why? future prediction (stock market), diagnosis (BayesNets), summarize (Google search), classify (digitrecognition)
Types: passive/active, online/offlineBased on output: classification vs. regressionBased on behavior: generative vs. discriminative
Günay Ch 18a – Intro to Machine Learning
![Page 18: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/18.jpg)
ML Taxonomy
Learn what? parameters, structure, hidden conceptsHow? from labels (supervised), from nature of data
(unsupervised), from environmental feedback(reinforcement)
Why? future prediction (stock market), diagnosis (BayesNets), summarize (Google search), classify (digitrecognition)
Types: passive/active, online/offline
Based on output: classification vs. regressionBased on behavior: generative vs. discriminative
Günay Ch 18a – Intro to Machine Learning
![Page 19: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/19.jpg)
ML Taxonomy
Learn what? parameters, structure, hidden conceptsHow? from labels (supervised), from nature of data
(unsupervised), from environmental feedback(reinforcement)
Why? future prediction (stock market), diagnosis (BayesNets), summarize (Google search), classify (digitrecognition)
Types: passive/active, online/offlineBased on output: classification vs. regression
Based on behavior: generative vs. discriminative
Günay Ch 18a – Intro to Machine Learning
![Page 20: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/20.jpg)
ML Taxonomy
Learn what? parameters, structure, hidden conceptsHow? from labels (supervised), from nature of data
(unsupervised), from environmental feedback(reinforcement)
Why? future prediction (stock market), diagnosis (BayesNets), summarize (Google search), classify (digitrecognition)
Types: passive/active, online/offlineBased on output: classification vs. regressionBased on behavior: generative vs. discriminative
Günay Ch 18a – Intro to Machine Learning
![Page 21: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/21.jpg)
Where did it all start?
Inputs on dendrites, outputs from axon
the perceptron
Günay Ch 18a – Intro to Machine Learning
![Page 22: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/22.jpg)
Where did it all start?
Inputs on dendrites, outputs from axon
the perceptronGünay Ch 18a – Intro to Machine Learning
![Page 23: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/23.jpg)
Let’s use the perceptron
Separate Tom from Jerry based on car preference?
Tom JerryTrucks 1 0Sedans 0 1Hybrids 0 1SUVs 1 0
×
W1W2W3W4
⇒∑i IiWi =?
Günay Ch 18a – Intro to Machine Learning
![Page 24: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/24.jpg)
Let’s use the perceptron
Separate Tom from Jerry based on car preference?
Tom JerryTrucks 1 0Sedans 0 1Hybrids 0 1SUVs 1 0
×
W1W2W3W4
⇒∑
i IiWi =?
Günay Ch 18a – Intro to Machine Learning
![Page 25: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/25.jpg)
Let’s use the perceptron
Separate Tom from Jerry based on car preference?
Tom JerryTrucks 1 0Sedans 0 1Hybrids 0 1SUVs 1 0
×
W1W2W3W4
⇒∑i IiWi =?
Günay Ch 18a – Intro to Machine Learning
![Page 26: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/26.jpg)
Let’s use the perceptron
Separate Tom from Jerry based on car preference?
Tom JerryTrucks 1 0Sedans 0 1Hybrids 0 1SUVs 1 0
×
W1W2W3W4
⇒∑i IiWi =?
Günay Ch 18a – Intro to Machine Learning
![Page 27: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/27.jpg)
In general. . .
ML learns mapping between features and labels:
x1 . . . xn → yn
Can be applied to different problems as long as can bevectorized (e.g., images)Need multiple examples (or samples)Question is to find function for each sample, m:
f (Xm) = Ym
Günay Ch 18a – Intro to Machine Learning
![Page 28: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/28.jpg)
Problem with choosing type of function: complexity
Occam’s razor:
prefer simplest solution
Günay Ch 18a – Intro to Machine Learning
![Page 29: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/29.jpg)
Problem with choosing type of function: complexity
Occam’s razor: prefer simplest solution
x
f(x)
Günay Ch 18a – Intro to Machine Learning
![Page 30: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/30.jpg)
Problem with choosing type of function: complexity
Occam’s razor: prefer simplest solution
x
f(x)
Günay Ch 18a – Intro to Machine Learning
![Page 31: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/31.jpg)
Problem with choosing type of function: complexity
Occam’s razor: prefer simplest solution
x
f(x)
Günay Ch 18a – Intro to Machine Learning
![Page 32: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/32.jpg)
Problem with choosing type of function: complexity
Occam’s razor: prefer simplest solution
x
f(x)
Günay Ch 18a – Intro to Machine Learning
![Page 33: CS325ArtificialIntelligence Ch18a–IntrotoMachineLearning · Overview Data-driven Lotsofdata(financial,internet,biology,etc.)? Butalsoproblemstoodifficultreflexivereasoning Machinelearningisinspiredbythebrainandneurons](https://reader036.vdocuments.us/reader036/viewer/2022080718/5f785330de33e6368e6e296f/html5/thumbnails/33.jpg)
Problem with choosing type of function: complexity
Occam’s razor: prefer simplest solution
x
f(x)
Günay Ch 18a – Intro to Machine Learning