introduction to machine learning anjeli singh computer science and software engineering april 28 th...
Post on 19-Dec-2015
217 views
TRANSCRIPT
![Page 1: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/1.jpg)
Introduction to Machine Learning
Anjeli SinghComputer Science and Software
Engineering
April 28th 2008
![Page 2: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/2.jpg)
Overview
• What is Machine Learning• Examples of Machine Learning
• Learning Associations• Classification• Regression• Unsupervised Learning• Reinforcement Learning• Notes
![Page 3: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/3.jpg)
What is Machine Learning?
• We store and process data• Supermarket chain – Hundred of stores – Selling thousands of good to million of customers– Record the details: date, customer ID, goods,
money– Gigabytes of data everyday– Turn this data into information to make prediction
![Page 4: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/4.jpg)
What is Machine Learning?
• Do we know which people are likely to buy a particular product?
• Which author to suggest to people which enjoy reading?
![Page 5: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/5.jpg)
What is Machine Learning?
Answer is : NO
![Page 6: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/6.jpg)
What is Machine Learning?
Answer is :NOBecause if we knew, we won’t need
any data analysis.Go ahead and write code
![Page 7: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/7.jpg)
What is Machine Learning?
• We can collect data • Try to extract answers to these similar questions• There is a process explains the data we observe• Its not completely random E.g., Customer behavior• People don’t buy random things• E.g. when they buy beer they buy chips • There are certain pattern in the data
![Page 8: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/8.jpg)
What is Machine Learning?
• We can’t identify process completely• We can construct good approximation• Detect certain pattern and regularities• This is the niche of Machine Learning• We can use these patterns for prediction• Assuming near future won’t be much different
the past
![Page 9: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/9.jpg)
What is Machine Learning?
• Application to large database: data mining• Its an analogy for extracting minerals from
Earth• Large volume of data is processed – To construct a simple model with valuable use– Having a high predictive accuracy– Application :• Retail, finance banks, manufacturing, medicine for
diagnosis
![Page 10: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/10.jpg)
What is Machine Learning?
• Its not a database problem• To be intelligent, a system that is in changing
environment should have the ability to learn• System should learn and adapt• Foresee and provide solution for all possible
situations
![Page 11: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/11.jpg)
Mathematics Problem
• 2 + 2=
![Page 12: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/12.jpg)
Mathematics
• 2 + 2= 4
![Page 13: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/13.jpg)
What is Machine Learning?
• How we did that ?• Can we write a program to add two
numbers ??
![Page 14: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/14.jpg)
Faces
• Who is He ??
![Page 15: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/15.jpg)
Faces
Denzel Washington
![Page 16: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/16.jpg)
What is Machine Learning?
• How do we acknowledge him ?• Can you write a program for that???
![Page 17: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/17.jpg)
What is Machine Learning?
• Pattern Recognition Problem• Analyzing sample face images of that person• Learning captures the pattern specific to the
person• Recognize by checking this person in a given
image
![Page 18: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/18.jpg)
What is Machine Learning?
“Machine learning is programming computers to optimize a performance criterion using example data or past experience”
• Uses theory of statistics to build mathematical models
• Core task is to make inferences from a sample
![Page 19: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/19.jpg)
What is Machine Learning?
• Role of Computer Science– Training, need efficient algorithms to solve
optimization problems– To store and process massive data– Its representation and algorithmic solutions for
inferences– Eg, the efficiency of learning and or inference
algorithm its space and time complexity may be as imp as predictive accuracy
![Page 20: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/20.jpg)
Examples of Machine Learning
• Learning Associations• Classification• Regression• Unsupervised Learning• Reinforcement Learning• Notes
![Page 21: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/21.jpg)
Learning Associations
• In case of retail: Basket Analysis• Finding associations between product bought
by customers• If a customer buy X, typically also buy Y• To find Potential Y customer• Target them from cross selling
![Page 22: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/22.jpg)
Learning Associations
• Association Rule– P(Y|X) where Y is the product we condition on X
and X is the product which a customer has already purchased
– Eg. P(chips|Beer) = 0.7 Then 70 % who buy beer also chips
– Distinction Attribute: • P(Y|X, D) D set of customer attributes• Gender, age, marital status
![Page 23: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/23.jpg)
Classification
• Credit card Example– Predict the chances of paying loan back– Customer will default /not pay the whole amount– Bank should get profit– Not inconvenience a customer over his financial
capacity
![Page 24: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/24.jpg)
Classification
• Credit Scoring– Calculate the risk given the amount and customer
information– Customer information. Eg., Income, savings,
profession, age, history etc.– Form a rule– Fits a Model to the past data– To calculate the risk for a new application
![Page 25: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/25.jpg)
Classification
• Classes– Low Risk Savings
– High Risk Ѳ2
• Rule (Prediction) Ѳ1 Income
If income> Ѳ1 AND savings> Ѳ2 THEN low-risk ELSE high-risk
Example of discriminant
Low Risk
High Risk
![Page 26: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/26.jpg)
Classification
• Decision Type– 0/1 (low-risk/high-risk)– P(Y|X) where Y customer attribute and X is 0/1– P(Y=1|X=x) =0.8
![Page 27: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/27.jpg)
Classification
• Pattern Recognition– Optical character Recognition– Recognizing character code from images– Multiple classes– Collection of strokes, has a regularity(not random
dots)– Capture in learning a program– Sequence of characters eg . T?e word– Face recognition
![Page 28: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/28.jpg)
Classification
• Speech recognition– Input is acoustic and classes are words
• Medical Diagnosis
![Page 29: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/29.jpg)
Classification
• Knowledge Extraction– Learning a rule from data
• Compression– Fitting a rule to the data– Outlier Detection– Finding the instances that do not obey rule are
exceptions– E.g. Fraud
![Page 30: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/30.jpg)
Regression• If something can predict
price of a used car?– Input: Brand, year, engine
capacity– Output: Car Price
![Page 31: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/31.jpg)
Regression
• X denote car attribute• Y be the price• Survey past transaction • Collect training data y:price
• Fitted function• Y = wx+w0 x: milage
![Page 32: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/32.jpg)
Supervised Learning
• There is an input and an output• Learn mapping from input to output• Model defined up to a set of parameters: y = g(x|Ѳ)
g(.) is the model and Ѳ are its parameters
y is the number of regressions or a class code (0/1)
![Page 33: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/33.jpg)
Unsupervised Learning
• Only have input data• Aim is to find regularities in the input• There is a pattern
![Page 34: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/34.jpg)
Unsupervised Learning
• To find the regularities in the input• Structure in the input space• Density Estimation• Clustering• Image Compression
![Page 35: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/35.jpg)
Reinforcement Learning
![Page 36: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/36.jpg)
Notes
• Evolution defines us• We change our behavior • To cope with change • We don’t hardwire all sort of behavior• Evolution gave us mechanism to learn• We recognize, recall the strategy• Learning has limitation also
![Page 37: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/37.jpg)
Can we grow a third arm??
![Page 38: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/38.jpg)
Evolution in ML
• Our aim is not understand the process underlying learning in human
• To build useful systems as in domain of engineering
![Page 39: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/39.jpg)
Science fitting models of data
• Design experiments, observe and collect data• Extract knowledge by finding out simple
models, that explains the data• Process of extracting general rules from a set
of cases is Induction • Going from particular observation to general
description Statistics: inference Learning: estimation
![Page 40: Introduction to Machine Learning Anjeli Singh Computer Science and Software Engineering April 28 th 2008](https://reader031.vdocuments.us/reader031/viewer/2022013004/56649d2f5503460f94a07814/html5/thumbnails/40.jpg)
Relevant Resources
• Journal of Machine Learning Research
• Neural Computation• Neural Information
Processing System (NIPS)• Book: Introduction to
Machine Learning by Ethem ALPAYDIN
The MIT Press