shirish shevade - visvesvaraya technological...
TRANSCRIPT
Machine Learning
Shirish Shevade
Department of Computer Science and AutomationIndian Institute of ScienceBangalore 560 012, India.
July 02, 2010
Shirish Shevade (IISc) Machine Learning July 02, 2010 1 / 18
Machine Learning
Humans have developed sophisticated skills for recognizing patterns
Spam RecognitionReading handwritingUnderstanding spoken wordsFace recognitionWeather Prediction
Shirish Shevade (IISc) Machine Learning July 02, 2010 2 / 18
Machine Learning
Humans have developed sophisticated skills for recognizing patterns
Spam RecognitionReading handwritingUnderstanding spoken wordsFace recognitionWeather Prediction
Can we write computer programs which learn these skills from pastexperience?
Shirish Shevade (IISc) Machine Learning July 02, 2010 2 / 18
Machine Learning
Humans have developed sophisticated skills for recognizing patterns
Spam RecognitionReading handwritingUnderstanding spoken wordsFace recognitionWeather Prediction
Can we write computer programs which learn these skills from pastexperience?
Machine Learning: Study of theory, algorithms and implementationsthat enable computers to learn from experience
Shirish Shevade (IISc) Machine Learning July 02, 2010 2 / 18
Applications of Machine Learning
Data Mining
Speech Recognition
Bioinformatics [Baldi and Brunak, 1998]
Statistical Debugging [Zheng et al, ICML, 2006]
Compiler Design [Stephenson et al, PLDI, 2003; Joseph et al, TheCompiler Design Handbook, 2008]
Computer vision
Stock market index prediction
Shirish Shevade (IISc) Machine Learning July 02, 2010 3 / 18
Applications of Machine Learning
Data Mining
Speech Recognition
Bioinformatics [Baldi and Brunak, 1998]
Statistical Debugging [Zheng et al, ICML, 2006]
Compiler Design [Stephenson et al, PLDI, 2003; Joseph et al, TheCompiler Design Handbook, 2008]
Computer vision
Stock market index prediction
Shirish Shevade (IISc) Machine Learning July 02, 2010 3 / 18
Applications of Machine Learning
Data Mining
Speech Recognition
Bioinformatics [Baldi and Brunak, 1998]
Statistical Debugging [Zheng et al, ICML, 2006]
Compiler Design [Stephenson et al, PLDI, 2003; Joseph et al, TheCompiler Design Handbook, 2008]
Computer vision
Stock market index prediction
Shirish Shevade (IISc) Machine Learning July 02, 2010 3 / 18
Applications of Machine Learning
Data Mining
Speech Recognition
Bioinformatics [Baldi and Brunak, 1998]
Statistical Debugging [Zheng et al, ICML, 2006]
Compiler Design [Stephenson et al, PLDI, 2003; Joseph et al, TheCompiler Design Handbook, 2008]
Computer vision
Stock market index prediction
Shirish Shevade (IISc) Machine Learning July 02, 2010 3 / 18
Applications of Machine Learning
Data Mining
Speech Recognition
Bioinformatics [Baldi and Brunak, 1998]
Statistical Debugging [Zheng et al, ICML, 2006]
Compiler Design [Stephenson et al, PLDI, 2003; Joseph et al, TheCompiler Design Handbook, 2008]
Computer vision
Stock market index prediction
Shirish Shevade (IISc) Machine Learning July 02, 2010 3 / 18
Applications of Machine Learning
Data Mining
Speech Recognition
Bioinformatics [Baldi and Brunak, 1998]
Statistical Debugging [Zheng et al, ICML, 2006]
Compiler Design [Stephenson et al, PLDI, 2003; Joseph et al, TheCompiler Design Handbook, 2008]
Computer vision
Stock market index prediction
Shirish Shevade (IISc) Machine Learning July 02, 2010 3 / 18
Applications of Machine Learning
Data Mining
Speech Recognition
Bioinformatics [Baldi and Brunak, 1998]
Statistical Debugging [Zheng et al, ICML, 2006]
Compiler Design [Stephenson et al, PLDI, 2003; Joseph et al, TheCompiler Design Handbook, 2008]
Computer vision
Stock market index prediction
Shirish Shevade (IISc) Machine Learning July 02, 2010 3 / 18
Different Learning Techniques
Supervised
Unsupervised
Semi-supervised
Reinforcement
. . .
Shirish Shevade (IISc) Machine Learning July 02, 2010 4 / 18
Types of Learning Problems
Classification
Shirish Shevade (IISc) Machine Learning July 02, 2010 5 / 18
Types of Learning Problems
Classification- Binary, multi-class,
multi-label etc.
Classify an image as
star or galaxy
Shirish Shevade (IISc) Machine Learning July 02, 2010 5 / 18
Types of Learning Problems
Regression- Real valued output
Predict tomorrow’s
rainfall
Shirish Shevade (IISc) Machine Learning July 02, 2010 6 / 18
Types of Learning Problems
Clustering
- Find similar data items
Application: MarketResearch
Shirish Shevade (IISc) Machine Learning July 02, 2010 7 / 18
Types of Learning Problems
Ranking
- Order examples by preference
- Application: Ordering of web search results
Shirish Shevade (IISc) Machine Learning July 02, 2010 8 / 18
Types of Learning Problems
Classification
Regression
Clustering
Ranking
Shirish Shevade (IISc) Machine Learning July 02, 2010 9 / 18
Algorithms
Linear models for classification and Regression
Naive Bayes Classifiers
Decision Trees
Perceptron
Support Vector Machines (SVM)
Gaussian Processes
Clustering algorithms (k-means, hierarchical)
. . .
Shirish Shevade (IISc) Machine Learning July 02, 2010 10 / 18
Research Challenges
Relational Learning
- Finding events, relationships in the data- Use of these relationships to achieve better classification accuracy- Application - Web page classification
Information Extraction
- Extraction of structure from unstructured, heterogeneous sources- Applications: Tracking News, Ad placement on webpages
Cross-language information retrieval
Finance
- Credit card fraud detection- Detection of stock market manipulation
Shirish Shevade (IISc) Machine Learning July 02, 2010 11 / 18
Research Challenges
Relational Learning
- Finding events, relationships in the data- Use of these relationships to achieve better classification accuracy- Application - Web page classification
Information Extraction
- Extraction of structure from unstructured, heterogeneous sources- Applications: Tracking News, Ad placement on webpages
Cross-language information retrieval
Finance
- Credit card fraud detection- Detection of stock market manipulation
Shirish Shevade (IISc) Machine Learning July 02, 2010 11 / 18
Research Challenges
Relational Learning
- Finding events, relationships in the data- Use of these relationships to achieve better classification accuracy- Application - Web page classification
Information Extraction
- Extraction of structure from unstructured, heterogeneous sources- Applications: Tracking News, Ad placement on webpages
Cross-language information retrieval
Finance
- Credit card fraud detection- Detection of stock market manipulation
Shirish Shevade (IISc) Machine Learning July 02, 2010 11 / 18
Research Challenges
Relational Learning
- Finding events, relationships in the data- Use of these relationships to achieve better classification accuracy- Application - Web page classification
Information Extraction
- Extraction of structure from unstructured, heterogeneous sources- Applications: Tracking News, Ad placement on webpages
Cross-language information retrieval
Finance
- Credit card fraud detection- Detection of stock market manipulation
Shirish Shevade (IISc) Machine Learning July 02, 2010 11 / 18
Research Challenges
Structured Prediction
Shirish Shevade (IISc) Machine Learning July 02, 2010 12 / 18
Research Challenges
Structured Prediction
- Output is not a scalar -sequence, tree, graphetc
- Use interdependence inthe outputs
- Applications:Computational Biology,Natural LanguageAnalysis
Shirish Shevade (IISc) Machine Learning July 02, 2010 12 / 18
Applications in Machine Learning
Linear Algebra
Non-negative matrixfactorization
Singular valuedecomposition
Optimization
Duality ideas
Efficient solutions ofoptimization problems
Probability
Bayesian networks
Markov networks
Graph Theory
Graph cut algorithms
Max flow algorithms
Game Theory
Feature selection
Adversarial learning
Shirish Shevade (IISc) Machine Learning July 02, 2010 13 / 18
Applications in Machine Learning
Linear Algebra
Non-negative matrixfactorization
Singular valuedecomposition
Optimization
Duality ideas
Efficient solutions ofoptimization problems
Probability
Bayesian networks
Markov networks
Graph Theory
Graph cut algorithms
Max flow algorithms
Game Theory
Feature selection
Adversarial learning
Shirish Shevade (IISc) Machine Learning July 02, 2010 13 / 18
Applications in Machine Learning
Linear Algebra
Non-negative matrixfactorization
Singular valuedecomposition
Optimization
Duality ideas
Efficient solutions ofoptimization problems
Probability
Bayesian networks
Markov networks
Graph Theory
Graph cut algorithms
Max flow algorithms
Game Theory
Feature selection
Adversarial learning
Shirish Shevade (IISc) Machine Learning July 02, 2010 13 / 18
Applications in Machine Learning
Linear Algebra
Non-negative matrixfactorization
Singular valuedecomposition
Optimization
Duality ideas
Efficient solutions ofoptimization problems
Probability
Bayesian networks
Markov networks
Graph Theory
Graph cut algorithms
Max flow algorithms
Game Theory
Feature selection
Adversarial learning
Shirish Shevade (IISc) Machine Learning July 02, 2010 13 / 18
Applications in Machine Learning
Linear Algebra
Non-negative matrixfactorization
Singular valuedecomposition
Optimization
Duality ideas
Efficient solutions ofoptimization problems
Probability
Bayesian networks
Markov networks
Graph Theory
Graph cut algorithms
Max flow algorithms
Game Theory
Feature selection
Adversarial learning
Shirish Shevade (IISc) Machine Learning July 02, 2010 13 / 18
Some Reputed Journals and Conferences
Journals
Machine Learning
Jl. of Machine LearningResearch
Neural Computation
Neural Networks
IEEE PAMI
IEEE NN
Pattern Recognition
Conferences
Intl Conf on MachineLearning (ICML)
Neural Inf. ProcessingSystems (NIPS)
Intl Joint Conf on AI(IJCAI)
IEEE Intl Conf on DataMining (IEEE ICDM)
SIGKDD
CIKM
Shirish Shevade (IISc) Machine Learning July 02, 2010 14 / 18
Some Reputed Journals and Conferences
Journals
Machine Learning
Jl. of Machine LearningResearch
Neural Computation
Neural Networks
IEEE PAMI
IEEE NN
Pattern Recognition
Conferences
Intl Conf on MachineLearning (ICML)
Neural Inf. ProcessingSystems (NIPS)
Intl Joint Conf on AI(IJCAI)
IEEE Intl Conf on DataMining (IEEE ICDM)
SIGKDD
CIKM
Shirish Shevade (IISc) Machine Learning July 02, 2010 14 / 18
Readings
Books
Pattern Recognition and Machine Learning
- C. M. Bishop
The Elements of Statistical Learning
- Hastie, Tibshirani and Friedman
Pattern Classification
- Duda, Hart and Stork
Machine Learning
- Mitchell
Learning with Kernels
- Scholkopf and Smola
Data Mining: Practical Machine Learning Tools and Techniques,
- Witten and Frank
Probabilistic Graphical Models: Principles and Techniques
- Koller and Friedman
Shirish Shevade (IISc) Machine Learning July 02, 2010 15 / 18
. . . Some Resources
Tutorials
The Discipline of Machine Learning
- Mitchell
A Tutorial on Support Vector Machines for Pattern Recognition
- C.J.C. Burges
Information Extraction
- Sunita Sarawagi
A Tutorial on Spectral Clustering
- Ulrike von Luxburg
Shirish Shevade (IISc) Machine Learning July 02, 2010 16 / 18
. . . Some Resources
Popular Sites
David Aha’s page
- http://home.earthlink.net/~dwaha/research/tutorials.html
Andrew Moore’s page
- http://www.cs.cmu.edu/~awm/tutorials
Tom Dietterich’s page
- http://web.engr.oregonstate.edu/~tgd/projects/tutorials.html
Kernel methods page
- http://www.kernel-machines.org
LIBSVM - Kernel Methods Software
- http://www.csie.ntu.edu.tw/~cjlin/libsvm/
Weka - Machine Learning Software
- http://sourceforge.net/projects/weka/
Machine Learning Database Repository
- http://mlearn.ics.uci.edu/MLRepository.html
Shirish Shevade (IISc) Machine Learning July 02, 2010 17 / 18