![Page 1: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/1.jpg)
GURLSEffective machine learning made easy
Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco
MLCC 2015
![Page 2: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/2.jpg)
Machine Learning in a “Nutshell”
Statistical modeling
+ + =
“Favorite” pre-processing
Data representation
A share of big data
![Page 3: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/3.jpg)
Machine Learning in a “Nutshell”
Statistical modeling
+ + =
“Favorite” pre-processing
Data representation
A share of big data
![Page 4: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/4.jpg)
Machine Learning in a “Nutshell”
Statistical modeling
+ + =
“Favorite” pre-processing
Data representation
A share of big data Several Algorithms
Available
![Page 5: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/5.jpg)
Machine Learning in a “Nutshell”
Statistical modeling
+ + =
“Favorite” pre-processing
Data representation
A share of big data
How to select a good hyperparameters range?
Several Algorithms Available
![Page 6: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/6.jpg)
Machine Learning in a “Nutshell”
Statistical modeling
+ + =
“Favorite” pre-processing
Data representation
A share of big data
How to select a good hyperparameters range?
Several Algorithms Available Lots of
Boilerplate Code
![Page 7: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/7.jpg)
GURLS Automates this boring part!
![Page 8: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/8.jpg)
Multiple “Depths"
Casual User
AdvancedUser
MachineLearner
![Page 9: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/9.jpg)
Statistical modeling
+ + =
“Favorite” pre-processing
Data representation
A share of big data
GURLS as a Casual User
model = gurls_train(Xtr, ytr)
ypred = gurls_test(model,Xts)
![Page 10: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/10.jpg)
Statistical modeling
+ + =
“Favorite” pre-processing
Data representation
A share of big data
What if we wanted to change… e.g. the kernel?
model = gurls_train(Xtr, ytr, ‘kernel’,’rbf’,’kerpar’,0.1)
ypred = gurls_test(model,Xts)
![Page 11: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/11.jpg)
General Parameters Feeding Syntax
model = gurls_train(Xtr, ytr, ‘par1’,val1,’par2’,val2,...)
ypred = gurls_test(model,Xts)
![Page 12: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/12.jpg)
Features of the Library Model Selection
Kernels- Linear - Polynomial - RBF - Chisquared - Quasi-periodic
Algoritms- KRLS with
- Tikhonov - Landweber - Nu-method - Truncated-SVD - Conjugate Gradient
- Kernel Logistic Regression - Gaussian Processes - Random Features
- Performance Measures: - RMSE - Macroavg - Precision/Recall
- Automatic Parameter Tuning: - Automatic split &
randomization - Hold out - Leave-one out - k-fold
- Automatic Range for Hyperparameters:
MATLAB & C++Interfaces
![Page 13: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/13.jpg)
Example
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
KRLS - RBF Kernel Landweber FilterLinear RLS
![Page 14: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/14.jpg)
Example
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
KRLS - RBF Kernel Landweber FilterLinear RLS
![Page 15: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/15.jpg)
Example
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
KRLS - RBF Kernel Landweber FilterLinear RLS
gurls_train(Xtr, ytr, ’algorithm’,’lrls’)
gurls_train(Xtr, ytr, ’algorithm’,’krls’, ’kernelfun’,’rbf’, ’filter’,’land’)
![Page 16: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/16.jpg)
Under the HoodThe Pipeline
![Page 17: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/17.jpg)
Under the hood - The Pipeline
SplitParam
sel Kernel Alg
highly modular, i.e. reuse of code components
![Page 18: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/18.jpg)
Under the hood - The Pipeline
SplitParam
sel Kernel Alg
highly modular, i.e. reuse of code components
![Page 19: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/19.jpg)
Example: from gurls_train to the pipeline
gurls_train(Xtr, ytr,’algorithm’,’lrls’)
![Page 20: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/20.jpg)
Example: from gurls_train to the pipeline
gurls_train(Xtr, ytr,’algorithm’,’lrls’)
With the pipeline…
![Page 21: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/21.jpg)
Examples with what we have seen
gurls_train(Xtr, ytr,’algorithm’,’krls’,’kernelfun’,’rbf’,’filter’,’land’)
![Page 22: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/22.jpg)
Examples with what we have seen
gurls_train(Xtr, ytr,’algorithm’,’krls’,’kernelfun’,’rbf’,’filter’,’land’)
With the pipeline…
![Page 23: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/23.jpg)
Structure of a stage
gurls_train(Xtr, ytr,’algorithm’,’lrls’)
![Page 24: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/24.jpg)
All stages have same interface
<stage>_<funcname>(X, y, opt)
![Page 25: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/25.jpg)
All blocks have same interface
<stage>_<funcname>(X, y, opt)
![Page 26: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/26.jpg)
Results - Classification Benchmarks
![Page 27: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/27.jpg)
Results - Classification Benchmarks
![Page 28: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/28.jpg)
Results - Comparison with the state of the art
![Page 29: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/29.jpg)
Results - Object Recognition in Robotics
![Page 30: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/30.jpg)
Download or Pull from Github: https://github.com/LCSL/GURLS
Reference Paper:
GURLS: a Least Squares Library for Supervised Learning. A. Tacchetti, P. K. Mallapragada, M. Santoro and L. Rosasco Journal of Machine Learning Research
![Page 31: gurls at mlcc - LCSLlcsl.mit.edu/courses/mlcc/mlcc2015/slides/MLCC_09_GURLS.pdfEffective machine learning made easy Alessandro Rudi Carlo Ciliberto and Lorenzo Rosasco MLCC 2015. Machine](https://reader030.vdocuments.us/reader030/viewer/2022041022/5ed38c0210258d03ed2368a7/html5/thumbnails/31.jpg)
This afternoon at 2PM: a Lab to get acquainted with GURLS
+
A small machine learning challenge (just for fun)