model-based machine learning - imperial college …nsjones/talkslides/bishopslides.pdfmodel-based...
TRANSCRIPT
![Page 1: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/1.jpg)
Model-based Machine Learning Chris Bishop
Microsoft Research Cambridge
Royal Society, March 2012
![Page 2: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/2.jpg)
![Page 3: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/3.jpg)
Traditional machine learning
Logistic regression
Neural networks
K-means, mixture of Gaussians
PCA, kernel PCA, ICA, FA
Support vector machines
Deep belief networks
Decision trees and random forests
… many others …
![Page 4: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/4.jpg)
Model-based machine learning
Goal: a single modelling framework which supports a wide range of models
Traditional:
“how do I map my problem onto a standard algorithm”?
Model-based:
“what is the model that represents my problem”?
![Page 5: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/5.jpg)
Realisation of model-based ML
Bayesian framework
Probabilistic graphical models
Efficient deterministic inference
![Page 6: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/6.jpg)
Movie recommender demo
![Page 7: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/7.jpg)
Probabilistic graphical models
Maths (M)
Algebra (A) Geometry (G)
P(M, G, A) = P(M) P(G|M) P(A|M)
Graph structure captures domain knowledge
![Page 8: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/8.jpg)
Efficient inference
![Page 9: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/9.jpg)
Local message-passing
?
Maths (M)
Algebra (A) Geometry (G)
![Page 10: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/10.jpg)
What if distributions are intractable?
True distribution Monte Carlo Variational Message Passing
Loopy belief propagation
Expectation propagation
⁞
![Page 11: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/11.jpg)
Algorithms Models
M. E. Tipping and C. M. Bishop (1997)
C. M. Bishop (1999)
![Page 12: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/12.jpg)
![Page 13: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/13.jpg)
Childhood Asthma
![Page 14: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/14.jpg)
Allergic Sensitisation Model
![Page 15: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/15.jpg)
Comparison with traditional ML
Separation of model and training algorithm
Auto-generated inference algorithm
Easy extension to more complex situations
Modify model, use the same inference algorithms
Flexible as requirements change
Compact code
Easy to write and maintain
Transparent functionality
Many traditional methods are special cases
One simple framework for newcomers to the field
![Page 16: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/16.jpg)
“Big data”
Computational size vs. statistical size
?
len
gth
temperature
![Page 17: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/17.jpg)
Noisy ranking
Conventional approach to ranking: “Elo” single strength value for each player cannot handle teams, or more than 2 players
![Page 18: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/18.jpg)
Bayesian Ranking: TrueSkillTM
y12
1 2
s1 s2
R. Herbrich, T. Minka, and T. Graepel; NIPS (2006)
![Page 19: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/19.jpg)
s1 s2 s3 s4
t1
y12
t2 t3
y23
Multi-player multi-team model
![Page 20: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/20.jpg)
y12
1 2
s1 s2
y12
1 2
s1 s2 ^ ^
^ ^
^
![Page 21: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/21.jpg)
TrueSkillTM
Sept. 2005; 10s of millions of users; millions of matches per day
![Page 22: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/22.jpg)
Convergence
0
5
10
15
20
25
30
35
40 Le
vel
0 100 200 300 400
Number of Games
char (Elo)
SQLWildman (Elo)
char (TrueSkill™)
SQLWildman (TrueSkill™)
![Page 23: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/23.jpg)
Infer.NET
1. Specify your machine learning problem as a probabilistic model in a .NET program (typically 10-20 lines of code).
2. Use Infer.NET to compile the model into optimized runtime code.
3. Run the code to make inferences on your data automatically.
research.microsoft.com/infernet
![Page 24: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/24.jpg)
research.microsoft.com/~cmbishop
![Page 25: Model-based Machine Learning - Imperial College …nsjones/TalkSlides/BishopSlides.pdfModel-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012](https://reader030.vdocuments.us/reader030/viewer/2022040812/5e549fbebbf31657300b84e6/html5/thumbnails/25.jpg)