software libraries for pgmsdechter/courses/ics-276/spring...“a c++ library for discrete graphical...
TRANSCRIPT
![Page 1: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/1.jpg)
Software Libraries for PGMsKevin Rothi
![Page 2: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/2.jpg)
Very popular tools for ML/NNs/Deep Learning...
- SciKit Learn- Tensorflow- Keras- Torch- CUDA- Theano- Caffe
![Page 3: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/3.jpg)
No shortage of small libraries for graphical models…
http://www.cs.ubc.ca/~murphyk/Software/bnsoft.html
(Last updated 16 June 2014)
69 Libraries
![Page 4: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/4.jpg)
![Page 5: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/5.jpg)
Of these...23 use junction trees for inference (some use Jtrees in addition to other algos)
5 use gibbs sampling
Many seem to be defunct, unsupported, or abandoned…
Why are there so many of these?
![Page 6: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/6.jpg)
“It’s hard to strike a balance between generality and usability.” -Prof. Ihler
![Page 7: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/7.jpg)
Positive qualities of software libraries… (CISQ)Reliable
Efficient
Secure
Maintainable
Appropriately Scoped (Size)
“CISQ has defined five major desirable characteristics of a piece of software needed to provide business value…” (https://en.wikipedia.org/wiki/Software_quality)
![Page 8: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/8.jpg)
The rest of this talk will focus on the libraries that can begin to convincingly claim to fulfill these qualities (in my opinion)
![Page 9: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/9.jpg)
...
![Page 10: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/10.jpg)
![Page 11: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/11.jpg)
Generality Usability
![Page 12: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/12.jpg)
“Python library for Probabilistic Graphical Models”- Details are sparse, but it seems that this library has its origins as a Google
Summer of Code project. There appear to be 4 major contributors: Ankur Ankan from Radboud University, Yashu Seth, Abinash Panda, UtkarshKhalibartan, and an unnamed GitHub user contributing under the handle “vivek425ster”.
- Open source- Version 0.1.2- Still under development (last commit on April 11)- MIT License- 48 contributors
![Page 13: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/13.jpg)
ModelsBayesian ModelMarkov ModelFactor GraphCluster GraphJunction TreeMarkov ChainNoisyOr ModelNaive BayesDynamicBayesianNetwork
![Page 14: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/14.jpg)
Sampling MethodsGibbs SamplerBayesian Model SamplersHamiltonian Monte CarloNo U-Turn Sampler
![Page 15: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/15.jpg)
AlgorithmsVariable EliminationBelief PropagationMPLPDynamic Bayesian Network Inference
![Page 16: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/16.jpg)
PositivesVery approachable (well documented)
Actively supported (bug fixes, features added)
Python
![Page 17: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/17.jpg)
NegativesNot backed by Big 4 company
Development seems to be slowing down (fewer commits over time)
![Page 18: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/18.jpg)
2nd half of talk will focus on examples of what you can do with pgmpy...
![Page 19: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/19.jpg)
![Page 20: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/20.jpg)
Generality Usability
![Page 21: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/21.jpg)
“A C++ Library for Discrete Graphical Models”- Developed at The Heidelberg Collaboratory for Image Processing at the
University of Heidelberg. There are 3 main developers: Bjoern Andres, Thorsten Beier, and Joerg H. Kappes.
- Open source- Version 2.0.2- Still under development (last commit on April 5)- MIT License- 38 contributors- Wrappers for Python and Matlab
![Page 22: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/22.jpg)
ModelsGraphs of any order and structure, from second order grid graphs to irregular higher-order models
![Page 23: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/23.jpg)
Algorithms- Combinatorial/Global Optimal Methods- Linear Programming Relaxations- Message Passing Methods- Move Making Methods- Sampling- Wrapped External Code for Discrete Graphical Models
(41 total by my count)
![Page 24: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/24.jpg)
PositivesHighly general
C++
Extensive Documentation
![Page 25: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/25.jpg)
NegativesNot backed by a Big 4 company
Highly general
C++
![Page 26: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/26.jpg)
![Page 27: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/27.jpg)
Generality Usability
![Page 28: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/28.jpg)
“Edward is a Python library for probabilistic modeling, inference, and criticism. It is a testbed for fast experimentation and research with probabilistic models, ranging from classical hierarchical models on small data sets to complex deep probabilistic models on large data sets. Edward fuses three fields: Bayesian statistics and machine learning, deep learning, and probabilistic programming.”
“Formally, Edward is a Turing-complete probabilistic programming language.”
- Developed at Columbia University. Primary Developer: Dustin Tran- Open source- Version 1.3.5- Still under development (last commit on June 1)- MIT License- 77 contributors
![Page 29: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/29.jpg)
An abstraction over tensorflowDirected graphical modelsNeural networks (via libraries such as tf.layers and Keras)Implicit generative modelsBayesian nonparametrics and probabilistic programs
![Page 30: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/30.jpg)
Inference with...Variational inference
Black box variational inferenceStochastic variational inferenceGenerative adversarial networksMaximum a posteriori estimation
Monte CarloGibbs samplingHamiltonian Monte CarloStochastic gradient Langevin dynamics
Compositions of inferenceExpectation-MaximizationPseudo-marginal and ABC methodsMessage passing algorithms
![Page 31: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/31.jpg)
![Page 32: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/32.jpg)
Generality Usability
![Page 33: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/33.jpg)
“SamIam is a comprehensive tool for modeling and reasoning with Bayesian networks”
- Developed at University of California, Los Angeles by the Automated Reasoning Group of Professor Adnan Darwiche.
- Closed source
![Page 34: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/34.jpg)
Kevin’s notes on SamIamI took a look at this tool. It’s impressive in the sense that the UI is very well designed and the fact that it’s a Java program means that it can run on any machine with a Java virtual machine implementation, but the project is not open source. I can call into the code, but I can neither see nor edit the code. In my opinion, this is a serious issue. Why not host the code on Github? Also, it’s not clear what the licensing is for this software. Can I use it in an industrial/commercial application? All of these factors limit SamIam’s utility, unfortunately.
![Page 35: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/35.jpg)
![Page 36: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/36.jpg)
Installation...pip install if you’re on linux
Easy, fast, basically error-proof
![Page 37: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/37.jpg)
(As an aside…)There’s an R package called bnlearn (http://www.bnlearn.com/)
If you go to http://www.bnlearn.com/bnrepository/ there are Bayesian networks (large and small) to test with!
![Page 38: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/38.jpg)
![Page 39: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/39.jpg)
(As another aside…)daft-pgm.org
![Page 40: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/40.jpg)
![Page 41: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/41.jpg)
Back to pgmpy...
![Page 42: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/42.jpg)
![Page 43: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/43.jpg)
![Page 44: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/44.jpg)
![Page 45: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/45.jpg)
![Page 46: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/46.jpg)
Generality Usability
![Page 47: Software Libraries for PGMsdechter/courses/ics-276/spring...“A C++ Library for Discrete Graphical Models” - Developed at The Heidelberg Collaboratory for Image Processing at the](https://reader035.vdocuments.us/reader035/viewer/2022063011/5fc4f38d6b71e9013e37f150/html5/thumbnails/47.jpg)
I hope this was helpful, interesting, or provided some ideas about potential future work.
Thank you!
Questions?