church tutorial - mit opencourseware · open the church tutorial document in the shared dropbox....
TRANSCRIPT
![Page 1: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/1.jpg)
Church Tutorial
CBMM Summer School 08.21.151
![Page 2: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/2.jpg)
Probabilistic Programming
Programming + probabilistic modeling
Good representation for AI and cognition
Increasing interest over the past 10 years: BLOG, Bugs, PyMC, ProbLog, Church, Stan, Venture…
Check out: http://probabilistic-programming.org/wiki/Home
https://moalquraishi.wordpress.com/2015/03/29/the-state-of-probabilistic-programming/
2
![Page 3: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/3.jpg)
The Church Language
Probabilistic program based on Scheme (based on Lisp based on the Lambda calculus)
Compositional, code is data
Several inference engines
Under construction! * *
Founding paper:Goodman, Mansinghka, Roy, Bonawitz and Tenenbaum, 2008
Check out forestdb.org
Check out Webppl3
![Page 4: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/4.jpg)
Objectives for Tutorial
Become familiar with Church syntax
Run ‘forward’ a few models
Get sense of program/distribution equivalence
mem
Query operator and sampling (rejection sampling, mcmc)
Examples:
Hypothesis-testing through coin-flipping example
Causal network inference (medical diagnosis, social inference)
Intuitive physics and intuitive psychology
4
![Page 5: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/5.jpg)
Prerequisites and Set-Up
Open local installation of Church if you have one(i.e. open ‘index.html’ under webchurch/online)
OR
Open https://probmods.org/ ]
AND
Open the ‘church tutorial’ document in the shared dropbox
AND
Play a game of Noisy Tomer Says
5
![Page 6: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/6.jpg)
Getting Started - Church Syntax
Similar to Scheme/Lisp
Based on λ-calculus, computing by applying functions
Polish notation: (+ 2 2) instead of 2 + 2
6
![Page 7: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/7.jpg)
Getting Started - Church Syntax
Math and logic: +, *, >, equal?, and, or…
Naming variables: define
Listing things: list
Quoting things: ‘ ( THIS IS NOT DIRT)
If-ing things: (if condition expression1 expression2)
7
![Page 8: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/8.jpg)
Getting Started - Church Syntax
Functions: lambda
(define function-name (lambda (var1 var2 … ))some-computation)
OR
(define (function-name var1 var2 …)some-computation)
8
![Page 9: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/9.jpg)
Getting Started - Church Syntax
Other useful notions (let, map, fold, case, …)
See:
https://www.probmods.org/
9
![Page 10: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/10.jpg)
Objectives for Tutorial
Become familiar with Church syntax
Run ‘forward’ a few models
Get sense of program/distribution equivalence
mem
Query operator and sampling (rejection sampling, mcmc)
Examples:
Hypothesis-testing through coin-flipping example
Causal network inference (medical diagnosis, social inference)
Size principle (number game)
10
![Page 11: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/11.jpg)
Forward sampling
Exchangeable Random Primitives (XRPs)
Distribution vs. Sampling
Examples:
Coin flipping
Gaussian samples
memoization
11
![Page 12: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/12.jpg)
Objectives for Tutorial
Become familiar with Church syntax
Run ‘forward’ a few models <- Generative modeling
Get sense of program/distribution equivalence
mem
Query operator and sampling (rejection sampling, mcmc, etc.)
Examples:
Hypothesis-testing through coin-flipping example
Causal network inference (medical diagnosis)
Planning and social reasoning
Intuitive physics
12
![Page 13: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/13.jpg)
Inference, Sampling and “query”
Sample generative models (‘run forward’)
Inference (‘run backward’)
© Python (Monty) Picture Ltd. All rights reserved. This content is excluded from our Creative
Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
13
![Page 14: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/14.jpg)
Inference, Conditioning, sampling and “query”
(querygenerative-modelwhat-we-want-to-knowwhat-we-know)
Syntax:
“What we know” is the condition
This procedure defines a distribution
Setting condition=true is simply sampling from the generative model
14
![Page 15: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/15.jpg)
Rejection Query
(rejection-querygenerative-modelwhat-we-want-to-knowwhat-we-know)
15
![Page 16: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/16.jpg)
Implementing Rejection Query
1. Run the model forward
2. Check the condition
3. Accept or repeat
16
![Page 17: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/17.jpg)
Rejection Query
Very general
Very simple
Very terrible
© Saturday Morning Breakfast Cereal. All rights reserved. This content is excluded from ourCreative Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
17
![Page 18: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/18.jpg)
MH-query
The backbone of inference in Church
(mh-querynum-samples lag
generative-modelwhat-we-want-to-knowwhat-we-know)
Random walk in program evaluation space
18
![Page 19: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/19.jpg)
MH-query
Very general
Some decisions to make
Could take a while
Biased (burn in)
19
![Page 20: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/20.jpg)
Objectives for Tutorial
Become familiar with Church syntax
Run ‘forward’ a few models <- Generative modeling
Get sense of program/distribution equivalence
mem
Query operator and sampling (rejection sampling, mcmc, etc.)
Examples:
Hypothesis-testing through coin-flipping example
Causal network inference (medical diagnosis)
Intuitive physics
Planning and social reasoning
20
![Page 21: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/21.jpg)
Example – Coin Flipping
P(H) = 0.1P(H) = 0.5
Courtesy of xkcd. License CC BY-NC 2.5.
21
![Page 22: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/22.jpg)
Example – Coin Flipping
Re-implement Josh’s example of the trick coin
New hypothesis: Biased coin
New new hypothesis: Markov coin
Newest hypothesis: Add your own!
Courtesy of xkcd. License CC BY-NC 2.5.
22
![Page 23: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/23.jpg)
Example – Causal Inference
Courtesy of xkcd. License CC BY-NC 2.5.
23
![Page 24: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/24.jpg)
Example – Intuitive Physics
Forward Sampling for Prediction
Inference
24
![Page 25: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/25.jpg)
Example – Intuitive Psychology
25
![Page 26: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/26.jpg)
Example – Intuitive Psychology
26
![Page 27: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/27.jpg)
Example – Social Communication
London
Ben
1.00.3
27
![Page 28: Church Tutorial - MIT OpenCourseWare · Open the church tutorial document in the shared dropbox. AND. Play a game of Noisy Tomer Says . 5. Getting Started - Church Syntax. Similar](https://reader036.vdocuments.us/reader036/viewer/2022062919/5edfb7a6ad6a402d666b09e6/html5/thumbnails/28.jpg)
MIT OpenCourseWarehttps://ocw.mit.edu
Resource: Brains, Minds and Machines Summer CourseTomaso Poggio and Gabriel Kreiman
The following may not correspond to a particular course on MIT OpenCourseWare, but has beenprovided by the author as an individual learning resource.
For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.