computer vision: models, learning and inferencejmagee/cs201/slides/02_intro_to_probability.pdf ·...
TRANSCRIPT
![Page 1: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/1.jpg)
Computer vision: models, learning and inference
Chapter 2
Introduction to probability
Please send errata to [email protected]
![Page 2: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/2.jpg)
Random variables
• A random variable x denotes a quantity that is uncertain
• May be result of experiment (flipping a coin) or a real world measurements (measuring temperature)
• If observe several instances of x we get different values
• Some values occur more than others and this information is captured by a probability distribution
2 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 3: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/3.jpg)
Discrete Random Variables
3 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 4: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/4.jpg)
Continuous Random Variable
4 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 5: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/5.jpg)
Joint Probability
• Consider two random variables x and y
• If we observe multiple paired instances, then some combinations of outcomes are more likely than others
• This is captured in the joint probability distribution
• Written as Pr(x,y)
• Can read Pr(x,y) as “probability of x and y”
5 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 6: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/6.jpg)
Joint Probability
6 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 7: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/7.jpg)
Marginalization We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
7 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 8: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/8.jpg)
Marginalization We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
8 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 9: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/9.jpg)
Marginalization We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
9 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 10: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/10.jpg)
Marginalization We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
Works in higher dimensions as well – leaves joint distribution between whatever variables are left
10 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 11: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/11.jpg)
Conditional Probability
• Conditional probability of x given that y=y1 is relative propensity of variable x to take different outcomes given that y is fixed to be equal to y1.
• Written as Pr(x|y=y1)
11 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 12: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/12.jpg)
Conditional Probability • Conditional probability can be extracted from joint probability
• Extract appropriate slice and normalize
12 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 13: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/13.jpg)
Conditional Probability
• More usually written in compact form
• Can be re-arranged to give
13 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 14: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/14.jpg)
Conditional Probability
• This idea can be extended to more than two variables
14 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 15: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/15.jpg)
Bayes’ Rule From before:
Combining:
Re-arranging:
15 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 16: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/16.jpg)
Bayes’ Rule Terminology
Posterior – what we know about y after seeing x
Prior – what we know about y before seeing x
Likelihood – propensity for observing a certain value of x given a certain value of y
Evidence –a constant to ensure that the left hand side is a valid distribution
16 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 17: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/17.jpg)
Independence • If two variables x and y are independent then variable x tells
us nothing about variable y (and vice-versa)
17 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 18: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/18.jpg)
Independence • If two variables x and y are independent then variable x tells
us nothing about variable y (and vice-versa)
18 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 19: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/19.jpg)
Independence • When variables are independent, the joint factorizes into a
product of the marginals:
19 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 20: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/20.jpg)
Expectation
Expectation tell us the expected or average value of some function f [x] taking into account the distribution of x
Definition:
20 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 21: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/21.jpg)
Expectation
Expectation tell us the expected or average value of some function f [x] taking into account the distribution of x
Definition in two dimensions:
21 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 22: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/22.jpg)
Expectation: Common Cases
22 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 23: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/23.jpg)
Expectation: Rules
Rule 1: Expected value of a constant is the constant
23 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 24: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/24.jpg)
Expectation: Rules
Rule 2: Expected value of constant times function is constant times expected value of function
24 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 25: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/25.jpg)
Expectation: Rules
Rule 3: Expectation of sum of functions is sum of expectation of functions
25 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 26: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/26.jpg)
Expectation: Rules
Rule 4: Expectation of product of functions in variables x and y is product of expectations of functions if x and y are independent
26 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 27: Computer vision: models, learning and inferencejmagee/cs201/slides/02_Intro_To_Probability.pdf · Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27 •](https://reader036.vdocuments.us/reader036/viewer/2022071002/5fbfa6b0b08fe4103619df7b/html5/thumbnails/27.jpg)
Conclusions
27 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Rules of probability are compact and simple
• Concepts of marginalization, joint and conditional probability, Bayes rule and expectation underpin all of the models in this book
• One remaining concept – conditional expectation – discussed later