continuous random variables expected values and...
TRANSCRIPT
![Page 1: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/1.jpg)
Continuous Random VariablesExpected Values and Moments
Statistics 110
Summer 2006
Copyright c©2006 by Mark E. Irwin
![Page 2: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/2.jpg)
Continuous Random Variables
When defining a distribution for a continuous RV, the PMF approach won’tquite work since summations only work for a finite or a countably infinitenumber of items. Instead they are based on the following
Definition: Let X be a continuous RV. The Probability Density Function(PDF) is a function f(x) on the range of X that satisfies the followingproperties:
0 5 10 15 20
0.00
0.04
0.08
0.12
X
f(x)
• f(x) ≥ 0
• f is piecewise continuous
• ∫∞−∞ f(x)dx = 1
Continuous Random Variables 1
![Page 3: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/3.jpg)
For any a < b, the probability that P [a < X < b] is the area under thedensity curve between a and b.
0 5 10 15 20
0.00
0.04
0.08
0.12
X
f(x)
a b
P [a < X < b] =∫ b
a
f(x)dx
Continuous Random Variables 2
![Page 4: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/4.jpg)
Note that f(a) is NOT the probability of observing X = a as
P [X = a] =∫ a
a
f(x)dx = 0
Thus the probability that a continuous RV takes on any particular value is0. (While this might seem counterintuitive, things do work properly.) Aconsequence of this is that
P [a < X < b] = P [a ≤ X < b] = P [a < X ≤ b] = P [a ≤ X ≤ b]
for continuous RVs. Note that this won’t hold for discrete RVs.
Continuous Random Variables 3
![Page 5: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/5.jpg)
Note that for small δ, if f is continuous at x
P
[x− δ
2≤ X ≤ x +
δ
2
]=
∫ x+δ2
x−δ2
f(u)du ≈ f(x)δ
X
f(x)
x −δ
2x +
δ
2x
So the probability of seeing an outcome in a small interval around x isproportional to f(x). So the PDF is giving information of how likely anobservation at x is.
Continuous Random Variables 4
![Page 6: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/6.jpg)
As with the PMF and the CDF for discrete RVs, there is a relationshipbetween the PDF, f(x), and the CDF, F (x), for continuous RVs
F (x) = P [X ≤ x] =∫ x
−∞f(u)du
f(x) = F ′(x)
assuming that f is continuous at x.
Based on this relationship, the probability for any reasonable event describinga RV can determined with the CDF as the probability of any interval satisfies
P [a < X ≤ b] = F (b)− F (a)
Note that this is slightly different than the formula given on page 47. Theabove holds for any RV (discrete, continuous, mixed). The form given onpage 47
P [a ≤ X ≤ b] = F (b)− F (a)only holds for continuous RVs.
Continuous Random Variables 5
![Page 7: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/7.jpg)
Example: Uniform RV on [0,1] (Denoted X ∼ U(0, 1))
What most people think of when we say pick a numberbetween 0 and 1. Any real number in the interval ispossible and equally likely, implying that any interval oflength h must have the same probability (which needsto be h). The PDF for X then must be
f(x) =
{1 0 ≤ x ≤ 10 x < 0 or x > 1
−1.0 −0.5 0.0 0.5 1.0 1.5 2.0
0.0
0.4
0.8
PDF of U(0,1)
x
f(x)
Continuous Random Variables 6
![Page 8: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/8.jpg)
The CDF for a U(0, 1) is
F (x) =
0 x < 0x 0 ≤ x ≤ 11 x > 1
−1.0 −0.5 0.0 0.5 1.0 1.5 2.0
0.0
0.2
0.4
0.6
0.8
1.0
CDF of U(0,1)
x
F(x
)
Continuous Random Variables 7
![Page 9: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/9.jpg)
One way to think of the CDF is that you give a value of the RV and it givesa probability associated with it (i.e. P [X ≤ x]). It can also be useful to gothe other way. Give a probability and figure out which value of the RV isassociated with it.
Lets assume that F is continuous and strictly increasing in some interval I(i.e. F = 0 to the left of I and F = 1 to the right of I) (note I mightbe unbounded). Under these assumptions the inverse function F−1 is welldefined (x = F−1(y) if F (x) = y).
Definition: The pth Quantile of the distribution F is defined to be thevalue xp such that
F (xp) = p or P [X ≤ xp] = p
Under the above assumptions xp = F−1(p).
Continuous Random Variables 8
![Page 10: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/10.jpg)
−4 −2 0 2 4
0.0
0.2
0.4
0.6
0.8
1.0
Quantiles
x
F(x
)
xp
p
Special cases of interest of the Median (p = 12) and the lower and upper
Quartiles (p = 14 and = 3
4)
Continuous Random Variables 9
![Page 11: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/11.jpg)
Note: Defining quantiles for discrete distributions is a bit tougher since theCDF doesn’t take all values between 0 and 1 (due to the jumps)
−1 0 1 2 3 4
0.0
0.2
0.4
0.6
0.8
1.0
CDF for number of heads in 3 flips
x (number of heads)
P[X
<=
x]
The definition above can be extended to solving the simultaneous equations
P [X ≤ xp] ≥ p and P [X < xp] ≤ p
Continuous Random Variables 10
![Page 12: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/12.jpg)
This can be though of as the place where the CDF jumps from below p toabove p
−1 0 1 2 3 4
0.0
0.4
0.8
CDF for number of heads in 3 flips
x (number of heads)
P[X
<=
x]
0.0 0.2 0.4 0.6 0.8 1.0
0.0
1.0
2.0
3.0
Quantile function for number of heads in 3 flips
p
x p
Continuous Random Variables 11
![Page 13: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/13.jpg)
Expected Values and Moments
Definition: The Expected Value of a continuous RV X (with PDF f(x))is
E[X] =∫ ∞
−∞xf(x)dx
assuming that∫∞−∞ |x|f(x)dx < ∞.
The expected value of a distribution is often referred to as the mean of thedistribution.
As with the discrete case, the absolute integrability is a technical point,which if ignored, can lead to paradoxes.
For an example of a continuous RV with infinite mean, see the Cauchydistribution (Example G, page 114)
Expected Values and Moments 12
![Page 14: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/14.jpg)
As with the discrete case, E[X] can be thought as a measure of center ofthe random variable.
For example, when X ∼ U(0, 1)
E[X] =∫ 1
0
xdx = 0.5
−1.0 −0.5 0.0 0.5 1.0 1.5 2.0
0.0
0.2
0.4
0.6
0.8
1.0
PDF of U(0,1)
x
f(x)
Expected Values and Moments 13
![Page 15: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/15.jpg)
Not surprisingly, expectations of functions of continuous RVs satisfy theexpected relationship
E[g(X)] =∫ ∞
−∞g(x)f(x)dx
For example, if X ∼ U(0, 1),
E[X2] =∫ 1
0
x2dx =13
This is often easier than figuring out the PDF of Y = g(X) and applyingthe definition as there is often some work to figure out the PDF of Y .(Which we will do later, it does have its uses)
As with discrete RVs, g(E[X]) 6= E[g(X)] in most cases. However, with alinear transformation Y = a + bX
E[a + bX] = a + bE[X]
Expected Values and Moments 14
![Page 16: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/16.jpg)
Spread of a RV
−2 −1 0 1 2
X
P[X
=x]
0.00
0.10
0.20
0.30
−2 −1 0 1 2
X
P[X
=x]
0.00
0.10
0.20
0.30
x -1 0 1
p(x) 13
13
13
x -2 -1 0 1 2
p(x) 19
29
39
29
19
Expected Values and Moments 15
![Page 17: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/17.jpg)
−2 −1 0 1 2
0.0
0.1
0.2
0.3
0.4
0.5
x
f(x)
−2 −1 0 1 2
0.0
0.1
0.2
0.3
0.4
0.5
x
f(x)
f(x) =
{0.5 −1 ≤ x ≤ 10 Otherwise
f(x) =
0.5 + x4 −2 ≤ x ≤ 0
0.5− x4 0 ≤ x ≤ 2
0 Otherwise
All these distributions have E[X] = 0 but the right hand side in each casehas a bigger spread. A common measure of spread is the Standard Deviation
Expected Values and Moments 16
![Page 18: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/18.jpg)
Definition: Let µ = E[X], then the Variance of the random variable X is
Var(X) = E[(X − µ)2]
provided the expectation exists.
The Standard Deviation of X is
SD(X) =√
Var(X)
For a discrete RV,Var(X) =
∑
i
(xi − µ)2p(xi)
For a continuous RV
Var(X) =∫ ∞
−∞(x− µ)2f(x)dx
Expected Values and Moments 17
![Page 19: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/19.jpg)
The variance measures the expected squared difference of an observationfrom the mean. While the interpretation of the standard deviation isn’tquite easy, it can be thought of a measure of the typical spread of a RV.
It can be shown that, assuming that the variance exists,
Var(X) = E[X2]− (E[X])2
This form is often useful for calculation purposes.
Notation: The variance is often denoted by σ2 and the standard deviationby σ.
Expected Values and Moments 18
![Page 20: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/20.jpg)
For the examples
−2 −1 0 1 2
X
P[X
=x]
0.00
0.10
0.20
0.30
x −1 0 1
p(x) 13
13
13
Var(X) = (−1− 0)213
+ (0− 0)213
+ (1− 0)213
=23
SD(X) =
√23
= 0.8165
Expected Values and Moments 19
![Page 21: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/21.jpg)
−2 −1 0 1 2
X
P[X
=x]
0.00
0.10
0.20
0.30 x −2 −1 0 1 2
p(x) 19
29
39
29
19
Var(X) = (−2− 0)219
+ (−1− 0)229
+ (0− 0)239
+ (1− 0)229
+ (2− 0)219
=109
SD(X) =
√109
= 1.0541
Expected Values and Moments 20
![Page 22: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/22.jpg)
−2 −1 0 1 2
0.0
0.1
0.2
0.3
0.4
0.5
x
f(x)
f(x) =
{0.5 −1 ≤ x ≤ 10 Otherwise
Var(X) =∫ 1
−1
(x− 0)212dx =
13
SD(X) =
√13
= 0.5774
Expected Values and Moments 21
![Page 23: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/23.jpg)
−2 −1 0 1 2
0.0
0.1
0.2
0.3
0.4
0.5
x
f(x)
f(x) =
0.5 + x4 −2 ≤ x ≤ 0
0.5− x4 0 ≤ x ≤ 2
0 Otherwise
Var(X) = 2∫ 2
0
(x− 0)2(0.5− x
4)dx =
43
SD(X) =
√43
= 1.1547
Expected Values and Moments 22
![Page 24: Continuous Random Variables Expected Values and Momentsmarkirwin.net/stat110/Lecture/Section22.pdf · that we want the standard deviation to scale by a factor of jbj. Since the variance](https://reader030.vdocuments.us/reader030/viewer/2022041015/5ec6ae8bfa78e972cd305e45/html5/thumbnails/24.jpg)
What is the effect of a linear transformation (Y = a + bX) on the varianceand standard deviation?
Var(a + bX) = b2Var(X) SD(a + bX) = |b|SD(X)
These two results are to be expected. For example, if two possible X valuesdiffer by d = |x1−x2|, the corresponding Y values differ by |b|d, suggestingthat we want the standard deviation to scale by a factor of |b|. Since thevariance measures squared spread, it needs to scale by a factor of b2.
The factor a not having an effect also makes sense. Adding a to a randomvariable shifts the location of its distribution, but doesn’t changes thedistance between corresponding pairs of points.
Expected Values and Moments 23