1

Probability

09. Limit

Infinity

郭俊利 2010/05/11

2

ProbabilityOutline

Estimation error Inequalities Large numbers Convergence Central limit

theorem

4.6 ~ 5.4

3

ProbabilityLinear Estimator

^X = E[X| Y] ≒ a Y + b

If the measurement Y = X2, X is uniformly in [0, 2], what is the optimal estimate after a measured value = 2.25 ?

^X = Y½ ≒ 1/3 Y + 2/3^X = 1.5 ≒ 1.416

4

ProbabilityLinear Least Estimator

^X = E[X| Y] a Y + b≒

a = cov(X, Y) / var(Y) = ρ(σX / σY)b = E[X] – a E[Y]E[~X] = (1 – ρ2) σX

2

σ is standard deviations, ρ is correlation coefficient

5

ProbabilityLinear Least Squares Estimator

X = aY + bX – aY = bE[b] = E[X – aY] ↓b = E[X] – a E[Y] ↓

~X = X – ^X Liner

= X – aY – bE[~X] = E[X – aY – b] ↓ = var(X – aY) ↓

a = cov(X, Y) / var(Y) = ρ(σX / σY)

E[~X] = (1 – ρ2) σX2

E[(X – ^X Liner) Y] = 0

6

ProbabilityLinear Estimated Example (1/2)

X = Y½ ^X Liner = 1/2 Y +1/3

a = ρ(σX / σY)ρ = cov(X, Y) / σXσYσ = √var = √E[2] – E[]2, cov(X, Y) = E[XY] – E[X] E[Y]

E[X] = 1, E[Y] = E[X2] = 4/3, E[X3] = 2, E[X4] = 16/5

cov(X, Y) = 2 – 1*4/3 = 2/3var(X) = 1/3, var(Y) 1.42≒

ρ 0.97 ≒ a 1/2≒

b = E[X] – a E[Y] = 1/3^X Liner = 1/2 Y +1/3

E[~X] = (1 – ρ2) σX2 0.02≒

7

ProbabilityLinear Estimated Example (2/2)

0 0.666667 0.333333 0 0.666667 0.3333330.01 0.67 0.338333 0.1 0.57 0.2383330.04 0.68 0.353333 0.2 0.48 0.1533330.09 0.696667 0.378333 0.3 0.396667 0.0783330.16 0.72 0.413333 0.4 0.32 0.0133330.25 0.75 0.458333 0.5 0.25 -0.041670.36 0.786667 0.513333 0.6 0.186667 -0.086670.49 0.83 0.578333 0.7 0.13 -0.121670.64 0.88 0.653333 0.8 0.08 -0.146670.81 0.936667 0.738333 0.9 0.036667 -0.16167

1 1 0.833333 1 0 -0.166671.21 1.07 0.938333 1.1 -0.03 -0.161671.44 1.146667 1.053333 1.2 -0.05333 -0.146671.69 1.23 1.178333 1.3 -0.07 -0.121671.96 1.32 1.313333 1.4 -0.08 -0.086672.25 1.416667 1.458333 1.5 -0.08333 -0.041672.56 1.52 1.613333 1.6 -0.08 0.0133332.89 1.63 1.778333 1.7 -0.07 0.0783333.24 1.746667 1.953333 1.8 -0.05333 0.1533333.61 1.87 2.138333 1.9 -0.03 0.238333

4 2 2.333333 2 0 0.333333

Y X1/3 Y + 2/3 1/2 Y +1/3 ~X

8

ProbabilityInequalities (1/4)

Markov

Chebyshev

When 0 ≤ E[X] ≤ a, Markov is informative;When 0 ≤ σ ≤ c, Chebyshev is informative.

If Chebyshev is informative, Chebyshev is exacter than Markov

P(X ≥ a) ≤E[X]

a

P(|X – μ| ≥ c) ≤σ2

c2

9

ProbabilityInequalities (2/4)

X is uniformly distributed in [0, 4]

Exact probabilities P(X ≥ 2) = 0.5 P(X ≥ 3) = 0.25 P(X ≥ 4) = 0

Markov P(X ≥ 2) ≤ 1 P(X ≥ 3) ≤ 2/3 P(X ≥ 4) ≤ 1/2

Chebyshev P(X ≥ 3) ≤ 4/3 ! P(X ≥ 4) ≤ 1/3

10

ProbabilityInequalities (3/4)

Xiao-Ming can type X words/minute and X is random variable. The average is 50 words/minute.(a) P(X ≥ 75) ≤

Real probability < 1/4

(b) If σX = 5, P(40 ≤ X ≤ 60) ≥

11

ProbabilityInequalities (4/4)

Both Markov and Chebyshev are suitable for exponential distribution, but not for normal distribution.

X is exponentially distributed with λ = 1. For c > 1, to find P(X > c). Exact probability = e–c

Markov ≤ 1 / c Chebyshev ≤ 1 / (c – 1)2

12

ProbabilityConvergence (1/4)

Sequence Xn Constant cn ∞ lim Xn = c

Mn = (ΣXi) / n = Sn / n E[Mn] = μ; var(Mn) = σ2/n

From ChebyshevP(| Mn – μ| ≥ε) ≤ 0

for all ε > 0, Mn converges to μ

σ2

nε2

13

ProbabilityConvergence (2/4)

lim P(| Yn – a | ≥ε) = 0

Xiao-Bao calculate Yn = min{X1, …, Xn}, Xi is uniformly in [0, 1], find the converged value of Yn.

n ∞

P(|Yn – 0| ≥ ε) = P(X1 ≥ ε, …, Xn ≥ ε) = P(X1 ≥ ε) … P(Xn ≥ ε) = (1 – ε)n

∵

∴ lim P(| Yn – 0 | ≥ε) = lim (1 – ε)n = 0

For all ε,ε = 0 ~ 1

14

ProbabilityConvergence (3/4)

If Xiao-Bao guess wrong, he gets the converged value = ½, what happened after finishing calculating?

P(|Yn – ½| ≥ ε) = P(Yn – ½ ≥ ε ½ – Y∪ n ≥ ε) = P(Yn ≥ ε+ ½) + P(Yn ≤ ½ – ε) = (½ – ε)n + (½ – ε)n

∵

ε = 0 ~ ½

∴ lim P(| Yn – 0 | ≥ε) when ε = ½ ~ 1 cannot support.

15

ProbabilityConvergence (4/4)

Yn = max{X1, …, Xn}, Xi is uniformly in [-1, 1], find the converged value of Yn.

= P(Yn ≥ ε + 1) + P(Yn ≤ 1 – ε)

= P(max(Xi) ≥ ε + 1) + P(max(Xi) ≤ 1 – ε)

= P(max(Xi) ≤ 1 – ε)

= [ P(X ≤ 1 – ε) ]n

= (1 – ½ ε)n

P(|Yn – 1| ≥ ε)

ε = 0 ~ 1

lim (1 – ½ ε)n = 0

16

ProbabilityCentral Limit

Let X1, X2, … be a sequence of independent random variables with mean μ and variance σ2

CDF Zn converges to the standard normal CDF

Zn =X1 + … + Xn – nμ

σ√n

N(z) =1/ √2π ∫e –x /2 dx2

lim∞ P(Zn ≤ z) = N(z)

17

ProbabilityDe Moivre - Laplace

P(k ≤ Sn ≤ l) N( )≒

– N( )

Xiao-Hua answers 100 questions and the probability of his correctness is 0.8. P(X ≥ 70) P(S ≒ ≥ 69.5) N(–2.875)≒ P(X > 70) P(S ≒ ≥ 70.5) P(X ≤ 85) N(1.375)≒ P(X = 88) N(8.5/4) – N(7.5/4)≒

l – np +1/2

√np(1 – p)

k – np – 1/2

√np(1 – p)