probability and statistics for engineering
DESCRIPTION
PROBABILITY AND STATISTICS FOR ENGINEERING. Hossein Sameti Department of Computer Engineering Sharif University of Technology. Let X represent a Binomial r.v ,Then from for large n . In this context, two approximations are extremely useful. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/1.jpg)
PROBABILITY AND STATISTICS FOR ENGINEERING
Hossein Sameti
Department of Computer EngineeringSharif University of Technology
![Page 2: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/2.jpg)
Let X represent a Binomial r.v ,Then from
for large n. In this context, two approximations are extremely useful.
2
1
2
1
.)(21
k
kk
knkk
kkn qp
kn
kPkXkP
.)()(
) " and between is of sOccurrence ("2
1
2
1
211 1
21
k
kk
knkk
kkkkkk qp
kn
XPXXXP
kkAP
![Page 3: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/3.jpg)
The Normal Approximation (Demoivre-Laplace Theorem)
If hen for k in the neighborhood of np, we can approximate
(for example, when with p held fixed) And we have:
where
n
npq
.2
1 2/)( 2 npqnpkknk enpq
qpkn
,21
21 2/
2/)(
21
22
1
22
1
dyedxenpq
kXkP yx
x
npqnpxk
k
. , 22
11 npq
npkxnpq
npkx
![Page 4: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/4.jpg)
As we know,
If and are within with approximation:
where
We can express this formula in terms of the normalized integral
that has been tabulated extensively.
2
1
2
1
)(21
k
kk
knkk
kkn qp
kn
kPkXkP
1k 2k , , npqnpnpqnp
,21
21 2/
2/)(
21
22
1
22
1
dyedxenpq
kXkP yx
x
npqnpxk
k
)(21)(
0
2/2
xerfdyexerfx y
. , 22
11 npq
npkxnpq
npkx
![Page 5: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/5.jpg)
x erf(x) x erf(x) x erf(x) x erf(x)
0.05
0.10 0.15 0.20 0.25
0.30 0.35 0.40 0.45 0.50
0.55 0.60 0.65 0.70 0.75
0.01994 0.03983 0.05962 0.07926 0.09871 0.11791 0.13683 0.15542 0.17364 0.19146 0.20884 0.22575 0.24215 0.25804 0.27337
0.80 0.85 0.90 0.95 1.00
1.05 1.10 1.15 1.20 1.25
1.30 1.35 1.40 1.45 1.50
0.28814 0.30234 0.31594 0.32894 0.34134 0.35314 0.36433 0.37493 0.38493 0.39435 0.40320 0.41149 0.41924 0.42647 0.43319
1.55
1.60 1.65 1.70 1.75
1.80 1.85 1.90 1.95 2.00
2.05 2.10 2.15 2.20 2.25
0.43943 0.44520 0.45053 0.45543 0.45994 0.46407 0.46784 0.47128 0.47441 0.47726 0.47982 0.48214 0.48422 0.48610 0.48778
2.30
2.35 2.40 2.45 2.50
2.55 2.60 2.65 2.70 2.75
2.80 2.85 2.90 2.95 3.00
0.48928 0.49061 0.49180 0.49286 0.49379 0.49461 0.49534 0.49597 0.49653 0.49702 0.49744 0.49781 0.49813 0.49841 0.49865
![Page 6: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/6.jpg)
A fair coin is tossed 5,000 times. Find the probability that the number of heads is between 2,475 to 2,525.
We need
Since n is large we can use the normal approximation. so that and
and So the approximation is valid for and
Example
Solution
).525,2475,2( XP
,21
p 500,2np .35npq
,465,2 npqnp475,21 k .525,22 k
,535,2 npqnp
![Page 7: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/7.jpg)
Here,
Using the table,
Example - continued
2
1
2
.21 2/
21
x
x
y dyekXkP
.75 ,
75 2
21
1
npq
npkxnpq
npkx
516.075erf2
|)(|erf)(erf )(erf)(erf525,2475,2
12
12
xxxxXP
. 258.0)7.0(erf
![Page 8: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/8.jpg)
The Poisson Approximation
For large n, the Gaussian approximation of a binomial r.v is valid only if p is fixed, i.e., only if and
What if is small, or if it does not increase with n?- for example when , such that is a fixed
number.
1np .1npq
np0p ,n np
![Page 9: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/9.jpg)
The Poisson Theorem
If
Then
,0p ,n np
ek
ppkkn
nkPk
knkn !
)1(!)!(
!)( 𝑛→ ∞→
![Page 10: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/10.jpg)
The Poisson Approximation
Consider random arrivals such as telephone calls over a line.
n : total number of calls in the interval as we have Suppose Δ : a small interval of duration Let be the probability of k calls in the interval Δ We have shown that
).,0( TT n
.Tn 0 T
1 2 n
𝑃𝑛 (𝑘 )=𝑃 (𝑘𝑖𝑛∆ )=(𝑛𝑘)𝑝𝑘𝑞𝑛−𝑘𝑤h𝑒𝑟𝑒𝑝= ∆𝑇
![Page 11: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/11.jpg)
The Poisson Approximation
p: probability of a particular call occurring in Δ: as
Normal approximation is invalid here because is not valid.
: probability of obtaining k calls (in any order) in an interval of duration Δ ,
.T
p
0p .T
T
Tnp
knkn pp
kknnkP
)1(!)!(
!)(
0 T
1 2 n
![Page 12: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/12.jpg)
The Poisson Approximation
Thus,
.)/1()/1(
!112111
)/1(!)()1()1()(
k
nk
knk
kn
nn
knk
nn
nnpk
npn
knnnkP
e
kkP
k
nnppn !)(lim
,0 ,the Poisson p.m.f
![Page 13: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/13.jpg)
Suppose - two million lottery tickets are issued - with 100 winning tickets among them.
a) If a person purchases 100 tickets, what is the probability of winning?
Example: Winning a Lottery
Solution
.105102
100 ticketsof no. Total
tickets winningof no. Total 56
pThe probability of buying a winning ticket
![Page 14: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/14.jpg)
X: number of winning tickets
n: number of purchased tickets ,
P: an approximate Poisson distribution with parameter
So, The Probability of winning is:
Winning a Lottery - continued
.005.0105100 5 np
,!
)(k
ekXPk
.005.01)0(1)1( eXPXP
![Page 15: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/15.jpg)
b) How many tickets should one buy to be 95% confident of having a winning ticket?
we need
But or
Thus one needs to buy about 60,000 tickets to be 95% confident of having a winning ticket!
Winning a Lottery - continued
Solution
.95.0)1( XP
.320ln implies 95.01)1( eXP
3105 5 nnp .000,60n
![Page 16: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/16.jpg)
A space craft has 100,000 components The probability of any one component being defective is The mission will be in danger if five or more components become defective. Find the probability of such an event.
n is large and p is small Poisson Approximation with parameter
Example: Danger in Space Mission
Solution
n
).0(102 5 p
2102000,100 5 np
.052.032
342211
!1
!1)4(1)5(
2
4
0
24
0
e
ke
keXPXP
k
kk
k
![Page 17: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/17.jpg)
Conditional Probability Density Function
, )( )( xXPxFX
.0)( ,)(
)()|(
BPBP
BAPBAP
.)(
)( |)( )|(BP
BxXPBxXPBxFX
.0)()(
)( )( )|(
,1)()(
)( )( )|(
BPP
BPBXPBF
BPBP
BPBXPBF
X
X
![Page 18: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/18.jpg)
Conditional Probability Density Function
Further,
Since for
),|()|( )(
)( )|)((
12
2121
BxFBxFBP
BxXxPBxXxP
XX
,12 xx
. )()()( 2112 xXxxXxX
,
)|()|(dx
BxdFBxf XX
2
1
21 .)|(|)(x
x X dxBxfBxXxP
x
XX duBufBxF
.)|()|(
![Page 19: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/19.jpg)
Toss a coin and X(T)=0, X(H)=1. Suppose Determine
has the following form. We need for all x. For so that and
Example
Solution
}.{HB).|( BxFX
)(xFX
)|( BxFX
,)( ,0 xXx ,)( BxX.0)|( BxFX
)(xFX
x
(a)
q1
1
( | )XF x B
x
(b)
1
1
![Page 20: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/20.jpg)
For so that
For and
Example - continued
, )( ,10 TxXx
HTBxX )( .0)|( and BxFX
,)( ,1 xXx
}{ )( BBBxX 1)()()|( and
BPBPBxFX
( | )XF x B
x
1
1
![Page 21: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/21.jpg)
Given suppose Find
We will first determine
For so that
For so that
Example
Solution
),(xFX .)( aXB ).|( BxfX
).|( BxFX
. )|(
aXPaXxXPBxFX
xXaXxXax ,
.
)()()|(
aFxF
aXPxXPBxF
X
XX
)( , aXaXxXax .1)|( BxFX
![Page 22: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/22.jpg)
Thus
and hence
Example - continued
, ,1
, ,)()(
)|(ax
axaFxF
BxFX
X
X
otherwise. ,0
,,)()(
)|()|( axaFxf
BxFdxdBxf
X
X
XX
)|( BxFX
)(xFX
xa
1
(a)
)|( BxfX
)(xfX
xa(b)
![Page 23: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/23.jpg)
Let B represent the event with For a given determine and
Example
Solution
.
)()()( )(
)( )()(
|)( )|(
aFbFbXaxXP
bXaPbXaxXP
BxXPBxF
XX
X
bXa )( .ab),(xFX )|( BxFX ).|( BxfX
![Page 24: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/24.jpg)
For we have and hence
For we have and hence
For we have so that
Thus,
Example - continued
,bxa })({ )( )( xXabXaxX
.)()()()(
)()()()|(
aFbFaFxF
aFbFxXaPBxF
XX
XX
XXX
,bx bXabXaxX )( )( )( .1)|( BxFX
,ax ,)( )( bXaxX.0)|( BxFX
otherwise.,0
,,)()(
)()|( bxa
aFbFxf
BxfXX
X
X
)|( BxfX
)(xfX
xa b
![Page 25: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/25.jpg)
Conditional p.d.f & Bayes’ Theorem First, we extend the conditional probability results to random variables: We know that If is a partition of S and B is an arbitrary event,
then:
By setting we obtain:
)()|()()|()( 11 nn APABPAPABPBP
],,,[ 21 nAAAU
Son partition a form ,
)()|()()|()(
)()|()()|()(
Hence
)()|()()|()(
1
11
11
11
n
nn
nn
nn
AA
APAxfAPAxfxf
APAxFAPAxFxF
APAxXPAPAxXPxXP
}{ xXB
![Page 26: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/26.jpg)
Conditional p.d.f & Bayes’ Theorem
Using:
We obtain:
For ,
.)(
)()|()|(BP
APABPBAP
)(
| )(
|)| AP
xFAxF
APxXP
AxXPxXAP
21 )( xXxB
).()(
)|()(
)()()|()|(
)(|
|
2
1
2
1
12
12
21
2121
APdxxf
dxAxfAP
xFxFAxFAxF
APxXxP
AxXxPxXxAP
x
x X
x
x X
XX
XX
![Page 27: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/27.jpg)
Conditional p.d.f & Bayes’ Theorem
Let so that in the limit as
or
we also get
or
,0 , , 21 xxxx ,0
).()(
)|(|)(|lim
0AP
xfAxf
xXAPxXxAPX
X
.)(
)()|()|(| APxfxXAPAxf X
AX
,)()|()|()(1
dxxfxXAPdxAxfAP XX
dxxfxXAPAP X )()|()(
(Total Probability Theorem)
![Page 28: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/28.jpg)
Bayes’ Theorem (continuous version)
using total probability theorem in
We get the desired result
,)(
)()|()|(| APxfxXAPAxf X
AX
.)()|(
)()|()|(|
dxxfxXAP
xfxXAPAxfX
XAX
![Page 29: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/29.jpg)
probability of obtaining a head in a toss. For a given coin, a-priori p can possess any value in (0,1). : A uniform in the absence of any additional information After tossing the coin n times, k heads are observed. How can we update this is new information?
Let A= “k heads in n specific tosses”. Since these tosses result in a specific sequence,
and using Total Probability Theorem we get
Example: Coin Tossing Problem Revisited
Solution
)(HPp
)( pfP
)(pfP
,)|( knkqppPAP
)( pfP
p0 1
.)!1(
!)!()1()()|()(1
0
1
0
nkkndpppdppfpPAPAP knk
P
![Page 30: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/30.jpg)
The a-posteriori p.d.f represents the updated information given the event A,
Using
This is a beta distribution. We can use this a-posteriori p.d.f to make further predictions. For example, in the light of the above experiment, what can we say about the
probability of a head occurring in the next (n+1)th toss?
Example - continued
| ( | )P Af p A
,)(
)()|()|(| APxfxXAPAxf X
AX
|( | ) ( )( | )
( )( 1)! , 0 1 ( , ).
( )! !
PP A
k n k
P A P p f pf p AP A
n p q p n kn k k
)|(| Apf AP
p10
![Page 31: PROBABILITY AND STATISTICS FOR ENGINEERING](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815dfd550346895dcc3a68/html5/thumbnails/31.jpg)
Let B= “head occurring in the (n+1)th toss, given that k heads have occurred in n previous tosses”.
Clearly From Total Probability Theorem,
Using (1) in (2), we get:
Thus, if n =10, and k = 6, then
which is more realistic compared to p = 0.5.
Example - continued
,)|( ppPBP
1
0 .)|()|()( dpApfpPBPBP P
,58.0127)( BP
1
0
( 1)! 1( ) .( )! ! 2
k n kn kP B p p q dpn k k n