![Page 1: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/1.jpg)
Chanan SinghTexas A&M University
Module 2-4Review of Probability
Theory
Copyright by Chanan Singh For educational purpose only
![Page 2: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/2.jpg)
PrXt = 3 = 0.2
PrXt = 2 = 0.4
PrXt = 1 = 0.4
A Markov ProcessThe probability distribution of Xt+1 is dependent only on the current state Xt.
Xt Xt+1
3
2
1
3
2
1
PrXt+1 = 3 = ?
PrXt+1 = 2 = ?
PrXt+1 = 1 = ?
Use concept of transition rate!
Copyright by Chanan Singh For educational purpose only
![Page 3: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/3.jpg)
Transition Probability
l Let Xij be the random variable of time duration from state i to j, then
Pij(Δt) = Pr t <Xij < t+ Δt | Xij > t l From hazard rate as Δt → 0,
λijΔt = Pr t <Xij < t+ Δt | Xij > t
Pij(Δ t) = λijΔt
Probability of being in state j at time t, given that the process started in state i, Pij (t)
Copyright by Chanan Singh For educational purpose only
![Page 4: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/4.jpg)
A Three-State Systeml Assume that the system started in state 1, find P13(t+Δt).
t = 0 Xt+Δt
3
2
1
3
2
1
Xt
1
2
3
Copyright by Chanan Singh For educational purpose only
![Page 5: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/5.jpg)
l From conditional probability as Δt → 0,P13(t+Δt) = + P12(t) P23(Δt)P11(t) P13(Δt) + P13(t) P33(Δt)
= P11(t) λ13 Δt + P12(t) λ23 Δt + P13(t) λ33 Δt
t = 0 Xt+Δt
3
2
1
3
2
1
Xt
1
2
3
Copyright by Chanan Singh For educational purpose only
![Page 6: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/6.jpg)
l Since P31(Δt) + P32(Δt) + P33(Δt) = 1, thenλ33 Δt + λ32 Δt + λ31 Δt = 1λ33 Δt = 1 - λ32 Δt - λ31 Δt
l We have, P13(t+Δt) = P11(t)λ13 Δt + P12(t) λ23 Δt
+ P13(t) (1 - λ32 Δt - λ31 Δt) l Then, [P13(t+Δt) - P13(t) ]/ Δt
= P11(t) λ13 + P12(t) λ23 + P13(t) (- λ31 -λ32) = P’13(t)
Copyright by Chanan Singh For educational purpose only
![Page 7: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/7.jpg)
l Thus,
l Wherel Similarly, we have
( ) ( ) ( ) ( )13
13 11 12 13 23
33
P t P t P t P tλ
λ
λ
⎡ ⎤⎢ ⎥ʹ′ ⎡ ⎤= ⎣ ⎦ ⎢ ⎥⎢ ⎥⎣ ⎦
( )33 31 32λ λ λ= − +
( ) ( ) ( )[ ] ( ) ( ) ( )[ ]⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=ʹ′ʹ′ʹ′
333231
232221
131211
131211131211
λλλ
λλλ
λλλ
tPtPtPtPtPtP
Copyright by Chanan Singh For educational purpose only
![Page 8: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/8.jpg)
Transition Rate Matrix
l λij is transition rate from state i to j, i ≠ j.l where λii = -Σ λij
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=
333231
232221
131211
λλλ
λλλ
λλλ
R
Copyright by Chanan Singh For educational purpose only
![Page 9: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/9.jpg)
Steady State Probabilityl From differential equation, at steady state the derivatives are zero,
l In addition, ΣP = 1,
[ ] [ ]RPPP 321000 =
1321 =++ PPP
Copyright by Chanan Singh For educational purpose only
![Page 10: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/10.jpg)
Transient Behavior
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 11: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/11.jpg)
Transient Behavior
Copyright by Chanan Singh, For educational use only - not for sale
It should be noted that the above equation implieshe approximation of a Markov process in continuous timeby a discrete time Markov process with steps equal to ∆𝑡
Copyright by Chanan Singh For educational purpose only
![Page 12: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/12.jpg)
Example
One of the processes commonly encountered in reliability studies is the two state Markov process. The state transition diagram for this process is shown below
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 13: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/13.jpg)
Example
Transition rate matrix
Take Laplace transform
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 14: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/14.jpg)
Example
The probability vector can be obtained
As 𝑡 → ∞
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 15: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/15.jpg)
Example
The steady state probability can also be obtained as
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 16: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/16.jpg)
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 17: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/17.jpg)
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
The kth moment of the first passage time can be found bypreviously mentioned transform method
Copyright by Chanan Singh For educational purpose only
![Page 18: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/18.jpg)
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 19: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/19.jpg)
Example System
Copyright by Chanan Singh, For educational use only - not for sale
l Two generators are identical;; each generator has generation capacity of 100MW, failure rate λ_G=0.1/day, repair rate μ_G=2/day
l Transmission line’s failure rate λ_T=0.01/day, repair rate μ_G=4/dayl Load is 100MWDefine failure of this system as load being not supplied. Assume all the failures of components are independent
Copyright by Chanan Singh For educational purpose only
![Page 20: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/20.jpg)
Example System
Copyright by Chanan Singh, For educational use only - not for sale
State transition diagram
Copyright by Chanan Singh For educational purpose only
![Page 21: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/21.jpg)
Example System
Copyright by Chanan Singh, For educational use only - not for sale
Transition rate matrix
𝑅
=
−(2𝜆+ +𝜆-) 𝜆+ 𝜆+ 0 𝜆- 0 0 0𝜇+ −(𝜆+ +𝜆- + 𝜇+) 0 𝜆+ 0 𝜆- 0 0𝜇+ 0 −(𝜆+ + 𝜆- +𝜇+) 𝜆+ 0 0 𝜆- 00 𝜇+ 𝜇+ −(𝜆- + 2𝜇+) 0 0 0 𝜆-𝜇- 0 0 0 −(2𝜆+ +𝜇-) 𝜆+ 𝜆+ 00 𝜇- 0 0 𝜇+ −(𝜆+ + 𝜇+ +𝜇-) 0 𝜆+0 0 𝜇- 0 𝜇+ 0 −(𝜆+ + 𝜇+ +𝜇-) 𝜆+0 0 0 𝜇- 0 𝜇+ 𝜇+ −(2𝜇+ + 𝜇-)
Copyright by Chanan Singh For educational purpose only
![Page 22: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/22.jpg)
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 23: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/23.jpg)
The probability of being in subset 𝑋2 at time t is 𝑝2(𝑡)𝑈526Where 𝑈526 is a unit column vector of dimension N-JPreviously, we have obtained
Thus the Laplace transform of the probability density function of the first passage time is
𝑓 𝑠 = 𝑠𝑝2 𝑠 𝑈526 = 𝑝;(0)(𝑠𝐼 − 𝑅)2=𝑅=>𝑈526Since the rows of the transition rate matrix sum to zero
𝑅=>𝑈526 + 𝑅==𝑈6 = 0Therefore
𝑓 𝑠 = 𝑝;(0)(𝑠𝐼 − 𝑅==)2=(−𝑅==)𝑈6
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 24: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/24.jpg)
𝑓 𝑠 = 𝑝;(0)(𝑠𝐼 − 𝑅==)2=(−𝑅==)𝑈6
The rth initial moment can be found 𝑇(@) = 𝑘! 𝑝;(0)(−𝑅==)2@𝑈6
The mean is𝑇C = 𝑇(=) = 𝑝;(0)(−𝑅==)2=𝑈6
If the process started in the first state𝑇C = 𝑇(=) = (1 0 0 … 0)(−𝑅==)2=𝑈6
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 25: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/25.jpg)
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
𝑇C = 𝑇(=) = 𝑝;(0)(−𝑅==)2=𝑈6
If 𝑋2 represents the failed condition, then 𝑇C is the MTTFF. It should be realized that 𝑇C = 𝑇(=) = 𝑝;(0)(−𝑅==)2=𝑈6 can be derived from the theory of discrete time Markov chains by assuming that each step of the chain is ∆𝑡 ≈ 0.The matrix of one step transition probability becomes [𝐼 + 𝑅∆𝑡] and by truncating the absorbing state, Q = [𝐼 + 𝑅==∆𝑡] and therefor the fundamental matrix
Copyright by Chanan Singh For educational purpose only
![Page 26: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/26.jpg)
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
Here𝜆JK is the transition rate from state I to state j
Copyright by Chanan Singh For educational purpose only
![Page 27: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/27.jpg)
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
The mean stay in 𝑋; is
Copyright by Chanan Singh For educational purpose only
![Page 28: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/28.jpg)
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
Copyright by Chanan Singh For educational purpose only
![Page 29: Module’2)4 Review’of’Probability’ Theory’ece.tamu.edu/~c-singh/coursenotes/module_2-4.pdf · Module_2-4 Review of Probability Theory IV Created Date: 11/11/2015 6:11:36](https://reader034.vdocuments.us/reader034/viewer/2022042419/5f358c48209c8828ff2a9dcb/html5/thumbnails/29.jpg)
First Passage Times
Copyright by Chanan Singh, For educational use only - not for sale
These relationships will be derived from the frequency balancing technique in following lectures
Copyright by Chanan Singh For educational purpose only