lecture01 02 intro probability theory
Post on 16-Feb-2018
224 Views
Preview:
TRANSCRIPT
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 1/62
Copyright © Syed Ali Khayam 2009
CSE801 Analysis of Stochastic
Systems
Welcome and Introduction
Dr. Muhammad Usman IlyasSchool of Electrical Engineering & Computer Science (SEECS)
National University of Sciences & Technology (NUST)
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 2/62
Copyright © Syed Ali Khayam 2008
Course Information
Lecture Timings:
Tuesday: 5:30pm-7:20pm Thursday: 5:30pm-6:20pm
My Office:
Room # A-312
Office Hours
Thursdays, 5:00-5:30pm, or by appointment.
usman.ilyas@seecs.edu.pk
The course will be managed through LMS
www.lms.nust.edu.pk
Facebook group2
The lecture notes are designed and developed by Dr. Ali Khayam.
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 3/62
Copyright © Syed Ali Khayam 2008
Timetable (MS EE-Telecom & Comp Networks)
3
Class Room # 5
TIME / DAYS Monday Tuesday Wednesday Thursday Friday Saturday
5:30pm-6:20pm Adv. Computer
NetworksStochastic Systems
Library/Make-up
Class/Seminar
Stochastic Systems Adv. Digital
Commnication
Library/Make-up
Class/Seminar
6:30pm-7:20pm Adv. Digital
Commnication Adv. Computer
Networks7:30pm-8:20pm
Library/Make-up
Class/Seminar
Library/Make-up
Class/Seminar
Library/Make-up
Class/Seminar
8:30pm-9:20pmLibrary/Make-up
Class/Seminar
course Code Subject Instructor Credit Hours
MS EE(Digital System and Signal Processing)-5
CSE 801 Stochastic Systems Dr. Shahzad Younis 3+0
EE 831 Advanced Digital
Signal ProcessingDr. Amir Ali Khan 3+0
EE 823 Advanced Digital
System DesignDr. Rehan Hafiz 3+0
MS EE(Telecommnication & Computer Networks)-5
CSE-801 Stochastic Systems Dr. Usman Illyas 3+0
CSE-820 Adv. Computer
NetworksDr. Junaid Qadir 3+0 Manager-PG
Academic Coord Branch
(Iftikhar Ahmed)
Sep 2013
EE-851 Adv. Digital
CommnicationDr. Rizwan Ahmed 3+0
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 4/62
Copyright © Syed Ali Khayam 2008
Timetable (MS EE-Power & Control)
4
MS EE(Power & Control )-1First Semester (9 Sep 2013 - 10 Jan 2014)
Class Room # 7
TIME / DAYS Monday Tuesday Wednesday Thursday Friday Saturday
5:30pm-6:20pmPower Electronics and
Electric Drives
Stochastic Systems
CR #5
Power Electronics
and
Electric Drives
Stochastic
Systems
CR #5
Library/Make-up
Class/Seminar
Library/Make-up
Class/Seminar
6:30pm-7:20pmLinear Control
Systems
Linear ControlSystems
7:30pm-8:20pmLibrary/Make-up
Class/Seminar
Library/Make-up
Class/Seminar
Library/Make-up
Class/Seminar8:30pm-9:20pm
Library/Make-up
Class/Seminar
course Code Subject Instructor Credit Hours
MS-EE(RF & MW )-4
EE-901
Power Electronics and
Electric Drives
Dr. Syed Raza
Kazmi 3+0
EE-871
Linear Control
Systems
Dr. Ammar
Hassan 3+0
CSE-801 Stochastic Systems Dr. Usman ilyas 3+0
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 5/62
Copyright © Syed Ali Khayam 2008
Textbooks
5
Probability & Random Processes
for Electrical Engineers, 2nd or 3rd ed.
Albert Leon-Garcia
Introduction to Probability Models,
9th ed.
Sheldon M. Ross
Elements of Information Theory Thomas M. Cover and Joy
A. Thomas
Chaos Theory Tamed Garnett P. Williams
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 6/62
Copyright © Syed Ali Khayam 2008
Course Outline
Syllabus Introduction to Probability Theory
Random Variables
Limits and Inequalities
Central Limit Theorem
Application Area: Information Theory
Stochastic Processes
Prediction and Estimation
Markov Chain and Processes (time permitting)
Application Area: Chaos Theory
6
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 7/62Copyright © Syed Ali Khayam 2008
Grading (subject to change)
Final Exam: 45%
Midterm Exam: 30%
Quizzes: 15%
Homework Assignments: 10%
7
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 8/62Copyright © Syed Ali Khayam 2008
Policies
Quizzes will be unannounced
Late homework submissions will be accepted for up to 24 hourswith a 50% penalty.
Strong disciplinary action will be taken in case of plagiarism orcheating in exams, homework or quizzes.
Attendance:
Will be taken at the beginning of the class.
The current rules of the school will be followed (75%minimum requirement).
8
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 9/62Copyright © Syed Ali Khayam 2008
What will we cover in this lecture?
This lecture is intended to be an introduction to elementary
probability theory
We will cover:
Random Experiments and Random Variables
Axioms of Probability Mutual Exclusivity
Conditional Probability
Independence
Law of Total Probability
Bayes’ Theorem
9
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 10/62Copyright © Syed Ali Khayam 2008
Definition of Probability
Probability:
1 : the quality or state of being possible
2 : something (as an event or circumstance) that is possible
3 : the ratio of the number of outcomes in an exhaustive set ofequally likely outcomes that produce a given event to thetotal number of possible outcomes, the chance that a given
event will occur
We will revisit these definitions in a little bit …
10
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 11/62Copyright © Syed Ali Khayam 2008
Definition of a Random Experiment
A random experiment comprises of:
A procedure
An outcome
Procedure(e.g., flipping a coin)
Outcome
(e.g., the value
observed [head, tail] afterflipping the coin)
Sample Space
(Set of All Possible
Outcomes)
11
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 12/62Copyright © Syed Ali Khayam 2008
Definition of a Random Experiment:Outcomes, Events and the Sample Space
An outcome cannot be further decomposed into other outcomes{s1 = the value 1}, …, {s6 = the value 6}
An event is a set of outcomes that are of interest to us
A = {s: such that s is an even number}
The set of all possible outcomes, S, is called the sample space
S = {s1, s2, s3, s4, s5, s6}
12
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 13/62Copyright © Syed Ali Khayam 2008
s 1
s 2
s 3
s 4
s 5
s 6
S
13
Definition of a Random Experiment:Outcomes, Events and the Sample Space
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 14/62Copyright © Syed Ali Khayam 2008
Example of a Random Experiment: Experiment: Roll a fair dice once and record the
number of dots on the top face
S = {1, 2, 3, 4, 5, 6}
A = “the outcome is even” = {2, 4, 6}
B = “the outcome is greater than 4” = {5, 6}
14
Definition of a Random Experiment:Outcomes, Events and the Sample Space
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 15/62Copyright © Syed Ali Khayam 2008
Axioms of Probability
Probability of any event A is non-negative:
Pr{ A} ≥ 0
The probability that an outcome belongs to the sample space is 1:
Pr{S} = 1
The probability of the union of mutually exclusive events is equalto the sum of their probabilities:
If A1 ∩ A
2=Ø,
=> Pr{ A1 U A2} = Pr{ A1} + Pr{ A2}
15
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 16/62Copyright © Syed Ali Khayam 2008
Mutual Exclusivity
Are A1 and A2 mututally exclusive?
For mutually exclusive events A1, A2 … AN, we have:
s 1
s 2
s 3
s 4
s 5
s 6
S
A 1
A 2
Find Pr{ A1 U A2}and Pr{ A1}+Pr{ A2}
in the fair dice
example
16
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 17/62Copyright © Syed Ali Khayam 2008
Mutual Exclusivity
Discarding the condition of exclusivity, in general, we have:
Pr{ A1 U A2} = ??
s 1
s 2
s 3
s 4
s 5
s 6
S
17
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 18/62Copyright © Syed Ali Khayam 2008
Mutual Exclusivity
Discarding the condition of exclusivity, in general, we have:
Pr{ A1 U A2} = Pr{ A1} + Pr{ A2} – Pr{ A1 ∩ A2}
s 1
s 2
s 3
s 4
s 5
s 6
S
18
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 19/62Copyright © Syed Ali Khayam 2008
Conditional Probability
Given that event B has already occurred, what is the probability
that event A will occur? Given that event B has already occurred, reduces the sample
space of A
s1
s2
s3
s4
s5
s6
S
s1
s2
s3
s4
s5
s6
Event B has
already occurred
=> s2, s4, s3
cannot occur
S
19
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 20/62Copyright © Syed Ali Khayam 2008
Conditional Probability
Given that event B has already occurred, we define a new
conditional sample space that only contains B’s outcomes The new event space for A is the intersection of A and B:
Event space -> E A|B = A ∩ B
s1
s2
s3
s4
s5
s6
S
s1
s2
s3
s4
s5
s6
Event B has
already
occurred
S
What’s missing here? S|B = {s1, s5, s6}
E A|B= A ∩ B = {s6}20
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 21/62Copyright © Syed Ali Khayam 2008
Conditional Probability
The probability of an event A in the conditional sample space is:
Pr = ∩
Pr ={}
{} = /
/ =
s1
s2
s3
s4
s5
s6
S
s1
s2
s3
s4
s5
s6
Event B has
already
occurred
S
S|B = {s1, s5, s6}
21
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 22/62Copyright © Syed Ali Khayam 2008
Independence
Two events are independent if they do not provide any
information about each other:
(|) = ()
In other words, the fact that B has already happened does notaffect the probability of A’s outcomes
Implications:
(|) = () ∩
() = ()
( ∩ ) = () ()
22
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 23/62
Copyright © Syed Ali Khayam 2008
Independence: Example
Are events A and C independent?
Assume that all outcomes are equally likely
s4
s1
s2
s3
s6
s5
S
23
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 24/62
Copyright © Syed Ali Khayam 2008
Independence: Example
Are events A and C independent?
s4
s1
s2
s3
s6
s5
S
24
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 25/62
Copyright © Syed Ali Khayam 2008
Independence: Example
Are events A and C independent?
Pr{ A ∩ C } = Pr{s5} = 1/6
Pr{ A}Pr{C } = (3/6)x(2/6) = 1/6
Yes!
s4
s1
s2
s3
s6
s5
S
25
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 26/62
Copyright © Syed Ali Khayam 2008
Independence: Example
Are events A and B independent?
Assume that all outcomes are equally likely
s4
s1
s2
s3
s6
s5
S
26
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 27/62
Copyright © Syed Ali Khayam 2008
Independence: Example
Are events A and B independent?
Pr{ A ∩ B} = Pr{s5} = 1/6
Pr{ A}Pr{B} = (3/6)x(3/6) = ¼
No!
s4
s1
s2
s3
s6
s5
S
27
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 28/62
Copyright © Syed Ali Khayam 2008
Mutual Exclusivity and Independence
Experiment:
Roll a fair dice twice and record the dots on the top face:
= {(1,1), (1,2), (1,3), (1,4), (1,5), (1,6),
(2,1), (2,2), (2,3), (2,4), (2,5), (2,6),
(3,1), (3,2), (3,3), (3,4), (3,5), (3,6),
(4,1), (4,2), (4,3), (4,4), (4,5), (4,6),
(5,1), (5,2), (5,3), (5,4), (5,5), (5,6), (6,1), (6,2), (6,3), (6,4), (6,5), (6,6) }
28
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 29/62
Copyright © Syed Ali Khayam 2008
Define three events:
1 = “first roll gives an odd number”
2 = “second roll gives an odd number” = “the sum of the two rolls is odd”
Find the probability of using probability of 1 and 2
29
Mutual Exclusivity and Independence
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 30/62
Copyright © Syed Ali Khayam 2008
A 1
A 2
30
S = { (1,1), (1,2), (1,3), (1,4), (1,5), (1,6),
(2,1), (2,2), (2,3), (2,4), (2,5), (2,6),
(3,1), (3,2), (3,3), (3,4), (3,5), (3,6),
(4,1), (4,2), (4,3), (4,4), (4,5), (4,6),
(5,1), (5,2), (5,3), (5,4), (5,5), (5,6),
(6,1), (6,2), (6,3), (6,4), (6,5), (6,6) }
Mutual Exclusivity and Independence
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 31/62
Copyright © Syed Ali Khayam 200831
Mutual Exclusivity and Independence
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 32/62
Copyright © Syed Ali Khayam 200832
Mutual Exclusivity and Independence
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 33/62
Copyright © Syed Ali Khayam 200833
Mutual Exclusivity and Independence
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 34/62
Copyright © Syed Ali Khayam 200834
Mutual Exclusivity and Independence
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 35/62
Copyright © Syed Ali Khayam 200835
Mutual Exclusivity and Independence
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 36/62
Copyright © Syed Ali Khayam 200836
Mutual Exclusivity and Independence
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 37/62
Copyright © Syed Ali Khayam 2008
Recap
1. Outcomes, events and sample space:
2. For mutually exclusive events A1, A2 ,…, AN, we have:
3. In general, we have:
37
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 38/62
Copyright © Syed Ali Khayam 2008
4. Conditional probability reduces the sample space:
5. Two events A and B are independent only if
6. For independent events:
38
Recap
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 39/62
Copyright © Syed Ali Khayam 2008
Four “Rules of Thumb”
1. Whenever you see two events which have an OR relationship (i.e., event A or
event B), their joint event will be their union, { A U B}Example: On a binary channel, find the probability of error?
An error occurs when
A: “a 0 is transmitted and a 1 is received” OR
B: “a 1 is transmitted and a 0 is received”
Thus probability of error is: Pr{ A U B}
T0
T1
R0
R1
Pr{R0|T0}
Pr{R1|T1}
39
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 40/62
Copyright © Syed Ali Khayam 2008
2. Whenever you see two events which have an AND relationship (i.e., both
event A and event B), their joint event will be their intersection, { A ∩ B}
Example: On a binary channel, find the probability that a 0 is transmitted and a
1 is received?
An error occurs when
A: “a 0 is transmitted” AND
B: “a 1 is received” Thus probability of above event is: Pr{ A ∩ B}
T0
T1
R0
R1
Pr{R0|T0}
Pr{R1|T1}
40
Four “Rules of Thumb”
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 41/62
Copyright © Syed Ali Khayam 2008
3. Whenever you see two events which have an OR relationship (i.e., A U B),
check if they are mutually exclusive. If so, set Pr{ A U B} = Pr{ A} + Pr{B}
Example: On a binary channel, find the probability of error?
An error occurs when
A: “a 0 is transmitted and a 1 is received” OR
B: “a 1 is transmitted and a 0 is received”
Thus probability of error is: Pr{error} = Pr{ A U B}
Are A and B are mutually exclusive?
T0
T1
R0
R1
Pr{R0|T0}
Pr{R1|T1}
41
Four “Rules of Thumb”
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 42/62
Copyright © Syed Ali Khayam 2008
3. Whenever you see two events which have an OR relationship (i.e., A U B), check if
they are mutually exclusive. If so, set Pr{ A U B} = Pr{ A} + Pr{B}
Example: On a binary channel, find the probability of error?
An error occurs when
A: “a 0 is transmitted and a 1 is received” OR
B: “a 1 is transmitted and a 0 is received”
Thus probability of error is: Pr{error} = Pr{ A U B}
YES!
A and B are mutually exclusive; transmission of a 0 precludes the possibility of
transmission of a 1, and vice versa. Therefore, we can set
Pr{error} = Pr{ A U B} = Pr{ A} + Pr{B}
T0
T1
R0
R1
Pr{R0|T0}
Pr{R1|T1}
42
Four “Rules of Thumb”
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 43/62
Copyright © Syed Ali Khayam 2008
4. Whenever you see two events which have an AND relationship (i.e., A ∩ B), check if
they are independent. If so, set Pr{ A ∩ B} = Pr{ A}Pr{B}
Example: On a binary channel, find the probability that a 0 is transmitted and a 1 is
received?
A: “a 0 is transmitted” AND
B: “a 1 is received”
Probability of above event is: Pr{ A ∩ B}
Are A and B independent?
T0
T1
R0
R1
Pr{R0|T0}
Pr{R1|T1}
43
Four “Rules of Thumb”
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 44/62
Copyright © Syed Ali Khayam 2008
4. Whenever you see two events which have an AND relationship (i.e., A ∩ B), check if
they are independent. If so, set Pr{ A ∩ B} = Pr{ A}Pr{B}
Example: On a binary channel, find the probability that a 0 is transmitted and a 1 is
received?
A: “a 0 is transmitted” AND
B: “a 1 is received”
Probability of above event is: Pr{ A ∩ B}
Are A and B independent?
No.
Pr ∩ = Pr Pr = 0 ×1
2 =
0
2
Pr Pr =1
2
×1
2
=1
4
T0
T1
R0
R1
Pr{R0|T0}
Pr{R1|T1}
44
Four “Rules of Thumb”
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 45/62
Copyright © Syed Ali Khayam 2008
Total Probability
B1, B2 ,…, BN form a partition of a sample space we have: S = B1 U B2 U … U BN
Bi ∩ B j = Ø, i ≠ j
B1
B2
B3 B4 s2 s4
s6
s1 s5
s3
45
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 46/62
Copyright © Syed Ali Khayam 2008
Total Probability
If B1, B2 ,…, BN form a mutually exclusive partition:
What does this imply?
B1
B2
B3
B4 A
s2s4
s6
s1 s5
s3
A
46
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 47/62
Copyright © Syed Ali Khayam 2008
Total Probability
If B1, B2 ,…, BN form a mutually exclusive partition:
What does this imply? B1 ∩ B2 ∩ ….. ∩ Bn = Ø and B1 U B2 U ….. U Bn = S
B1
B2
B3
B4 A
s2s4
s6
s1 s5
s3
A
47
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 48/62
Copyright © Syed Ali Khayam 2008
Total Probability
If B1, B2 ,…, BN form a mutually exclusive partition:
What does this imply? B1 ∩ B2 ∩ ….. ∩ Bn = Ø and B1 U B2 U ….. U Bn = S
How to express A in term of Bi?
B1
B2
B3
B4 A
s2s4
s6
s1 s5
s3
A
48
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 49/62
Copyright © Syed Ali Khayam 2008
Total Probability
If B1, B2 ,…, BN form a mutually exclusive partition:
What does this imply? B1 ∩ B2 ∩ ….. ∩ Bn = Ø and B1 U B2 U ….. U Bn = S
How to express A in term of Bi? A = ( A ∩ B1) U ( A ∩ B2) U … U ( A ∩ BN)
B1
B2
B3
B4 A
s2s4
s6
s1 s5
s3
A
49
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 50/62
Copyright © Syed Ali Khayam 2008
Total Probability
If B1, B2 ,…, BN form a mutually exclusive partition:
What does this imply? B1 ∩ B2 ∩ ….. ∩ Bn = Ø and B1 U B2 U ….. U Bn = S
How to express A in term of Bi? A = ( A ∩ B1) U ( A ∩ B2) U … U ( A ∩ BN)
What is the probability of A?
B1
B2
B3
B4 A
s2s4
s6
s1 s5
s3
A
50
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 51/62
Copyright © Syed Ali Khayam 2008
Total Probability
If B1, B2 ,…, BN form a mutually exclusive partition:
What does this imply? B1 ∩ B2 ∩ ….. ∩ Bn = Ø and B1 U B2 U ….. U Bn = S
How to express A in term of Bi? A = ( A ∩ B1) U ( A ∩ B2) U … U ( A ∩ BN)
What is the probability of A? Pr{ A} = Pr{ A ∩ B1} + Pr{ A ∩ B2} + … + Pr{ A ∩ BN}
B1
B2
B3
B4 A
s2s4
s6
s1 s5
s3
A
51
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 52/62
Copyright © Syed Ali Khayam 2008
Total Probability
Using the definition of conditional probability:
Pr{ A| Bi } = Pr{ A ∩ Bi } / Pr{Bi }
=> Pr{ A ∩ Bi } = Pr{ A| Bi } Pr{Bi }
B1
B2
B3
B4 As2
s4s6
s1 s5
s3
A
52
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 53/62
Copyright © Syed Ali Khayam 2008
The Law of Total Probability
The Law of Total Probability states:
B1
B2
B3
B4
As2 s4
s6
s1 s5
s3
A
If B1, B2,…, BN form a partition then for any event A
Pr{ A} = Pr{ A|B1} Pr{B1} + Pr{ A|B2} Pr{B2} + … + Pr{ A|BN} Pr{BN}
53
’ h
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 54/62
Copyright © Syed Ali Khayam 2008
Based on the Law of Total Probability, Thomas Bayes decided to
look at the probability of a partition given a particular event, theso-called inverse probability.
Bayes’ Theorem
B1
B2
B3
B4
As2
s4s6
s1 s5
s3
A
54
’ h
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 55/62
Copyright © Syed Ali Khayam 2008
Bayes’ Theorem
Based on the Law of Total Probability, Thomas Bayes decided to
look at the probability of a partition given a particular eventPr{Bi | A} = Pr{ A ∩ Bi } / Pr{ A}
=> Pr{ A ∩ Bi } = Pr{ A|Bi } Pr{Bi }
=> Pr{Bi | A} = Pr{ A|Bi } Pr{Bi } / Pr{ A}
B 1
B 2
B 3
B 4As 2
s 4s 6
s 1 s 5
s 3
A
55
B ’ Th
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 56/62
Copyright © Syed Ali Khayam 2008
Bayes’ Theorem
Pr{Bi | A} = Pr{ A|Bi } Pr{Bi } / Pr{ A}
From the Law of Total Probability, we have:Pr{ A} = Pr{ A|B1} Pr{B1} + Pr{ A|B2} Pr{B2} + … + Pr{ A|BN} Pr{BN}
B 1
B 2
B 3
B 4A
s 2s 4
s 6
s 1 s 5
s 3
A
Bayes’ Rule
56
B ’ Th
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 57/62
Copyright © Syed Ali Khayam 2008
Bayes’ Theorem
B 1
B 2
B 3
B 4A
s 2s 4
s 6
s 1 s 5
s 3
A
57
A Fi h P bl
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 58/62
Copyright © Syed Ali Khayam 2008
Azeem (Iqbal) is a fisherman.
Azeem is an educated man.Azeem builds a fishing robot that will do his work for him.
A Fishy Problem …
58
B 4
B 2
B 1
B 3
A Fi h P bl
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 59/62
Copyright © Syed Ali Khayam 2008
Question: If Azeem’s robot catches a fish that is detected red, what species is it?
Answer: It could be any of four species in Azeem’s part of the sea.
A Fishy Problem …
59
B 4
B 2
B 1
B 3
A Fi h P bl
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 60/62
Copyright © Syed Ali Khayam 2008
Let’s change the question:
What is the chance that a red fish is a species B1, B2, B3 and B4?
A Fishy Problem …
60
B 4
B 2
B 1
B 3
A Fi h P bl
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 61/62
Copyright © Syed Ali Khayam 2008
A Fishy Problem …
61
B 4
B 2
B 1
B 3
7/23/2019 Lecture01 02 Intro Probability Theory
http://slidepdf.com/reader/full/lecture01-02-intro-probability-theory 62/62
top related