foundations of stochastic processes · probability faculty of electrical engineering and computer...
Post on 29-Jun-2020
1 Views
Preview:
TRANSCRIPT
Sample Lectures
Foundations of Stochastic Processes
Instructor: Giuseppe Caire
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Copyright G. Caire (Sample Lectures)
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire Lecture 1:
Events and their Probabilities
Copyright G. Caire (Sample Lectures) 1
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Events as sets
• Often the nature of an action, or experiment is too complex to be describedin purely deterministic terms.
• A more convenient and compact description is probabilistic: the possibleoutcomes of the experiment may occur with a given chance, that we callprobability.
Definition 1. The set of all possible outcomes of an experiment is calledsample space and it is denoted by ⌦. ⌃
Example 1. A coin is tossed, and there are two possible outcomes, H or T,hence, ⌦ = {H,T}.
Example 2. A die is thrown once. There are six possible outcomes, hence⌦ = {1, 2, 3, 4, 5, 6}.
Copyright G. Caire (Sample Lectures) 2
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
• Typically we are interested in events, defined by coditions on the outcomesof our experiment.
1. “the outcome is Head”.2. “the outcome is Head or Tail”.3. “the outcome is both Head and Tail”.4. “the outcome is even and not equal to 4”.5. “the outcome is less than or equal to 3”.
• We see that all the above events are conveniently represented as subsets of⌦:
1. A = {H}.2. A = {H} [ {T}.3. A = {H} \ {T}.4. A = {2, 6}.5. A = {1, 2, 3}.
Copyright G. Caire (Sample Lectures) 3
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
• Formally: we think of events as subsets of the sample space ⌦.
• Furthermore, we wish that the set of all events has certain properties, forexample, given two events A ✓ ⌦ and B ✓ ⌦ we wish that A [ B, Ac andA \B are also events.
Definition 2. The set of all events of an experiment with sample space ⌦ is asubset F of the set of all subsets of ⌦ that enjoys the following properties:
1. If A,B 2 F , then A [B 2 F and A \B 2 F .
2. If A 2 F , then Ac 2 F .
3. The empty set ; belongs to F .
Such a set F of subsets of ⌦ is called a field. ⌃
Copyright G. Caire (Sample Lectures) 4
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
• The above definition is sufficient to handle the case where ⌦ is a finite set.However, we may have cases where ⌦ is infinite. In this case we requirethat the set of all events is closed under countable unions. In this case, F isreferred to as a �-field.
Definition 3. A set F of subsets of ⌦ is called a �-field if it enjoys the followingproperties:
1. If A1, A2, . . . are elements of F , thenS1
i=1Ai 2 F .
2. If A 2 F , then Ac 2 F .
3. The empty set ; belongs to F .
⌃
Copyright G. Caire (Sample Lectures) 5
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Example 3. The smallest F associated with the sample space ⌦ is given byF = {;,⌦}.
Example 4. The smallest F associated with the sample space ⌦ andcontaining a given event A ✓ ⌦ is given by F = {;, A,Ac,⌦}.
Example 5. When ⌦ is finite, the power set of ⌦, denoted by 2
⌦, or {0, 1}⌦, isthe set of all subsets of ⌦. Clearly, {0, 1}⌦ is a �-field.
SUMMARY. Any experiment may be associated with a pair (⌦,F), where ⌦ isthe sample space, that is, the set of all possible experiment outcomes, and Fis a �-field, i.e., a collection of subsets of ⌦ satisfying the conditions ofDefinition 3. The elements of F (subsets of ⌦) are referred to as “events”.
Copyright G. Caire (Sample Lectures) 6
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Elementary set theory
• Set = collection of objects, or “elements”. Ordering is immaterial and objectsare not repeated.
• |A| denotes the number of elements of A (cardinality). When |A| is a finiteinteger, we say that A is finite.
• When |A| = 1 (infinite set) and the elements can be put in one-to-onecorrespondence with N we say that the set A is countable.
• It might be that A is infinite, and its elements cannot be “counted”, for exampleA = R, the real line, then A is said to be uncountable.
• Identical sets A = B.
• Distinct sets A 6= B.
• A is contained in B, A ✓ B.
Copyright G. Caire (Sample Lectures) 7
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
• A contains B, A ◆ B or B ✓ A.
• When A is strictly a subset of B we write A ⇢ B, this means that there existsat least one element b 2 B such that b /2 A.
• The “subset of” relation induces a partial ordering on the set of subsets of ⌦.
• Empty set ;, universe set ⌦.
• Any subset A of ⌦ satisfies ; ✓ A ✓ ⌦.
• Intersection,A \B = {a : a 2 A and a 2 B}
• Union,A [B = {a : a 2 A or a 2 B}
Copyright G. Caire (Sample Lectures) 8
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
• Complement set (with respect to ⌦)
Ac= {a : a 2 ⌦ and a /2 A}
clearly, (Ac)
c= A, ;c = ⌦, ⌦c
= ;.
• Distributive lawA \ (B [ C) = (A \B) [ (A \ C)
A [ (B \ C) = (A [B) \ (A [ C)
• Associative law
(A \B) \ C = A \ (B \ C) = A \B \ C
(A [B) [ C = A [ (B [ C) = A [B [ C
Copyright G. Caire (Sample Lectures) 9
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
• De Morgan laws: for any finite or countable collection of subsets A1, A2, . . .of ⌦
[
i
Ai
!c
=
\
i
Aci
and obviously \
i
Ai
!c
=
[
i
Aci
• A,B are said to be disjoint if A \B = ;.
• Difference set, A�B also denoted by A\B is defined as
A�B = {a : a 2 A and a /2 B} = A \Bc
Copyright G. Caire (Sample Lectures) 10
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Probability as empirical frequency
• The concept of probability is intuitively motivated by a repeated experimentargument.
• Suppose that we repeat an experiment a large number of times N , and countthe number of times N(A) the event A occurs.
• In most cases encountered in practice, the ratio N(A)/N converges to somenumber in [0, 1], that intuitively represents the likelihood of the event A, or itsprobability.
• If A is the impossible event, A = ;, then N(;) = 0. If A is the certain event,A = ⌦, then N(A) = N .
• If A and B are disjoint, it is clear that when A occurs then B does not occurand viceversa, therefore, N(A [B) = N(A) +N(B).
Copyright G. Caire (Sample Lectures) 11
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Axiomatic definition of probability space
Definition 4. A probability measure P on (⌦,F) is a set function P : F ! [0, 1]satisfying:
1. P(;) = 0, P(⌦) = 1.
2. if A1, A2, . . . is a countable collection of disjoint elements of F (i.e., Ai\Aj = ;for all i 6= j) then
P 1[
i=1
Ai
!=
1X
i=1
P(Ai)
The triple (⌦,F ,P) is called probability space. ⌃
Example 6. The underlying experiment consists of a biased coin tossed once.Let ⌦ = {T,H} and F = 2
⌦= {;, {H}, {T}, {T,H}}. For any given p 2 [0, 1]
P(T ) = p, P(H) = 1� p
defines a probability space (⌦,F ,P).
Copyright G. Caire (Sample Lectures) 12
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Example 7. The underlying experiment consists of a biased coin tossedmany times. We stop tossing when T appears. In this case, ⌦ =
{T,HT,HHT,HHHT, . . .} is countably infinite. In general, we may haveseveral different F associated to this ⌦. For example, F may contain eventsof the type A = {the number of tosses is even}. For any given p 2 [0, 1], let
P(T ) = p, P(HT ) = (1� p)p, P(HHT ) = (1� p)2p, . . .
This defines a probability space (⌦,F ,P), where the associated probabilitymeasure is called geometric. Notice that
P(⌦) =1X
i=0
P(HiT ) =1X
i=0
(1� p)ip =
p
1� (1� p)= 1
Copyright G. Caire (Sample Lectures) 13
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Example 8. The underlying experiment consists of a die thrown once. Let⌦ = {1, 2, 3, 4, 5, 6} and F = 2
⌦. For any given non-negative numbers p1, . . . , p6such that their sum is equal to 1, for all A 2 F let P(A) =
Pi2A pi. Then,
(⌦,F ,P) is a probability space.
Copyright G. Caire (Sample Lectures) 14
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Some properties
Lemma 1. The following statements hold:
1. P(Ac) = 1� P(A).
2. If A ✓ B then P(B) = P(A) + P(B �A) � P(A).
3. P(A [B) = P(A) + P(B)� P(A \B).
4. More in general, for A1, . . . , An 2 F , we have the inclusion/exclusion formula
P ([ni=1Ai) =
X
i
P(Ai)�X
i<j
P(Ai \Aj) +
X
i<j<k
P(Ai \Aj \Ak)� · · ·
· · ·+ (�1)
n+1P (\ni=1Ai)
Copyright G. Caire (Sample Lectures) 15
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Continuity of the probability measure
• Increasing sequence of sets: A1 ✓ A2 ✓ A3 ✓ · · · , we write
A = lim
n!1An
�= lim
n!1
n[
i=1
Ai =
1[
i=1
Ai
• Decreasing sequence of sets: A1 ◆ A2 ◆ A3 ◆ · · · , we write
A = lim
n!1An
�= lim
n!1
n\
i=1
Ai =
1\
i=1
Ai
Lemma 2. Let {Ai} denote an increasing or decreasing sequence of events inF such that limn!1An = A, then
lim
n!1P(An) = P(A)
Copyright G. Caire (Sample Lectures) 16
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Null events and impossible events
• An event A 2 F such that P(A) = 0 is said to be a null event, or an event ofmeasure zero.
• An event A 2 F such that P(A) = 1 is said to be an almost sure event.
• Null events should not be confused with the impossible event, ;.
• For example, consider the probability that a bullet hits in exactly a given pointof coordinates (x, y) on the real plane...DISCUSSION.
Copyright G. Caire (Sample Lectures) 17
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Conditional probability
• Many questions about the likelihoof of events are in the form: “what is theprobability of A given that B occurs?”.
• For example: what is the probability that the LAX–SFO flight will arrive ontime if the weather is bad?
• Going back to the repeated experiment intuition, we may run our experimentN times and count the occurrences of B and of A \ B, it follows that thefraction of times A occurs given that B occurs is given by
N(A \B)
N(B)
=
N(A \B)/N
N(B)/N! P(A \B)
P(B)
• This argument prompts us to the definition of conditional probability.
Copyright G. Caire (Sample Lectures) 18
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Definition 5. Consider a probability space (⌦,F ,P), and A,B 2 F . If P(B) >0, the conditional probability of A given B is defined as
P(A|B) =
P(A \B)
P(B)
⌃
Notice: If P(B) = 0, then P(A|B) is undefined.
A collections of events {B1, . . . , Bn} is said to be a partition of ⌦ ifSn
i=1Bi = ⌦
and Bi \Bj = ; for all i 6= j.
Lemma 3. (Total probability) Let {B1, . . . , Bn} be a partition of ⌦ and A be anevent, then
P(A) =
nX
i=1
P(A|Bi)P(Bi)
Copyright G. Caire (Sample Lectures) 19
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Lemma 4. (Bayes rule) Let A,B be events, then
P(B|A) =
P(A|B)P(B)
P(A)
Lemma 5. Let A1, . . . , An be a collection of events with positive probability.The probability of their intersection (called “joint probability”) has the followingtelescopic property:
P(A1 \A2 \ · · · \An) = P(A1)P(A2|A1)P(A3|A1, A2) · · ·P(An|A1, . . . , An�1)
Notice: P(C|A,B) and P(C|A \B) are alternative equivalent notations. Jointevents (intersections) can be indicated by A \B or (A,B), equivalently.
Copyright G. Caire (Sample Lectures) 20
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Statistical independence
Definition 6. A collection of events {Ai : i 2 I} is called independent if
P \
i2J
Ai
!=
Y
i2J
P(Ai)
for all finite subsets J ✓ I. ⌃
WARNING! a common mistake consists of confusing independence with“disjoint”. Consider A,B such that A \B = ;. Then, P(A \B) = 0. If A and Bwere independent, then P(A \B) = P(A)P(B) = 0 which implies that A or Bare null events. On the contrary, if P(A) > 0 and P(B) > 0, we see clearly thatthe condition A \B = ; implies that the two events are dependent.
Definition 7. A collection of events {Ai : i 2 I} is called pairwise independentif
P (Ai \Aj) = P(Ai)P(Aj)
for all i 6= j 2 I. ⌃
Copyright G. Caire (Sample Lectures) 21
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
• An equivalent condition for independence of A and B when P(B) > 0 is
P(A|B) = P(A)
• Two events A and B are said to be conditionally independent given an eventC (with P(C) > 0) if
P(A \B|C) = P(A|C)P(B|C)
• Three events A,B,C with positive probability form a Markov chain in theorder A ! B ! C if
P(A \B \ C) = P(A)P(B|A)P(C|B)
or, equivalently, if P(C|A,B) = P(C|B).
Copyright G. Caire (Sample Lectures) 22
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
• If A and B are conditionally independent given C, then A ! C ! B andB ! C ! A are Markov chains, in fact:
P(A\B\C) = P(A\B|C)P(C) = P(B|C)P(A|C)P(C) = P(B|C)P(C|A)P(A)
and
P(A\B\C) = P(A\B|C)P(C) = P(A|C)P(B|C)P(C) = P(A|C)P(C|B)P(B)
• If A and B are independent, then Ac and B are independent and then alsoAc and Bc are independent.
Copyright G. Caire (Sample Lectures) 23
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Completeness
Lemma 6. If F and G are two �-fields of subsets of ⌦, their intersection F \ Gis also a �-field.
• The union of two �-fields F and G of ⌦ is not necessarily a �-field. However,we can define the smallest �-field containing F [G. This is denoted by �(F [G): take all �-fields {Gi : i 2 I} that contain F [ G. Then,
�(F [ G) =\
i2I
Gi
• Given a probability space (⌦,F ,P), any A 2 F such that P(A) = 0 is a nullevent. Let B ⇢ A, then, it is reasonable that also P(B) = 0. However, B maynot be in F and therefore P(B) may not be defined.
Copyright G. Caire (Sample Lectures) 24
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Definition 8. Completeness. A probability space (⌦,F ,P) such that allsubsets of null events are in F is called complete. ⌃
• Any probability space can be completed as follows: take the set N of allsubsets of all null events of (⌦,F ,P), then let G = �(F [N ) and extend P toG in the obvious way (subsets of null events are null events). The resultingspace is called the completion of the original space.
Copyright G. Caire (Sample Lectures) 25
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
Product spaces
• In many cases we wish to combine the results of two independentexperiments into a single probability space.
• Consider (⌦1,F1,P1) and (⌦2,F2,P2) be two probability spaces.
• Running independently the two experiments underlying these spaces yieldsoutcomes of the type (!1,!2) with !1 2 ⌦1 and !2 2 ⌦2.
• The sample space of the combined experiment is then the Cartesian product⌦ = ⌦1 ⇥ ⌦2.
• The associated �-field is the smallest that contains F1 ⇥ F2, i.e., it is G =
�(F1 ⇥ F2).
Copyright G. Caire (Sample Lectures) 26
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire
• The associated probability measure is given by
P(A1 ⇥A2) = P1(A1)P2(A2)
and it can be extended to the whole G (Notice: this technicality is not neededmost of the times, and we shall not go into the details here).
• Eventually, the resulting (⌦1 ⇥ ⌦2,G,P) is called the product space of(⌦1,F1,P1) and (⌦2,F2,P2).
• The product space operation can be applied to any number n of spaces, thusdefining n-fold product spaces.
Copyright G. Caire (Sample Lectures) 27
TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin
www.mk.tu-berlin.de
Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Systems Information and Communication Theory Prof. Dr. Giuseppe Caire Einsteinufer 25 10587 Berlin Telefon +49 (0)30 314-29668 Telefax +49 (0)30 314-28320 caire@tu-berlin.de Sekretariat HFT6 Patrycja Chudzik Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 sekretariat@mk.tu-berlin.de
Firma xy Herrn Mustermann Beispielstraße 11 12345 Musterstadt
Berlin, 1. Month 2014
Subject: Text…& Prof. Dr. Giuseppe Caire End of Lecture 1
Copyright G. Caire (Sample Lectures) 28
top related