set 1 - introduction & probability model
TRANSCRIPT
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 1/19
1
Dr. M. Hassan
ELE 360
Probability & Stochastic Processes
Introduction & Probability Model
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 2/19
2
Dr. M. Hassan
What is Probability?
• We are all aware of:– Deterministic events
• Daily sunrises and sunsets• Tides movement at the sea shore• Phases of the moon• Seasonal changes in weather
• Annual flooding of the Nile river … etc• What about:
– Randomevents• The outcome when you toss a coin or roll a die• Results of horse & car races• Variations of the stock market• Noise in communications systems strength of a received signal!• Waiting time in a traffic signal• Lifetime of a system• Waiting time at a cashier
Probability is the likelihood of an event happening (or not happening). Probabilitytypically tells us how certain we are about an event.
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 3/19
3
Dr. M. Hassan
Approaches to Assigning Probabilities
• Classical approach: make certain assumptions (such asequally likely, independence) about a situation
• Relative frequency approach: assign probabilities based onexperimentation or historical data
• Subjective approach: assign probabilities based on the
assignor’s judgment
• There are three ways to assign a probability, , to an outcome,, namely:
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 4/19
4
Dr. M. Hassan
Relative Frequency & Probability• Consider a random experiment the outcome of which is uncertain i.e. it
differs from a run to the other– Examples of such experiments would be a coin toss, dice rolling, ..etc
• Probability is a numerical measure of the likelihood of an event– It is a number that we attach to an event– A probability is a number from 0 to 1
• In general, the probability of an event can be approximated by therelative frequency , or the proportion of times that event occurs
The probability of an event is approximately =# of times the event occurs
# of experiments
As the probability is given by:
• If an outcome occurs times out of trials, its relative frequencyis and we define its probability to be approximately equal to
– “The probability of heads showing is ½ because when tossed repeatedly ,the coin will show heads half the time”
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 5/19
5
Dr. M. Hassan
Random Experiments & Trials• An random experiment is any act or process of observation that is when
performed leads to a single outcome that CANNOT be predicted withcertainty by the observer before the experiment– Theoretically, an experiment can be repeated (under the same conditions)
for an infinite number of times– The possible outcomes are defined by the observers according to their
interests– Each distinct outcome is called a “ sample point ” or a “basic outcome ”– A random experiment typically has a well-defined set of outcomes
• The experiment could be performed by a human, e.g. tossing a coin orrolling a die
• On the other hand, an outcome of an experiment could be just themeasurement of a naturally occurring random phenomenon, e.g. a noise
voltage– Atrial is one run of the experiment
The sample space of a random experiment is the set of all mutuallyexclusive and collectively exhaustive outcomes
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 6/19
6
Dr. M. Hassan
Probability Theory
• Consider for example the experiment of rolling a die:
• Sample space is the collection of all possible samplepoints (outcomes)
• Events are the results of experiments
• An event is a set of outcomes that meet certain criteria• An event with no sample points is called a null event
• An outcome or a sample point is an event that
cannot be decomposed into other events
The sample space is
An event could be “an even number is thrown”
An outcome
Events
• Random events are defined on a probability space that consists of asample space of all possible outcomes , a set of events that aresubsets of , and a probability measure which assigns a probability
to every event
What is ?
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 7/19
7
Dr. M. Hassan
Set Definitions• A set is a collection of objects (outcomes)
– Objects are called elements of the set– Outcomes are the elements of the sample space
• A set can be written as
• When an element belongs to a set , we write
• When an element does not belong to a set , we write
• A set can also be written asThis is called the rule method
This is called the tabular method
Tabularmethod isnot alwayspractical !!
• If all elements in a set areelements of the set , we say
that is a subset of ,denoted by:
• The subset relationship is atransitive relationship i.e.,
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 8/19
8
Dr. M. Hassan
Set Definitions (cont.)
• A set can be classified as:• Countable set
• The elements of which can be placed in one-to-one with the setof integers
• A set is said to be empty if it has no elements• An empty set is also called the null set and is referred to bythe symbol
• Uncountable set• A not countable set is uncountable• Example:
• Finite set• It is the set the elements of which are counted with a
terminating process• Infinite set
• If the set is not finite it is infinite. An infinite set could becountably infinite
Notice thedifference
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 9/19
9
Dr. M. Hassan
The Sample Space: Examples• Recall that the set of all possible outcomes of an experiment is calledthe sample space of the experiment
Examples• The experiment of a coin toss has the sample
space:
• The experiment of rolling a die has the samplespace:
These are examples ofdiscrete &finitesample space
• The experiment of measuring noise voltage has thesample space:
This is an example ofinfinite/continuous
sample space
• The experiment of randomly choosing a positive integerhas the sample space:
This is an exampleof countably infinite
&discrete samplespace
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 10/19
10
Dr. M. Hassan
Basic Set Operations• Set operations are better explained using Venn diagrams which areclosed-plane figures
The intersection of A and B
The union of A and BThe complement of A
The difference of A and B
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 11/19
11
Dr. M. Hassan
Set Algebra• Three theorems relate to laws of set operations including union &
intersection
• The distributive law
• The commutative law
• The associative law
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 12/19
12
Dr. M. Hassan
De Morgan’s Law & Duality Principle
• The De Morgan’s law:
• De Morgan’s law states that the complement of a union (intersection)of two sets and equals the intersection (union) of thecomplements and
• In application of De Morgan’s law to any expression, if we replaceunions by intersections and intersections by unions and each set byits complement, the identity is preserved
The duality principle:• The duality principle states that, for any identity, if we replace
unions by intersections and intersections by unions and the samplespace by the empty set and by the identity ispreserved
Example: Consider the identity given by
Applying the duality principle, we get
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 13/19
13
Dr. M. Hassan
What is an Event?• An event is a specific collection of sample points (outcomes) that meets
certain conditions– It is a subset of the sample space of an experiment– Events are defined on the sample space in a probabilistic way to select
subsets of the sample space– Events specify the interest in the characteristics of outcomes rather the
outcomes themselves– Two events are mutually exclusive when they have no common outcomes– All definitions & operations applicable to sets are also applicable to events
• The probability of an event is derived from the probabilities assigned tothe sample points associated with that event
and are events defined on the samplespace
– and are also events defined on
• Any subset of is called an eventExample:
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 14/19
14
Dr. M. Hassan
Example: Many Events Occur• In a roll of a die experiment, we define the events:
Events and have both occurredEvents and did not occur
The event has occurred
The event has also occurred … … and so on
• Assume now the observed outcome is 4, then:
Elementary or singleton events• An event that contains a single outcome is called an elementary event or
singleton event
– is an elementary event
- Also note that if 4 is a member of while is a subset of
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 15/19
15
Dr. M. Hassan
Outcomes versus Events (revisited)• Every trial results in only one outcome, say x is the outcome
• Fundamental notion: On each trial of the experiment, one outcome ( x inthis case) occurs, but many events may occur
• Of course, only the one elementary event { x} has occurred on this trial;all the other many events that have occurred have { x} as a subset, that
is, A occurs iff { x} AEvents Occur & Do Not Occur in Pairs
• Many (non-elementary) events may occur on this trial
One of the events A
and A c
always occurs
• Suppose that a trial resulted in outcome• Exactly one elementary event, occurred on this trial
• For any event , exactly one of the two events and occurred, andthe other did not
• Example: assume that the outcome of rolling a die is 4, then:– If the event occurred, this implies that the
event did not occur
– When occurs then the event willnot occur
16
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 16/19
16
Dr. M. Hassan
Two Special Events• can be regarded as a subset of• On any trial, the event always occurs
• The event is called the certain event or the sure event
• , the empty set, is also a subset of
• On any trial, the event never occurs
• The event is called the null event or the impossible event
17
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 17/19
17
Dr. M. Hassan
Disjoint Events• Events A and B are said to be disjoint or mutually exclusive if
• A and B have no elements in commonProbability of a disjoint union
• If events A and B are disjoint , then is said to be a disjoint unionof events
• For a disjoint union of events A and B ,• A consequence of this is the event A is the disjoint union of the elementary
events corresponding to its members
• If and are disjointevents
The probability of the union is given by:
18
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 18/19
18
Dr. M. Hassan
Axioms of Probability for Finite Spaces• The ideas described thus far are the basis of the axioms of probability
theory Probabilities are numbers assigned to events that satisfy thefollowing rules:
• Axiom I: for all events A
• Axiom II:• Axiom III: If events A and B are disjoint , then:
Consequences of the Axioms• and are disjoint events, then:
(Axiom III)
(Axiom II)
• But,
• Hence
19
7/27/2019 Set 1 - Introduction & Probability Model
http://slidepdf.com/reader/full/set-1-introduction-probability-model 19/19
19
Dr. M. Hassan
More Consequences of the Axioms
• and are disjoint events;
• Axiom I: for all events A
• Axiom II:• Axiom III: If events A and B are disjoint , then:
••
• Since (Axiom I), we deduce that: