1

Basic Probability Concepts

Irrigation and Hydraulics Department

Faculty of Engineering – Cairo University

Academic Year 2011-2012

Dr. Mohamed Attia

Assistant Professor

by

Probability Experiments

� A random experiment is an experiment that

can result in different outcomes, even though

it is repeated in the same manner every time.

2

Probability Experiments

� An experiment:

� Flipping a coin once.

� Rolling a die once.

� Pulling a card from a deck.

Sample Spaces

� The set of all possible outcomes of a random

experiment is called a sample space.

� You may think of a sample space as the set of

all values that a variable may assume.

� We are going to denote the sample space by S.

3

Examples

� Experiment: Tossing a coin once.

� S = {H, T}

� Experiment: Rolling a die once.

� S = {1, 2, 3, 4, 5, 6}

Examples

� Experiment: Drawing a card.

S = 52 cards in a deck

� K, Q, J, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 (or Ace)

� 13 spades (♠), 13 hearts (♥), 13 diamonds (♦)

and 13 clubs (♣)

4

Classification of Sample

Spaces

� We distinguish between discrete and continuous sample spaces.

� The outcomes in discrete sample spaces can be counted. The number of outcomes can be finite or infinite.

� In the case of continuous sample spaces, the outcomes fill an entire region in the space (interval on the line).

Events� An event, E, is a set of outcomes of a probability

experiment, i.e. a subset in the sample space.

� E ⊆ S.

� Simple event

� Outcome from a sample space with one

characteristic

� e.g.: A red card from a deck of cards

� Joint event

� Involves two outcomes simultaneously

� e.g.: An ace that is also red from a deck of cards

5

Visualizing Events

� Contingency Tables

� Tree Diagrams

Red 2 24 26

Black 2 24 26

Total 4 48 52

Ace Not Ace Total

Full Deck

of Cards

Red

Cards

Black

Cards

Not an Ace

Ace

Ace

Not an Ace

Visualizing Events

� Venn Diagram

� Sample space S

� Events A and B

A BS

Sometimes it is convenient to represent the

sample space by a rectangular region, with

events being circles within the region. Such a

representation is called Venn diagrams.

6

Simple Events

The Event of a Triangle

There are 55 triangles in this collection of 18 objects

The event of a triangle AND red in color

Joint Events

Two triangles that are red

7

Special Events

� Impossible event

� e.g.: Club & diamond on one card

draw

� Complement of event

� For event A, all events not in A

� Denoted as A’

� e.g.: A: queen of diamonds

A’: all cards in a deck that are

not queen of diamonds

♣♣

Null Event

Special Events

� Complement of an event

� Sample space S

� Event A

� Complement of A � A’

A

S

A’

8

Mutually Exclusive Events

� Two events, E and F, are called mutually

exclusive, if

� That is, it is impossible for an outcome to be

an occurrence of both events.

Φ=FE I

e.g.: A: queen of diamonds; B: queen of

clubs

Events A and B are mutually exclusive

One of the events must occur

The set of events covers the whole sample space

e.g.: -- A: all the aces; B: all the black cards; C: all

the diamonds; D: all the hearts

Events A, B, C and D are collectively

exhaustive

Events B, C and D are also collectively

exhaustive

Collectively exhaustive events

9

Special Events

� Collectively Exhaustive Events

� A, B, C, D are collectively exhaustive

� A and B are NOT mutually exclusive

� A and C are NOT mutually exclusive

� A and D are NOT mutually exclusive

� B, C, and D are also collectively exhaustive

� B, C, and D are mutually exclusive

B♦

C D♥

A

Special Events

A B

SBAU

BAIB

S

A

10

Contingency Table

A Deck of 52 Cards

Ace Not anAce

Total

Red

Black

Total

2 24

2 24

26

26

4 48 52

Sample

Space

Red Ace

Full Deck of Cards

Tree Diagram

Event Possibilities

Red Cards

Black

Cards

Ace

Not an Ace

Ace

Not an Ace

11

Probability

� Probability is the numerical

measure of the likelihood

that an event will occur

� The probability of an event E

is a number, P(E), such that

� Sum of the probabilities of

all mutually exclusive and

collective exhaustive events

is 1

1)(0 ≤≤ EP

Certain

Impossible

.5

1

01 )( =SP

(There are 2 ways to get one 6 and the other 4)

e.g. P( ) = 2/36

Computing Probabilities

� The probability of an event E:

� Each of the outcomes in the sample space is

equally likely to occur

number of event outcomes( )

total number of possible outcomes in the sample space

P E

X

T

=

=

12

Computing Joint Probability

� The probability of a joint event, A and B:

( and ) = ( )

number of outcomes from both A and B

total number of possible outcomes in sample space

P A B P A B∩

=

E.g. (Red Card and Ace)

2 Red Aces 1

52 Total Number of Cards 26

P

= =

P(A1 and B2) P(A1)

TotalEvent

Joint Probability Using

Contingency Table

P(A2 and B1)

P(A1 and B1)

Event

Total 1

Joint Probability Marginal (Simple) Probability

A1

A2

B1 B2

P(B1) P(B2)

P(A2 and B2) P(A2)

13

Computing Compound (or

Multiple) Probability

� Probability of a compound event, A or B:

( or ) ( )

number of outcomes from either A or B or both

total number of outcomes in sample space

P A B P A B= ∪

=

E.g. (Red Card or Ace)

4 Aces + 26 Red Cards - 2 Red Aces

52 total number of cards

28 7

52 13

P

=

= =

P(A1)

P(B2)

P(A1 and B1)

Compound Probability

(Addition Rule)

P(A1 or B1 ) = P(A1) + P(B1) - P(A1 and B1)

P(A1 and B2)

TotalEvent

P(A2 and B1)

Event

Total 1

A1

A2

B1 B2

P(B1)

P(A2 and B2) P(A2)

For Mutually Exclusive Events: P(A or B) = P(A) + P(B)

14

Computing Conditional Probability

� The probability of event A given that event B has

occurred:

� It means the likelihood of realizing a sample point in

A assuming that it belongs to B

( and )( | )

( )

P A BP A B

P B=

E.g.

(Red Card given that it is an Ace)

2 Red Aces 1

4 Aces 2

P

= =/52

/52= 0.5

B

A&B

A

Conditional Probability Using

Contingency Table

Black

ColorType Red Total

Ace 2 2 4

Non-Ace 24 24 48

Total 26 26 52

Revised Sample Space

(Ace and Red) 2 / 52 2(Ace | Red)

(Red) 26 / 52 26

PP

P= = =

15

Conditional Probability and

Statistical Independence

� Conditional probability:

� Multiplication rule:

( and )( | )

( )

P A BP A B

P B=

( and ) ( | ) ( )

( | ) ( )

P A B P A B P B

P B A P A

=

=

Conditional Probability and

Statistical Independence

� Events A and B are independent if

� Events A and B are independent when the probability of one event, A, is not affected by another event, B

(continued)

( | ) ( )

or ( | ) ( )

or ( and ) ( ) ( )

P A B P A

P B A P B

P A B P A P B

=

=

=

16

� B1, B2, …, Bk are mutually exclusive and

collectively exhaustive

� Suppose it is known that event A has

occurred

� What is the (conditional) probability that event

Bi has occurred?

Bayes’s Theorem

B1 B2 Bk

A

Bayes’s Theorem

( )( ) ( )

( ) ( ) ( ) ( )

( )( )

1 1

||

| |

and

i i

i

k k

i

P A B P BP B A

P A B P B P A B P B

P B A

P A

=+ •••+

=Adding up

the parts

of A in all

the B’sSame

Event

17

Bayes’s Theorem Using Contingency

Table (Example 1)

Fifty percent of graduating engineers in a certain year worked as water resources (WR) managers. Out of those who worked as WR managers, 40% had a Masters degree. Ten percent of those graduating engineers who did not work as WR managers had a Masters degree. What is the probability that a randomly selected engineer who has a Masters degree is a WR manager?

?)/(

1.0)/(4.0)/(5.0)(

=

===

MWRP

RWMPWRMPWRP

Bayes’s Theorem

Using Contingency Table (continued)

WR WR

Masters

Masters

1.0.5 .5

.2

.3

.05

.45

.25

.75

Total

Total

8.025.0

2.0

)5.0)(1.0()5.0)(4.0(

)5.0)(4.0(

)()|()()|(

)()|()/(

==+

=

+=

RWPRWMPWRPWRMP

WRPWRMPMWRP

18

Example 2

� A water treatment plant may fail for two reasons: inadequacy of materials (event A) or mechanical failure (event B).

If P(A) = 2 P(B), P(A | B) = 0.8 and the probability of failure of the treatment plant equals 0.001 , what is the probability that a mechanical failure occurs? What is the probability of a failure due to inadequate materials?

The probability of failure P (A U B) may be written as:

P (A U B) = P (A) + P (B) – P (A ∩ B)

= 2 P (B) + P (B) – P (B) P (A | B)

= 3 P (B) – 0.8P (B)

0.001 = 3 P (B) – 0.8P (B)

� P (B) = 0.00045

� P (A) = 0.0009

Example 3

� Each of two pumps Q1 and Q2 may not operate. On inspection

one may observe one of the outcomes of the following set: {(ƒ,

ƒ), (ƒ, o), (o, ƒ), (o, o)}. The notation (ƒ, o) means that pump Q1

fails and pump Q2 operates. The sample space has four

elements. From experience one knows that:

� P ((ƒ, ƒ)) = 0.1, P ((ƒ, o )) = 0.2

� P ((o, ƒ)) = 0.3, P ((o, o )) = 0.4

� Let

� A: Q1 operates

� B: Q2 operates

� C: at least one of the pumps operates

� Compute P(A), P(B), P(A ∩ B), P(C)

19

Example 3 (Cont.)

� P ( A ) = 0.7

� P ( B ) = 0.6

� P ( A ∩ B ) = P (( o , o )) = 0.4

� P ( C ) = ( A U B )

= P ( A ) + P ( B ) – P ( A ∩ B ) = 0.9

Example 4

� The demand of a water supply system can be low (event L) , moderate (event M) or high (event H) with known probabilities P(L) = 0.1, P(M) = 0.7 and P(H) = 0.2. Failure of the system (event F) can only occur if a certain pump fails to function. From past experience the following conditional probabilities are known: P(F | L) = 0.05, P(F | M) = 0.10 and P(F | H) = 0.25. What is the probability of a pump failure P(F).

First we observe that the events L, M and H are clearly disjoint and that their union is just the sure event S. Hence

P ( F ) = P (( L ∩ F ) U ( M ∩ F ) U ( H ∩ F ))

= P (L ∩ F) + P (M ∩ F) + P (H ∩ F)

Using the definition of conditional probability, we get

P ( F ) = P (L) P (F | L) + P (M) P (F | M) + P (H) P (F | H)

= 0.1 × 0.05 + 0.7 × 0.10 + 0.2 × 0.25

= 0.125

20

Example 5

� Weather forecast information is sent using one of four routes. Let Ri denote the event that route i is used for sending a message (i = 1, 2, 3, 4). The probabilities of using the four routes are 0.1, 0.2, 0.3 and 0.4, respectively. Further, it is known that the probabilities of an error being introduced while transmitting messages are 0.10, 0.15, 0.20 and 0.25 for routes 1, 2, 3 and 4, respectively. In sending a message an error occurred. What is the probability that route 2 was used for transmission?

Let E denote the event of having an erroneous message

� P(E | R1) = 0.10, P(E | R2) = 0.15,

� P(E | R3) = 0.20, P(E | R4) = 0.25.

Example 5 (Cont.)

� Using Bayes' rule one can compute the probability that the message was sent via route 2 given the event that an error has occurred. This probability is given by

� P(R2 | E) = )(

)( 2

EP

ERP I

∑ =

=4

1

22

)|()(

)|()(

i ii REPRP

REPRP

25.04.020.03.015.02.010.01.0

15.02.0

×+×+×+×

×=

.15.020.0

03.0==

21

Example 6

� The probability of receiving more than 50 mm of rain in the months of the year is 0.25, 0.30, 0.35, 0.40, 0.20, 0.10, 0.05,0.05, 0.05, 0.05, 0.10 and 0.20 for January, February, ..., December, respectively. A monthly rainfall record selected at random is found to be more than 50 mm. What is the probability that this record belongs to month m (m = 1 for January, ..., m = 12 for December).

� Denote the probability of selecting month m as P(Mm).

� Denote the conditional probability of receiving more than 50 mm of rain, given that month m is selected as P(E | Mm).

� Then P(Mm | E) can be computed using Bayes' rule as indicated in the following equation and table

∑=

==12

1

)|()(

)|()(

)(

)|()()|(

m

mm

jjjj

j

MEPMP

MEPMP

EP

MEPMPEMP

Example 6 (Cont.)

Application of Bayes’ rule

m P(Mm) P(E | Mm) P(Mm) P(E | Mm) P(Mm | E)

1 1/12 0.25 0.02083 0.1191

2 1/12 0.30 0.02500 0.1429

3 1/12 0.35 0.02917 0.1667

4 1/12 0.40 0.03333 0.1905

5 1/12 0.20 0.01667 0.0952

6 1/12 0.10 0.00833 0.0476

7 1/12 0.05 0.00417 0.0238

8 1/12 0.05 0.00417 0.0238

9 1/12 0.05 0.00417 0.0238

10 1/12 0.05 0.00417 0.0238

11 1/12 0.10 0.00833 0.0476

12 1/12 0.20 0.01667 0.0952

12/12 0.17500 = P(E) 1.0000

22

Thank you