intro to probability

Intro to Probability

Martina Litschmannová[email protected]

K210

mailto:[email protected]



Probability basics

Statistical (random) experiment

A random experiment is an action or process that leads to one of several possible outcomes.

For example:

Experiment Outcomes

Flip a coin Heads, Tails

Exam Marks Numbers: 0, 1, 2, ..., 100

Assembly Time t > 0 seconds

Course Grades F, D, C, B, A, A+

Statistical (random) experiment

All statistical experiments have three things in common:

The experiment can have more than one possible outcome. Each possible outcome can be specified in advance. The outcome of the experiment depends on chance.

Probability Terminology

A sample space Ω is a set of elements that represents all possible outcomes of a statistical experiment. No two outcomes can occur at the same time!

A simple event is an element of a sample space. Simple events are denoted by ,

An event is a subset of a sample space - one or more sample points. Events are denoted by , or

Probability Terminology

Using notation from set theory, we can represent the sample space and its outcomes as

.

Types of Events

Two events are mutually exclusive if they have no sample points in common.

Two events are independent when the occurrence of one does not affect the probability of the occurrence of the other.

Requirements of Probabilities

Given a sample space , the probabilities assigned to the outcome must satisfy these requirements:

The probability of any outcome is between 0 and 1,i.e. for each i.

The sum of the probabilities of all the outcomes equals 1,i.e. + +…+ =1.

represents the probability of outcome i

Approaches to Assigning Probabilities

There are three ways to assign a probability, , to an outcome, , namely:

Classical approach: make certain assumptions (such as equally likely, independence) about situation.

Relative frequency: assigning probabilities based on experimentation or historical data.

Subjective approach: Assigning probabilities based on the assignor’s judgment.

Classical Approach

If an experiment has n possible outcomes (all equally likely to occur), this method would assign a probability of 1/n to each outcome.

1. You are rolling a fair die. What is the probability that one will fall?

Experiment: Rolling a dieSample Space: Probabilities: Each sample point has a 1/6 chance of

occurring.

2. What about randomly selecting a student from this class and observing their gender?

Experiment: Random selection of a student from this class Sample Space: Probabilities: ???

Are these probabilities ½?

3. Experiment: Rolling 2 (fair) die (dice) and summing 2 numbers on top.

Sample Space: Probability Examples:

1 2 3 4 5 6

1 2 3 4 5 6 7

2 3 4 5 6 7 8

3 4 5 6 7 8 9

4 5 6 7 8 9 10

5 6 7 8 9 10 11

6 7 8 9 10 11 12

Sample Space: Probability Examples:

1 2 3 4 5 6

1 2 3 4 5 6 7

2 3 4 5 6 7 8

3 4 5 6 7 8 9

4 5 6 7 8 9 10

5 6 7 8 9 10 11

6 7 8 9 10 11 12

3. Experiment: Rolling 2 (fair) die (dice) and summing 2 numbers on top.

Relative Frequency Approach

4. Computer Shop tracks the number of desktop computer systems it sells over a month (30 days):

For example,10 days out of 302 desktops were sold.

From this we can construct the “estimated” probabilities of an event (i.e. the # of desktop sold on a given day).

Desktops Sold # of Days0 11 22 103 124 5

Relative Frequency Approach…Desktops Sold [X] # of Days Desktops Sold probabilities

0 11 22 103 124 5


0 11 22 103 124 5

30


0 11 22 103 124 5

30 1,00


0 11 22 103 124 5

30 1,00

There is a 40% chance Computer Shop will sell 3 desktops on any given day (Based on estimates obtained from sample of 30 days).

Subjective Approach

In the subjective approach we define probability as the degree of belief that we hold in the occurrence of an event.

For example:

P(you drop this course) P(NASA successfully land a man on the moon) P(girlfriend/boyfriend says yes when you ask her to marry

you)

Events & Probabilities

An individual outcome of a sample space is called a simple event (cannot break it down into several other events).

An event is a collection or set of one or more simple events in a sample space.

For example:Roll of a die: Simple event: The number “3” will be rolled. Event: An even number (one of 2, 4, or 6) will be rolled.

Events & Probabilities

The probability of an event is the sum of the probabilities of the simple events that constitute the event.

For example:Assuming a fair die, roll of a die: and .

Then:

Interpreting Probability

One way to interpret probability is this:

If a random experiment is repeated an infinite number of times, the relative frequency for any given outcome is the probability of this outcome.

For example, the probability of heads in flip of a balanced coin is 0,5, determined using the classical approach. The probability is interpreted as being the long-term relative frequency of heads if the coin is flipped an infinite number of times.

Joint, Marginal and Conditional Probability

We study methods to determine probabilities of events that result from combining other events in various ways.

There are several types of combinations and relationships between events: Complement of an event - (everything other than that event) Intersection of two events - (event A and event B) Union of two events - (event A or event B)

5. Why are some mutual fund managers more successful than others? One possible factor is where the manager earned his or her MBA. The following table compares mutual fund performance against the ranking of the school where the fund manager earned their MBA: Where do we get these probabilities from?

Mutual fund outperforms the

market

Mutual fund doesn’t outperform

the marketTop 20 MBA program .11 .29

Not top 20 MBA program .06 .54

E.g. This is the probability that a mutual fund outperforms AND the manager was in a top-20 MBA program; it’s a joint probability [intersection].

Alternatively, we could introduce shorthand notation to represent the events:A1 = Fund manager graduated from a top-20 MBA programA2 = Fund manager did not graduate from a top-20 MBA programB1 = Fund outperforms the market B2 = Fund does not outperform the market

0,11 0,29

0,06 0,54

E.g. = the probability a fund outperforms the market and the manager isn’t from a top-20 school.

Marginal Probabilities

Marginal probabilities are computed by adding across rows and down columns; that is they are calculated in the margins of the table:

0,11 0,29

0,06 0,54



0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83



BOTH margins must add to 1(useful error check)

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00



What’s the probability a fund manager isn’t from a top school?

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00



What’s the probability a fund manager isn’t from a top school?

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00

𝑃 (𝐴2 )=0,60



What’s the probability a fund outperforms the market?

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00



What’s the probability a fund outperforms the market?

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00

𝑃 (𝐵1 )=0,17

Conditional Probability

Conditional probability is used to determine how two events are related; that is, we can determine the probability of one event given the occurrence of another related event.

Experiment: random select one student in class.

Conditional probabilities are written as and read as “the probability of A given B” and is calculated as:

Rule of Multiplication

Again, the probability of an event given that another event has occurred is called a conditional probability.

both are true - keep this in mind!

6. What’s the probability that a fund will outperform the market given that the manager graduated from a top-20 MBA program?

Recall:A1 = Fund manager graduated from a top-20 MBA programA2 = Fund manager did not graduate from a top-20 MBA programB1 = Fund outperforms the market B2 = Fund does not outperform the marketThus, we want to know: What is ?

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00

𝑃 (𝐵1∨𝐴1 )=𝑃 (𝐵1∩ 𝐴1 )𝑃 ( 𝐴1 )

Recall:A1 = Fund manager graduated from a top-20 MBA programA2 = Fund manager did not graduate from a top-20 MBA programB1 = Fund outperforms the market B2 = Fund does not outperform the marketThus, we want to know: What is ?

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00

𝑃 (𝐵1∨𝐴1 )=0,110,40=0,275


Thus, there is a 27,5% chance that a fund will outperform the market given that the manager graduated from a top-20 MBA program.

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00

𝑃 (𝐵1∨𝐴1 )=0,110,40=0,275


Independence

One of the objectives of calculating conditional probability is to determine whether two events are related.

In particular, we would like to know whether they are independent, that is, if the probability of one event is not affected by the occurrence of the other event.

Two events A and B are said to be independent if

and .

Independence

For example, we saw that

The marginal probability for B1 is:

Since B1 and A1 are not independent events.

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00

Union

7. Determine the probability that a fund outperforms (B1) or the manager graduated from a top-20 MBA program (A1).

A1 or B1 occurs whenever: A1 and B1 occurs, A1 and B2 occurs, or A2 and B1 occurs.

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00

Rule of addition

7. Determine the probability that a fund outperforms (B1) or the manager graduated from a top-20 MBA program (A1).

0,11 0,29 0,40

0,06 0,54 0,60

0,17 0,83 1,00

Probability Rules and Trees

We introduce three rules that enable us to calculate the probability of more complex events from the probability of simpler events:

The Complement Rule: (May be easier to calculate the probability of the complement of an event and then substract it from 1,0 to get the probability of the event.)

The Multiplication Rule:

The Addition Rule:

8. A graduate statistics course has seven male and three female students. The professor wants to select two students at random to help her conduct a research project. What is the probability that the two students chosen are female?

Let F1 represent the event that the first student is female. Let F2 represent the event that the second student is female.

and events are NOT independent!

9. The professor in last example is unavailable. Her replacement will teach two classes. His style is to select one student at random and pick on him or her in the class. What is the probability that the two students chosen are female?(Both classes have 3 female and 7 male students.)

Let F1 represent the event that the student from 1-st class is female. Let F2 represent the event that the student from 2-st class is female.

and events are independent!

10. In a large city, two newspapers are published, the Sun and the Post. The circulation departments report that 22% of the city’s households have a subscription to the Sun and 35% subscribe to the Post. A survey reveals that 6% of all households subscribe to both newspapers. What proportion of the city’s households subscribe to either newspaper?That is, what is the probability of selecting a household at random that subscribes to the Sun or the Post or both?

+-

Probability Trees [Decision Trees]A probability tree is a simple and effective method of applying the probability rules by representing events in an experiment by lines. The resulting figure resembles a tree.

P( M1 ) = 7/10

P(F2|M1) = 3/9

P( M2|M1) = 6/9

P( M2 |F

1) = 7/9P(F 1) =

3/10

This is P(F1), the probability of selecting a female student first.

P(F 2|F1) = 2/9

This is P(F2|F1), the probability of selecting a female student second, given that a female was already chosen first.

First selection Second selection

F1

M1

F2

F2

M2

M2

Probability Trees (Decision Trees)At the ends of the “branches”, we calculate joint probabilities as the product of the individual probabilities on the preceding branches.

First selection Second selection

𝑃 (𝐹 1∩ 𝐹 2 )= 310 ⋅

29

Joint probabilities

𝑃 (𝐹 1∩𝑀 2 )= 310 ⋅

79

𝑃 (𝑀 1∩ 𝐹 2 )= 710 ⋅

39

𝑃 (𝑀 1∩𝑀 2 )= 710 ⋅

69

P(F 1) = 3/10

P( M1 ) = 7/10

P(F2|M1) = 3/9

P(F2|F1) = 2/9

P( M2|M1) = 6/9

P( M2|F1) = 7/9

F1

M1

F2

F2

M2

M2

Sample space:

Probability Trees

Note: there is no requirement that the branches splits be binary, nor that the tree only goes two levels deep, or that there be the same number of splits at each sub node…

11. Law school grads must pass a bar exam. Suppose pass rate for first-time test takers is 72%. They can re-write if they fail and 88% pass their second attempt. What is the probability that a randomly grad passes the bar?

00

=

First exam

P(Pass1) = 0,72

P( Fail1) = 0,28

Second exam

P(Pass2|Fail1) = 0,88

P( Fail2|Fail1) = 0,12

𝑃 (𝑃𝑎𝑠𝑠 )=0,7200+0,2464=𝟎 ,𝟗𝟔𝟔𝟒

11. Law school grads must pass a bar exam. Suppose pass rate for first-time test takers is 72%. They can re-write if they fail and 88% pass their second attempt. What is the probability that a randomly grad passes the bar?

Bayes’ Law

Bayes’ Law is named for Thomas Bayes, an eighteenth century mathematician.

In its most basic form, if we know we can apply Bayes’ Law to determine .

P(B|A) P(A|B)for example …

Breaking News: New test for early detection of cancer has been developed.

12. Clinical trials indicate that the test is accurate 95% of the time in detecting cancer for those patients who actually have cancer, but unfortunately will give a “+” 8% of the time for those patients who are known not to have cancer. It has also been estimated that approximately 10% of the population have cancer and don’t know it yet. You take the test and receive a “+” test results. Should you be worried?

LetC … event that patient has cancer … event that patient does not have cancer+ … event that the test indicates a patient has cancer - … event that the test indicates that patient does not have cancer


𝑃 ¿

Cancer Test𝑃 (+∩𝐶 )=0,095Joint probabilities

𝑃 (−∩𝐶 )=0,005

𝑃 (+∩𝐶 )=0,072

𝑃 (−∩𝐶 )=0,828

P(C) = 0,10

P() = 0,90P(+|) = 0,08

P(+|C) = 0,95

P( -|) = 0,92

P( -|C) = 0,05

C

+

+

-

-𝑪

=0,095/(0,095+0,072)=0,569

𝑃 ¿


Bayesian Terminology

The probabilities and are called prior probabilities because they are determined prior to the decision about taking the preparatory course.

The conditional probability is called a posterior probability (or revised probability), because the prior probability is revised after the decision about taking the preparatory course.

Students Work – Bayes Problem

1. Transplant operations for hearts have the risk that the body may reject the organ. A new test has been developed to detect early warning signs that the body may be rejecting the heart. However, the test is not perfect. When the test is conducted on someone whose heart will be rejected, approximately two out of ten tests will be negative (the test is wrong). When the test is conducted on a person whose heart will not be rejected, 10% will show a positive test result (another incorrect result). Doctors know that in about 50% of heart transplants the body tries to reject the organ.

*Suppose the test was performed on my mother and the test is positive (indicating early warning signs of rejection). What is the probability that the body is attempting to reject the heart?*Suppose the test was performed on my mother and the test is negative (indicating no signs of rejection). What is the probability that the body is attempting to reject the heart?

2. The Rapid Test is used to determine whether someone has HIV [H]. The false positive and false negative rates are 0.05 and 0,09 respectively.The doctor just received a positive test results on one of their patients [assumed to be in a low risk group for HIV]. The low risk group is known to have a 6% probability of having HIV. What is the probability that this patient actually has HIV [after they tested positive].

3. Marie is getting married tomorrow, at an outdoor ceremony in the desert. In recent years, it has rained only 5 days each year. Unfortunately, the weatherman has predicted rain for tomorrow. When it actually rains, the weatherman correctly forecasts rain 90% of the time. When it doesn't rain, he incorrectly forecasts rain 10% of the time. What is the probability that it will rain on the day of Marie's wedding?

intro to probability

Documents