Chapter 6

ST 370 - Probability

Readings: Chapter 2.2-2.6

Recap: So far we’ve learned:

• Why we want a ‘random’ sample and how to achieve it (Sampling Scheme)

• How to use randomization, replication, and control/blocking to create a valid experi-ment.

• Methods for summarizing and graphing data in meaningful ways.

• Analyzing CRD type designs to investigate which factors are important for a particularresponse.

• How to analyze studies with quantitative explanatory variables and a quantitative re-sponse.

• Next up we will learn about probability and random variables which will allow us tounderstand the inferences we’ve looked at!

In ANOVA and Regression we used the mean squares to find F-stats.

The F-stats were used to calculate the p-values.

But how did we come up with that? Now we will look at the mechanics of probabilityand eventually sampling distributions.

87

Interpretation of Probability and Inference

• Probability is the of an event, in an ex-periment repeated an infinite number of times

– Ex: the probability of rolling snake eyes ((1, 1)) on two fair dice

– Ex: the probability of getting a head on a flipped coin

Theory of probability provides the basis for statistical inference (making claims about apopulation based on sample data)

• Ex: Suppose a coin is tossed n = 10 times and yields y = 10 heads. We want to testthe hypothesis:

– If hypothesis is true, how likely is this event?

– With 10/10 heads, reasonable to conclude not fair. What about 9/10? 7/10?

– To make a decision, need to know

The above plot gives the probability of observing a given number of heads from a fair coinin 10 tosses.

88

Probability

• The Probability of the event is the likelihood or chance that a particular outcome orevent from a random process will occur. We write

• Probabilities are numbers between

• May be written as proportion (0.15), percent (15%), or a fraction (3/20).

• P(Event)=1 implies

• P(Event)=0 implies

• P(one of all possible events occur) =1

Example - Use the relative frequency idea to find the probabilities associated with theevents possible from flipping a fair coin twice.

• ∩ is the intersection of two events

• ∪ is the union

Example - What is the probability of getting 0 heads? 1 head? 2 heads? The probabilityof getting 0, 1, or 2 heads? Consider the events A= I get 1 or 2 heads and B= I get 0 or 1heads. What is the probability of both A and B occurring?

89

Example - Try this with flipping a coin three times! That is, write out the events and findthe probability of obtaining 0 heads in 3 flips, 2 heads in 3 flips, 1 head in 3 flips, and 0heads in 3 flips. Also, find the probability of obtaining at least 1 head. Find the probabilityof obtaining at least 1 head and no more than 2 heads.

Conditional Probs, Laws of Probs, and Independence

The probability of an event will sometimes depend on whether we know that other eventshave occurred.

For example, the probability of getting a 1 in the toss of a six-sided die is .

If we know that an odd number has fallen, then the probability of occurrence of a 1 is.

The conditional probability of an event A given that an event B has occurred is equal to

Basic Example - The game host puts $10K, $100 and $1 in three briefcases. You, the player,can select any case you like. Then you are offered the second chance to select another caseand the case will be opened. Suppose you see the second case does not have $10k, then whatis the chance that the first chosen case has $10K dollars?

90

Independent EventsTwo events and are said to be if and only if any one of thefollowing 3 conditions hold:

P (A|B) = P (A)

P (B|A) = P (B)

P (A ∩B) = P (A)P (B).

Otherwise, the events are said to be

Independence Examples

• Consider a couple that has 2 children. We are looking at the gender of the children inorder. The events possible here are MF (1st chile male, second female), FM , FF , andMM . All events are equally likely. Let A be the event that the younger child is female,and B be the event that the older child is male. Are A and B dependent?

• (Credit Card Example) - The proportion of NCSU students with a VISA card is 0.48,the proportion with a MasterCard is 0.64, the proportion with both is 0.35.

1. Calculate the conditional probability that a randomly sampled student has a VISAgiven he/she has a MasterCard.

2. Are the events ‘having a VISA’ and ‘having a MasterCard’ independent?

91

Laws of ProbabilitySometimes probabilities of events can be obtained by using multiplicative and additive rules.

:

P (A ∩B) = P (B|A)P (A) = P (A|B)P (B)

Notice that if A and B are , then

P (A ∩B) = P (A)P (B)

The multiplicative law can be extended to cover 3 or more events:

P (A ∩B ∩ C) = P (A ∩B)P (C|A ∩B) = P (A)P (B|A)P (C|A ∩B).

Using the Multiplicative Law(Urn Example) - An urn contains 10 marbles, 4 are red (R) and 6 are black (B).

1. If 2 are randomly chosen from the urn, what is the probability that both are black?

2. If 1 is randomly chosen, then replaced, and then another randomly chosen (making theselections independent evenets), what is the probability of selecting a red then a black?

Example - Flip a fair coin 3 times, find the probability of observing 3 heads (HHH) assumingindependent flips.

92

:

P (A ∪B) = P (A) + P (B)− P (A ∩B)

Note: If A and B are disjoint, then P (A ∪B) = P (A) + P (B).

:

P (A ∪B ∪ C) = P (A) + P (B) + P (C)− P (A ∩B)

−P (A ∩ C)− P (B ∩ C) + P (A ∩B ∩ C)

Using the Additive Law

• (Credit Card Example) - The proportion of NCSU students with a VISA card is 0.48,the proportion with a MasterCard is 0.64, the proportion with both is 0.35.Find the probability that a randomly sampled student has a VISA or MasterCard (orboth).

• Can A and B be disjoint if P (A) = 0.4 and P (B) = 0.7? What if P (B) = 0.3?

93

A special case of additive law is obtained by taking B = Ac, where Ac is the complement ofA.

:

P (A) + P (Ac) = 1 implies P (A) = 1− P (Ac)

(de M’er’e’s problem) - This marked the birth of probability theory.In 17th century, once De M’er’e asked Pascal, which is more likely:

• A: rolling at least one six in four throws of a single dice

• B: rolling at least one double six in 24 throws of a pair of dice?

94

Combining Rules (System reliability Example) - Ai=“Component i works”, P (Ai) = 0.9,i = 1, · · · , 4. Suppose components work or fail independently.

Calculate the probability that the system works, i.e. either both 1 and 2 work, or both 3and 4 work.

95

Chapter 7

ST 370 - Random Variables

Readings: Chapter 2.8, Chapter 3.1-3.4, 3.6, Chapter 4.1-4.4, 4.6-4.7

Motivating Ex.:Assume that all men’s basketball teams playing this season are equally strong. We areinterested in the number of points scored by NC State in each game.

• Before each game, we know the population of possible values.

• Each value occurs with some probability.

• However, we do not know what will be the number of points scored by NC State duringthe next game.

The outcome is random, hence a random variable.

• A Random Variable (RV) is a real-valued function

– Domain =

– Range =

An RV assigns a real number to each outcome possible.

Two Types of RVs we’ll discuss

• : takes on a subset of intervals of real numbers

• : takes on finite or countably infinite # of values

96

Discrete Random Variables

Example

• Discrete random variable assumes only a finite or countably infinite # of values

• Ex: Flip a coin 3 times - Let Y = # of heads from the 3 tosses

– Range of Y ?

– Called in probability

• Each outcome has a

• Function f(y) = P (Y = y) is called the

Distribution

• PMF can be represented as a table:

Value of Y y1 y2 ... ynProbability f(y1) f(y2) ... f(yn)

• Or by a formula

fY (y) = P (Y = y) =

(3

y

)0.5y(1− 0.5)3−y

for y = 0, 1, 2, 3 (and fY (y) = 0 otherwise).

97

• Our interest will often lie in characteristics of the RV such as:

– What is the

– What is the

– How much

• In the end, knowing these things will allow us to make inference!

Properties of discrete probability distributions

To be a pmf, f(y) must have the properties -

1. for every y

2. , where∑

y is the sum taken over the support of Y

(i.e. all realizations of Y where f(y) > 0)

Let’s check the pmf of the coin example meets these requirements.

98

Computing probabilities

• Note that for different realizations yi and yj, the events

• Using what we know about probabilities:

P (Y = yi or Y = yj) =

• Let’s compute P (Y ≥ 2) for the coin example

Computing probabilities

• Generally, we can use this logic to find: (Note: We do need to be careful about ≥ vs >and ≤ vs <)

=∑y:y≤b

f(y)

=∑y:y>a

f(y)

=∑

y:a≤y≤b

f(y).

99

Another Example Let X (Any capital can be used to denote a RV) = # of attempts topass a driving test. Suppose the tests are independent and you pass with probability 0.7.

• Determine the support of X

• Find f(x), the pmf of X

• Find∑

x f(x)

• Find P (X =≤ 2 or X = 4)

• Find P (X ≥ 3)

100

The pmf defines a RV’s

• The does as well

Example - What is the CDF for the driving attempts example?

Example - There is a chance that a bit transmission is received in error. Let Y = # of bitsin error for the next four bits transmitted.

• Possible outcomes of Y ?

Value of YProbability 0.6561 0.2916 0.0486 0.0036 0.0001

• What is the cdf of Y ? Let’s plot it as well.

101

Binomial Distribution

Recognizing a Distribution

• Note: Coin example has a common structure to it.

• Similar experiments with similarly defined RV’s yield the same

• This particular distribution so common, it is called the

• Knowing and being able to recognize common distributions will save us from having toderive things over and over!

When does a RV follow the Binomial Distribution?Consider the following experiments:

• a coin is flipped, the outcome is either a head or a tail.

• a baby is born, the baby is either born in March or is not.

In each of these examples, there are two

For convenience, we can label these

Bernoulli Trials

• An experiment with only two possible mutually exclusive outcomes (such as S or F) iscalled a Beroulli Trial.

– Bernoulli trials are the basis of three families of distributions:

∗ distribution

∗ distribution

∗ distribution

• Denote the as P (S) = p

• Then the is P (F ) = 1− p = q

102

We have a Binomial Experiment if:

1. Full experiment consists of a sequence of

2. on each trial (BernoulliTrials)

3. Probability of success P (S) = p is , where 0 ≤p ≤ 1

Define the RV Y =

Then Y is said to follow a binomial distribution.

Defining the PMF of the Binomial DistributionEx: Suppose we have a Binomial Experiment with n = 3 trials and P (S) = p where p is anunknown parameter

• Let Y be the # of successes –

• Support of Y ?

Computing P (Y = y)

Outcome P (Outcome) y Reps P (Y = y)SSS ppp = p3 3 1 p3

SSF

SFS

FSS

SFF

FSF

FFS

FFF

103

PMF of the Binomial Distribution For general n, the event (Y = y) occurs when thereare exactly

• Consider one such outcome w/1st y trials successful last n− y failures:

SSS · · ·SFFF · · ·F

• Probability of this outcome?

• How many different sequences with exactly y successes in n trials?

The pmf of a binomial distributed random variable Y is:

y = 0, 1, 2, · · · , n, 0 ≤ p ≤ 1

Binomial Distribution ExampleSuppose 60% of NCSU students favor closed-book exams. A random sample (outcomesindependent) of 5 NCSU students is drawn.

1. Define Success/Failure, n, p, and a RV Y that follows the Binomial distribution

2. Calculate P (exactly 1 in favor)

3. Calculate P (less than 2 in favor)

4. Calculate P (4 or more in favor)

104

Characteristics of RVs that are of InterestQuestions of interestAs we said earlier, we will be interested in things like the of the

RV or the in the observed outcomes etc.

• Want to know the mean or average value of the RV, i.e. Expected Value - denoted as

• Interested in the variability of the RV, i.e. Variance - denoted as

• Expected Value of a discrete RV Y with pmf f(y) is

– Sum over the entire support of Y

• If Y has support y1, y2, · · · , ym, then

• Notice this mean is denoted by µY , a greek symbol. This implies it is the true

Binomial Expected Value

• Let’s calculate the expected value of Y from the 3 coin example

• Generally, if Y ∼ Bin(n, p),

105

• Variance of a discrete RV Y with pmf p(y) is

– Again sum over entire support of Y

• Standard Deviation of Y is defined as the positive square root of V ar(Y )

• To actually find a variance, it is often easier to work with the Variance–ComputingFormula

V ar(Y ) = E(Y 2)− [E(Y )]2

– where E(Y 2) =∑

y y2 × f(y)

• Let’s calculate V ar(Y ) from the 3 coin example

• Generally, if Y ∼ Bin(n, p), then

Multiple Choice Test ExampleConsider a multiple choice test with 20 questions, each with five possible answers (a,b,c,d,e),only one of which is correct. Let Y= # of questions guessed correctly.

1. Let’s verify Y follows a binomial, calculate E(Y ), and calculate V ar(Y ).

2. If scores of 50% and higher are passing, find the formula (i.e. don’t simplify) for theprobability of passing by guessing.

106

Other Discrete Distributions

• We have looked at our quantities of interest only using the Binomial Distribution

• There are many other distributions that aren’t named but some others that are - suchas:

– Hypergeometric

– Negative Binomial

– Poisson

– Discrete Uniform

A No-Name Distribution Example

• Suppose the probability of winning a given amount on a scratch off lottery ticket isWin($) Chance

0 0.821 0.15 0.0510 0.0220 0.009

1000 0.001Total 1

• Let’s find the mean, variance, and standard deviation of the RV: Y = Amount Won

107

Chapter 3 Making Inference Hypothesis Testing Idea:You love Pepsico and their products. They are having a promotion where their bottle capsare either winners (a free Pepsico product) or losers. Your friend claims you will hardly everwin, in fact he thinks only 1 in 20 bottles is a winner. You think the chance of winning ismuch higher than that.

To prove him wrong you grab 50 randomly selected Pepsico bottles and find that 12 ofyour caps are winners. How can we show your friend you are most likely correct?

108

Continuous Random Variables

A has an interval or collection of intervals as its supportEx:

• Y=maximum daily temperature (interval [−40◦F, 130◦F ]).

• Y=lifetime (in years) of electronic equipment 0 < Y <∞

• Y=weight loss (or gain) after a 6 month period −∞ < Y <∞.

For Discrete RVs we had the pmf, f(y) = P (Y = y).

For Continuous RVs we have the pdf, or

• A pdf, f(x), describes the probability distribution of a continuous random variable.

• Probability a randomly chosen value will lie between any 2 given values is representedin terms of the area between the two values under the pdf.

A function f(y) is a pdf if

1. f(y) is a , i.e. f(y) ≥ 0 for all y

2. The area under f(y) is one, i.e. the P (−∞ < Y <∞) = 1

Similar to pmf property∑

y f(y) = 1

We can find probabilites using integrals:

109