hypothesis testing and the gamma functionribet/10a/2016/nov29.pdfheads. in that situation, we...
TRANSCRIPT
Hypothesis testing and the Gamma function
Kenneth A. Ribet
Math 10ANovember 29, 2016
Kenneth A. Ribet Hypothesis testing and the Gamma function
Announcements
Please send me email to sign up for the next two (last two?)breakfasts:
This Thursday (December 1) at 9AM.Next Monday (December 5), also at 9AM.
Pop-in lunch on Friday, December 2 at 12:10PM (no sign-upsneeded, just show up)
Coffee at Blue Bottle Coffee Company (University Avenue nearLi Ka Shing) on Friday, December 2 at 9AM.
Kenneth A. Ribet Hypothesis testing and the Gamma function
Today we will discuss the Z -test (through examples), meet theGamma function and (probably) start our discussion ofStudentās t-testing.
The Z -test business is sort of a review of last weekās class.
The t-subject is the last topic in the Math 10A syllabus.
On Thursday, we will discuss t distributions and testing, maybetake a class photo and then have a course evaluation sessionwhere I leave and the students fill out their course evaluationsonline while theyāre still in the room.
Kenneth A. Ribet Hypothesis testing and the Gamma function
This comes from the math department:
If your class meets on Tuesdays and Thursdays youcould mention on Tuesday that you will be devoting thelast 15 minutes of Thursdayās class to the onlinecourse evaluations, and you could ask the students tobring their laptop computers to Thursdays class. . . . Itwould be a good idea for you to leave the classroomfor those last 15 minutes while students arecompleting their evaluations of your teaching.Students can access the evaluation system throughthe following link:https://course-evaluations.berkeley.edu.
As of 1PM on Tuesday, about 100 students had completed theirMath 10A evaluations.
Kenneth A. Ribet Hypothesis testing and the Gamma function
Last Thursdayās pop-in lunch (non-vegetarian version)
Kenneth A. Ribet Hypothesis testing and the Gamma function
Z -test example
Wikipedia (rephrased):
Suppose that in a particular geographic region, themean and standard deviation of reading test scoresare 100 points, and 12 points, respectively. . . .Fifty-five students in School X . . . received a meanscore of 96. . . . are the students in this schoolcomparable to a random sample of 55 students fromthe region as a whole, or are their scores surprisinglylow?
Kenneth A. Ribet Hypothesis testing and the Gamma function
The scores of random generic students are given as values of arandom variable X with mean Āµ = 100 and standard deviationĻ = 12. If we have N independent copies Xi of X (with N = 55,for example), then
X =X1 + Ā· Ā· Ā·+ XN
N
is (approximately) normally distributed with mean Āµ andstandard deviation
ĻāN
.
As we all saw last Tuesday,
Z =(X ā Āµ)
āN
Ļ
is then normal(ish) with mean 0 and standard deviation 1.
An essentially correct statement is that Z is a standard normalvariable in the limit (i.e., for N āā).
Kenneth A. Ribet Hypothesis testing and the Gamma function
When we evaluate this random variable on our sample ofstudents, X = 96 (the mean score of the 55 students inquestion) and N = 55. Thus
Z =(96ā 100)
ā55
12ā ā2.472.
This value occurs far out from the mean (which is 0). Theprobability of getting a Z thatās ā2.472 or less can be boundedby using the table on page 566 of Schreiber.
Namely, 49.18% of the scores occur between 0 and 2.4; thusfewer than 1% of the scores are larger than 2.4. Symmetrically,fewer than 1% of the scores are smaller than ā2.4. Thus theprobability of getting ā2.472 or a lower score is way less than1%. (Wikipedia says itās about 0.0068.)
Kenneth A. Ribet Hypothesis testing and the Gamma function
If you prefer two-sided statements, you can say that fewer than2% of students in a random sample will get adjusted scores thatare either greater than 2.472 or less than ā2.472. In any case,weāre discussing p-values (probabilities) that are less than 5%.
Kenneth A. Ribet Hypothesis testing and the Gamma function
Conclusion? We reject the null hypothesis that the 55 studentsfrom the particular school were drawn from a random sample ofstudents in the entire region.
Other ways to communicate this information:The students from this school have surprisingly low scores.There is statistical significance to the low scores of thestudents.
Kenneth A. Ribet Hypothesis testing and the Gamma function
Second example: Example 5.2 from Prob-Stat onlinenotes
āWe see a man flipping a coin and he gets 65 heads out of 100tosses. If p is the probability of heads, we wish to test whetherp 6= 0.5.ā
Why is the coin flipper a man?
This example is like that in the HW where 100 tosses led to 70heads. In that situation, we rejected the null hypothesis that thecoin is fair. This is again the null hypothesis, but 65 is lessthan 70, so the situation is less clear-cut.
Kenneth A. Ribet Hypothesis testing and the Gamma function
If we plug in the data, we get for
(X ā Āµ)ā
NĻ
the numerical value
(0.65ā 0.5) Ā· 101/2
= 0.15 Ā· 20 = 3.
Being three standard deviations away from the mean is anextremely low-probability event. (Itās worse than being 2.4standard deviations away from the mean.) Specifically, beingthree standard deviations above the mean occurs withprobability 1ā 0.9987 (according to page 566 of the book).This number is about 0.13%. Being at least three standarddeviations away from the mean (i.e., three above or threebelow) occurs with double the probability. The number involved,about .26%, is still minuscule.
We reject the null hypothesis. We think that the coin is not fair.Kenneth A. Ribet Hypothesis testing and the Gamma function
A Penn State example
Boys of a certain age have a mean weight of 85pounds. A complaint is made that the boys living in amunicipal childrenās home are underfed. Twenty-fiveboys (of this age) are weighed and found to have amean weight of 80.94 pounds. The populationstandard deviation is 11.6 pounds. . . . what should beconcluded concerning the complaint?
The null hypothesis is that the mean weight of children in this
home is 85 pounds. The statistic Z is80.94ā 85
11.6
ā25 = ā1.75.
For a normal distribution with mean 0 and standard deviation 1,the probability of being ā¤ ā1.75 is 0.5ā 0.4599, according totable 7.3 on page 566 of Schreiber. This number is around0.04, which is less than 0.05. Hence we reject the nullhypothesis and conclude that the complaint is valid.
Kenneth A. Ribet Hypothesis testing and the Gamma function
Kenneth A. Ribet Hypothesis testing and the Gamma function
In quite a few situations involving statistics, one encountersvalues of the Gamma function Ī(s). Here is a quick tutorialon Ī.
Definition: for real numbers s > 0,
Ī(s) =
ā« ā0
tseāt dtt
=
ā« ā0
tsā1eāt dt .
For example,
Ī(1) =
ā« ā0
eāt dt = āeāt]ā
0= 1.
Kenneth A. Ribet Hypothesis testing and the Gamma function
Integration by parts:
Ī(s) =
ā« ā0
tsā1eāt dt = ātsā1eāt]ā
0+
ā« ā0
(s ā 1)tsā2eāt dt
= 0 + (s ā 1)Ī(s ā 1).
Thus, for example:
Ī(2) = 1 Ā· Ī(1), Ī(3) = 2 Ā· Ī(2), Ī(4) = 3 Ā· Ī(3), . . . .
Since Ī(1) = 1, we get Ī(2) = 1, Ī(3) = 2, Ī(4) = 6, etc. Ingeneral, we see recursively that Ī(n) = (n ā 1)!. (In 10B, onewould prove this by induction.)
Another way to write the formula:
Ī(s + 1) = sĪ(s).
Kenneth A. Ribet Hypothesis testing and the Gamma function
There is a well known duplication formula for the Gammafunction that is usually ascribed to Gauss. According toWikipedia, it is also known as the Legendre duplication formula:
Ī(s)Ī(s +12
) = 21ā2sāĻ Ī(2s).
Take s =12
:
Ī(12
)Ī(1) = 20āĻ Ī(1).
Because Ī(1) = 1, we get
Ī(12
) =āĻ.
This is good to know because theāĻ in the formula for the
standard Gaussian is often written Ī(1/2) in the literature.
Kenneth A. Ribet Hypothesis testing and the Gamma function
For another example, we can calculate Ī(3/2) in two ways. Bythe duplication formula,
Ī(3/2)Ī(2) = 21ā3āĻ Ī(3),
so
Ī(3/2) =11Ā· 1
4āĻ Ā· 2 =
āĻ
2.
We can also write
Ī(3/2) = Ī(12
+ 1) =12
Ī(12
) =
āĻ
2.
Kenneth A. Ribet Hypothesis testing and the Gamma function
In the Math 10B notes, thereās a critical moment where youread: ā. . . Recall from last semester what you learned about theGamma function. . . .ā
Last semester, I was just barely able to get some of mystudents to admit that they had been exposed to the Gammafunction in 10A. Iām hoping that you all will nod enthusiasticallynext semester when the Gamma function arises.
Spoiler: youāll see Gamma values in the formula for the PDF ofChi Squared distributions, whatever they are. (Youāll learn aboutthem in 10B.)
Kenneth A. Ribet Hypothesis testing and the Gamma function
In the Math 10B notes, thereās a critical moment where youread: ā. . . Recall from last semester what you learned about theGamma function. . . .ā
Last semester, I was just barely able to get some of mystudents to admit that they had been exposed to the Gammafunction in 10A. Iām hoping that you all will nod enthusiasticallynext semester when the Gamma function arises.
Spoiler: youāll see Gamma values in the formula for the PDF ofChi Squared distributions, whatever they are. (Youāll learn aboutthem in 10B.)
Kenneth A. Ribet Hypothesis testing and the Gamma function
Student
āStudentā was William Sealy Gosset, 1876ā1937. He was theinventor of Studentās t-test, which we will now explore.
The first thing to understand is that there are multipledistributions named after Student; one speaks of āStudentāst-distribution," but there is a parameter in the distribution, thenumber of degrees of freedom. This number is Ī½ (ānuā).Because Ī½ can vary (Ī½ = 1,2,3 . . .), there is really a family ofdistributions.
The number Ī½ will be one less than the number of samples (Nor n, depending on notation) that one draws.
Kenneth A. Ribet Hypothesis testing and the Gamma function
Wikipedia:
Whereas a normal distribution describes a fullpopulation, t-distributions describe samples drawnfrom a full population; accordingly, the t-distribution foreach sample size is different, and the larger thesample, the more the distribution resembles a normaldistribution.
What is t? Itās a statistic like Z thatās defined by a formula thatāsnearly a clone of the formula
Z =(X ā Āµ)
āN
Ļ.
The main difference is that t is applied in situations where Āµ isknown but Ļ is unknown and is replaced by an estimate. Usingthe estimate in place of the (unkown) true value changes thedistribution. . . .
Kenneth A. Ribet Hypothesis testing and the Gamma function
On Thursday we will see how the t-distributions are used forhypothesis testing.
Today we are just trying to figure out what the distributions are.
Kenneth A. Ribet Hypothesis testing and the Gamma function
The Ī½th Student PDF is given by
C Ā· (1 +x2
Ī½)ā(Ī½+1)/2,
where C is a constant chosen so that the total area under thecurve is 1 (as itās supposed to be).
If you graph y = (1 +x2
Ī½)ā(Ī½+1)/2 for different values of Ī½, youāll
see a function thatās symmetrical about the y -axis (because ofthe x2) and that looks qualitatively like the PDF of the standardnormal.
As Ī½ āā, you can check (and please do check!) that
(1 +x2
Ī½)ā(Ī½+1)/2 ā eāx2/2.
Kenneth A. Ribet Hypothesis testing and the Gamma function
To perform the check, you need to know the formula
limnāā
(1 +
an
)n= ea
when a is a real number. This can be verified (for example) bytaking logs and using lāHĆ“pitalās Rule.
Kenneth A. Ribet Hypothesis testing and the Gamma function
Graphs of (1 +x2
Ī½)ā(Ī½+1)/2 for Ī½ = 1,2,3:
-3 -2 -1 0 1 2 3
0.2
0.4
0.6
0.8
1
Color code: Ī½ = 1, magenta; Ī½ = 2, gold; Ī½ = 3, blue.
Kenneth A. Ribet Hypothesis testing and the Gamma function
The constant C
We need to take
C = 1/ā« ā
āā(1 +
x2
Ī½)ā(Ī½+1)/2 dx .
The easiest case is that where Ī½ = 1. Because theantiderivative of
11 + x2 is tanā1(x),
ā« āāā
dx1 + x2 = Ļ, so C = 1/Ļ.
Kenneth A. Ribet Hypothesis testing and the Gamma function
In general,
C =1āĪ½Ļ
Ī(Ī½+12 )
Ī(Ī½2 ).
This work when Ī½ = 1 because
Ī(Ī½ + 1
2) = Ī(1) = 1, Ī(
Ī½
2) = Ī(
12
) =āĻ.
According to Wikipedia, C involves Ļ when Ī½ is odd but notwhen Ī½ is even.
Kenneth A. Ribet Hypothesis testing and the Gamma function
About the final
Itās cumulative but probably will have more on the last third ofthe course than on the second third, and more on the secondthird than on the first third. Thatās only because whatās fresh inoneās mind usually gets extra attention.
The final exam is on Friday night, 7ā10PM, exam group 20.
However, we donāt yet know where it is. Do not show up in 2050VLSB without first checking the final exam schedule!!!!
Kenneth A. Ribet Hypothesis testing and the Gamma function