Testing Differences between Testing Differences between Means, continuedMeans, continued

Statistics for Political ScienceLevin and FoxChapter Seven

Testing Differences between MeansTesting Differences between Means

To test the significance of a To test the significance of a mean difference mean difference we need to find the standard we need to find the standard deviation for any obtained mean difference. deviation for any obtained mean difference.

However, we rarely know the However, we rarely know the standard deviation of the distribution of mean standard deviation of the distribution of mean differences differences since we rarely have population datasince we rarely have population data. . Fortunately, it can be Fortunately, it can be estimated based on two samples that we draw from the same populationestimated based on two samples that we draw from the same population..

Step 2bStep 2b: Translate our sample : Translate our sample mean difference mean difference into units of standard deviation.into units of standard deviation.

33

Z = ( 1 – 2) - 0

Where = mean of the first sample

= mean of the second sample

0 = zero, the value of the mean of the sampling distribution of differences between means (we assume that µ1 - µ2 = 0)

= standard error of the mean (standard deviation of the distribution of the difference between means)

We can reduce this equation down to the following:

X X

21 XX

1X

2X

21 XX

21

21

XX

XXz

Remember this formula required the standard deviation of the distribution of mean differences.

Result:Result: (assuming equals 2) (assuming equals 2)

44

Z = ( 45 – 40)

Thus, a difference of 5 between the means of the two samples (women and men) falls 2.5 standard deviations from a mean of zero.

21 XX

2

Z = + 2.5

Child Rearing: Comparing Males and Females

Standard Error of the Difference Standard Error of the Difference between Meansbetween Means

Here is how the Here is how the standard error of the difference between means standard error of the difference between means can be can be calculated.calculated.

21

21

21

222

211

21 2 NN

NN

NN

sNsNs

xx

The formula for combines the information from the two samples.21 XX

s

Where Where

The formula for combines the information from the two samples.

A large difference between Xbar1 and Xbar2 can result if (1) one mean is very small, (2) one mean is very large, or (3) one mean is moderately small and the other is moderately large.

21 XXs

22

2

222

2

21

1

212

1

XN

Xs

XN

Xs

Variance: Weeks on Unemployment: Variance: Weeks on Unemployment:

X(weeks) N=6

Deviation:

(raw score from the mean)

(raw score from the mean, squared)

Variance:

9 8 6 4 2 1

9-5= 48-5=36-5=14-5=-12-5=-31-5=-4

42 = 1632 = 912 = 1-12 = 1-32 = 9-42 = 16

(weeks squared)

ΣX=30 χ= 30=5 6

(X X) (X X)2

(X X)2 52

Step 1: Calculatethe Mean

Step 3: CalculateSum of square Dev

Step 2: CalculateDeviation

Step 4: Calculatethe Mean of squared dev.

s2 X X 2N

52

68.67

Testing the Difference between MeansTesting the Difference between Means

Let’s say that we have the following information about Let’s say that we have the following information about two samplestwo samples, one of , one of liberalsliberals and one of and one of conservativesconservatives, on the progressive scale:, on the progressive scale:

Liberals Conservatives

N1 = 25 N2 = 35

= 60 = 49

S1 = 12 S2 = 14

We can use this information to calculate the estimate of the We can use this information to calculate the estimate of the standard standard error of the difference between meanserror of the difference between means::

1X 2X

21

21

21

222

211

21 2 NN

NN

NN

sNsNs

xx

)35)(25(

3525

23525

)14)(35()12)(25( 22

21 xxs

52.3

3717.12

)0686)(.3448.180(

875

60

58

860,6600,3

We start with We start with our formula:our formula:

The standard error of the difference between means is The standard error of the difference between means is 3.523.52..

We can now use our result to translate the difference between sample We can now use our result to translate the difference between sample means to a means to a t ratiot ratio..

We can now use our standard error results to change difference between sample mean into a t ratio:

21

21

XXs

XXt

t = 60 – 49

3.52

t = 11

3.52

t = 3.13

REMEMBERREMEMBER: We : We useuse tt instead of instead of zz because we do because we do not know the true population not know the true population standard deviation.standard deviation.

We aren’t finished yet!We aren’t finished yet!Turn to Table CTurn to Table C..

1) Because we are estimating for both σ1 and σ2 from s1 and s2, we use a wider t distribution, with degrees of freedom N1+ N2 – 2.

2) For each standard deviation that we estimate, we lose 1 degree of freedom from the total number of cases.

N = 60Df ( 25 + 35 - 2) = 58

In Table C, use a critical value of 40 since 58 is not given.We see that our t-value of 3.13 exceeds all the standard critical points except

for the .001 level.

Therefore, based on what we established BEFORE our study, we reject the null hypothesis at the .10, .05, or .01 level.

df .20 .10 .05 .02 .01 .001

40 1.303 1.684 2.021 2.423 2.704 3.551

Comparing the Same Sample Measured TwiceComparing the Same Sample Measured Twice

Some research employs a Some research employs a panel design panel design oror before and before and after test after test (testing the (testing the same samplesame sample at two points in time). at two points in time).

In these types of studies, the In these types of studies, the same samplesame sample is tested twice. It is not two is tested twice. It is not two samples from the same population, samples from the same population, it is a measuring the same group of it is a measuring the same group of people twicepeople twice. .

CRITICAL POINTS TO NOTECRITICAL POINTS TO NOTE::1. The 1. The samesame sample measured twice uses the sample measured twice uses the t-test of difference between means..2.2. Different samples from the same population selected at two points in Different samples from the same population selected at two points in time use the time use the t-test of difference between means for independent groups..

Example Problem of Test of Difference Between Example Problem of Test of Difference Between Means for Same Sample Measured TwiceMeans for Same Sample Measured Twice

Null Hypothesis (Null Hypothesis (µµ11 = µ = µ22)):: The degree of neighborliness The degree of neighborliness does not differdoes not differ before and before and after relocation.after relocation.

Research Hypothesis (Research Hypothesis (µµ11 ≠ µ ≠ µ22)):: The degree of neighborliness The degree of neighborliness differsdiffers before and after before and after relocation.relocation. Where µ1 is the mean score of neighborliness at time 1 Where µ2 is the mean score of neighborliness at time 2Respondent Before

(X1)After(X2)

Difference(D = X1 – X2)

Difference2

(D2)

Johnson 2 1 1 1

Robinson 1 2 -1 1

Brown 3 1 2 4

Thomas 3 1 2 4

Smith 1 2 -1 1

Holmes 4 1 3 9

∑ X1 = 14 ∑ X2 = 8 ∑ D2 = 20

221

2

)( XXN

DsD The formula for obtaining

the standard deviation for the distribution of before-after difference scores

sD = standard deviation of the distribution of before-after difference scores

D = after-move raw score subtraction from before-move raw score

N = number of cases or respondents in sample

From this, we get the formula for the standard error of the difference between the means:

SD

SDN 1

SD

1.53

6 1X1

X1n

X 2 X 2n

14 6

=

=

=

=2.33 1.33

8 6

sD 20

6 (2.33 1.33)2

= 1.53

= .68

t X1 X2

sD t = 60 – 49

3.52

t = 3.13

Step 1: Find mean for each point in time

Step 2: Find the SD for the diff between the times

Step 3: Find the SE for the diff between the times

Step 4: Translate the mean diff into a t score

Comparing the Same Sample Measured TwiceComparing the Same Sample Measured Twice

df = (n – 1) df = (n – 1) = 6 – 1 = 6 – 1 = 5= 5

Step 5: Calculate the degrees of freedom

Step 6: compare the obtained t ratio with t ratio in Table C

Obtained t = 1.47Obtained t = 1.47Table t = 2.571Table t = 2.571df = 5df = 5α = .05

To order reject the null hypothesis at the .05 significance with five degrees of freedom we must obtain a calculated t ratio of 2.571. Because our t ratio is only 1.47 – we retain the null hypothesis.

df .20 .10 .05 .02 .01 .001

5 1.476 2.015 2.571 3.365 4.032 6.859

Two Sample Test of ProportionsTwo Sample Test of Proportions

21

21

PPs

PPz

21

21*)1(*21 NN

NNPPs PP

21

2211*NN

PNPNP

Where P1 and P2 are respective sample proportions.

The standard error of the difference in proportions is:

Where P* is the combined sample proportion

Requirements when considering the appropriateness of the t-Requirements when considering the appropriateness of the t-ratio as a test of significance. (For Testing the Difference ratio as a test of significance. (For Testing the Difference between Means):between Means):

1.1. TheThe t ratio t ratio is used to make comparisons between is used to make comparisons between two meanstwo means..2.2. The assumption is that we are working with The assumption is that we are working with intervalinterval level data. level data.3.3. We used a We used a random sampling processrandom sampling process..4.4. The sample characteristic is The sample characteristic is normally distributednormally distributed..5.5. The t ratio for independent samples assumes that the population The t ratio for independent samples assumes that the population

variances are equal. variances are equal.

So how do you interpreting the results and state them for So how do you interpreting the results and state them for inclusion in your research?inclusion in your research?

““Since the observed value of t (state the test statistic) exceeds the critical value (state the critical value), the null hypothesis is rejected in favor of the directional alternative hypothesis. The probability that the observed difference (state the difference between means) would have occurred by chance, if in fact the null hypothesis is true, is less than .05.”