Psychology Statistics

Mrs. Raina Cook

November 19, 2018

Fun with data

(Or how to use the χ2, Mann-Whitney, McNemar, t-tests, and Wilcoxon signed rank tests after one hour! With Dr. Donna Molinek, Savannah

Williams ’19, and Tanya Nair ’19 from Davidson College, Department of Mathematics and Computer Science and Department of Psychology)

1

1

Some statistical tests and when to use them. The basic idea is this: you collect data and depending on the type of data and the question you want to answer or explore, you find a certain number associated with the data. Then you compare that number to a baseline number and draw conclusions at some confidence level.

You must also decide if you want a parametric or non-parametric test.

Parametric tests have more statistical power.

Use a parametric test if your data is continuous and normally distributed.

What kind of data is continuous? If your data is numerical and you could observe any value in your range, then your data is continuous.

Examples: Test scores from 0-100, height, time to complete a task

If your data is not continuous, then you want to use a non-parametric test

Examples: Counts of people, letter grades, “yes” or “no”

You should also use a non-parametric test if your data is not normally distributed.

Normal Not Normal

2

2

Types of data:

1. Categorical data— This is data with categories. Dichotomous is a kind of data under categorical data, which has two categories. For example, asking someone if their favorite ice cream is chocolate, vanilla, or strawberry, is categorical. Asking whether someone likes ice-cream (yes/no) is categorical-dichotomous data, since there are only two possible answers.

2. Ordinal data—This data is ranked. For example, if our data is socio-economic status, we could have participants report their SES as high, medium or low. The responses have a clear ordering but are not continuous.

3. Interval data—The data is numerical with equal intervals and no true zero. For example, temperature can have equal intervals and no true zero. 0oC is not meaningless- it indicates a certain temperature.

4. Ratio data—This data is numerical and similar to interval data, but this time we have a true zero. For example, speed, where 0mph means that there is no movement.

What are we testing?Null hypothesis: no significant difference; any difference we see could be due to random chance

Alternative hypothesis: significant difference; the difference is larger than we would expect from random chance

Say we decide to use a p value of 0.05. This means there is a 5% chance that the statistical value we are observing would occur randomly.

3

3

Testst-tests

Use a t-test if you have continuous, normally distributed data, and you want to compare mean difference. If you have two different groups, then you use an unrelated, or independent, t-test. If you have one group which you measure twice, then you use a related, or paired, t-test.

The following tests are non-parametric (your data does not have to be normally distributed)

Wilcoxon signed rank test

Use this test if you have continuous or ordinal (on an ordered scale) data and you want to see the effect of an experiment on the same group of people. You’ll test to see if the before data is significantly different from the after data.

Mann-Whitney U test

Use this test if you have continuous or ordinal data and you want to compare two data sets (with different groups of people) to see if they are different. For example, if you want to test whether a treatment helps a certain condition, your two data sets come from the control group and a treatment group.

χ2 (chi-square) test for goodness of fit

Use this test if you have categorical data and want to see if there is a difference in the responses of two different groups. For example, you want to see whether students are equally likely to choose a class independent of time or professor.

McNemar test

Use this test if you have categorical data that is dichotomous (only two possible outcomes) and you want to see the effect of an experiment on the same group of people.

4

4

t-test:

1. Repeated measures/ Paired Samples t-test

Use this test if you have continuous, normally distributed data and you want to see the effect of an experiment on the same group of people. For example, 16 employees’ proficiency in using adobe acrobat was measured before and after an adobe acrobat training session. The following data was collected:

Mean difference between scores before and after training session

20

Standard Deviation of the mean difference of the scores

3

An employee’s score went up, on average, 20 points after training.

The method: We use the following test statistic:

m is the sample mean, μ is the expected mean, s is the sample standard deviation, and n is our sample size

We will use μ=0, as we would expect no change as our null hypothesis. Thus,

we have t=20−03/√16

=26.66. For a t-test, we have n-1 degrees of freedom. If we

check our table for t values, we see 2.131 is the critical value for p=0.05 and df=15. This means that t=2.131 is so far away from zero, there is only a 5% chance the scores would change that much on their own. Since 26.66>2.131, we can reject our null hypothesis that there was no difference in an employee’s score before and after training.

5

5

t-test (continued):

2. Independent samples t-test

Use this test if you have continuous, normally distributed data and you want to see the effect of an experiment on two separate groups of people. For example, 22 students in a math class were divided into two equal groups. The 11 students in group 1 studied for 3 hours, and the 11 students in group 2 studied for 6 hours. They then took a math test and their scores were compared. Their mean scores on the test are:

Mean scores of students in group 1

85

Standard Deviation of the scores in group 1

5

Mean scores of students in group 2

65

Standard Deviation of the scores in group 2

5

The method: For an independent samples t-test, we use the statistic:

mA is the mean of the first sample, mB is the mean of the second sample, nA is the size of the first sample, nB is the size of the second sample, and s is a “pooled” standard deviation (there is a formula to combine them, but we’ll let the calculator worry about that). Our degrees of freedom are nA+ nB-2.

For our test, we get t= 85−65

√ 52

11+ 52

11

=9.381. With 11+11-2=20 degrees of freedom, if

we check our table of t values, we can see that t=9.381 > 2.086, so we reject our null hypothesis that there is no difference between the two sample means.

6

6

Wilcoxon signed rank test:

Here we have data that measures some characteristic in the same person before and after a treatment. For example, let's suppose that we have 10 people who we ask to stand on their hands. Then we give them a huge bowl of spaghetti to eat and ask them to stand on their hands again. We record the number of seconds they stand on their hands before and after the spaghetti binge. The question is - does eating a huge bowl of spaghetti affect their ability to stand on their hands?

The method: See Table 4 for computations. First find the difference before - after. Then rank the absolute value of this difference with 1 being the lowest rank (smallest absolute value of the difference). After ranking, keep track of whether the difference was positive or negative.

Then add the ranks of the positive differences and then the ranks of the negative differences. The numbers we need in this statistical test are the number of participants (ties are ignored) and the minimum of the sums of the positive ranks and the negative ranks. See the table 4 for the numbers.

7

7

Mann-Whitney U test:

Use this test if you do two different things to a population. For example, you may give everyone in the class one of two pills that look the same. One pill has no active ingredients and the other is supposed to make you drowsy. Then you give all the students the same test and record their scores. We wish to know if the pills affect test scores. So, the null hypothesis is Ho: the pills make no difference in scores.

The Method: Record the scores and the type of pill for each student. Then list the test scores from highest to lowest and rank each test (ties get an average). Now compute the following quantities.

nA = number of students who took pill AnB= number of students who took pill BRA = sum of the ranks of the scores of students who took pill ARB = sum of the ranks of the scores of students who took pill B

UA =nA nB+nB (nB+1 )

2−R

B

UB=nA nB+nA (n A+1 )

2−R

A

And finally, let U be the smallest of UA and UB.

We use the table with nA= nB= 10 and a =.05 and see that the critical value is 23.Since our value of U = 31, we fail to reject the null hypothesis. That is we cannot say that the pills made a difference.

8

8

x 2 test

Example 1: We test to see if students are equally likely to choose a section of a mathematics course independent of time or professor. Since there are 119 students and 7 sections, we would expect to see 17 students per class if the number in each class was uniformly randomly distributed. Table one shows the observed number in each section.

The method. For each section, compute the quantity Oi is the observed number of students in section i. E is the expected value of the number of people in each section and is the same for each section. Then we

add up all those weighted squares of differences, and get the statistic

The two numbers we need to test the hypothesis that the number of students is equally distributed over the sections are X 2 and df, the degrees of freedom for this situation. The degrees of freedom for this statistic is n-1 where n is the number of categories. We look this up in the table and see that for a =.05 , (a is called the confidence level) the critical value is 12.6. Since our value of X2 = 12.94 so we reject the null hypothesis and conclude that there is a difference in how the students choose a section.

Example 2: We want to see if the type of flower for a snap pea is distributed according to the ratio 9:3:3:1. Since there are 556 plants, this means we would expect to see

916* 556 = 312.75 flowers that are round and yellow,

316 * 556 =104.25 flowers that are wrinkled and yellow,

316 * 556 = 104.25 flowers that are round and green, and

116* 556 = 34.75 flowers that are wrinkled and green.

We form the same statistic as above, except note that E varies over the categories. See the computations in Table 2.

777777777777777

9

9

McNemar test:

Use this test if you have dichotomous data (only two possible outcomes) and you want to see the effect of an experiment on the same group of people. For example, we might ask students if they like the cafeteria food at the beginning of the school year and then again at the end of the school year.

End

No Yes

Beginning No 80 100 180

Yes 10 110 120

90 210 300

The Method: We compute our statistic: χ2=(c−b) ²c+b

Using values b and c from the table, as labeled below.

End

No Yes

Beginning No a b

Yes c d

χ2=(10−100) ²100+10

¿ (90) ²110

¿73.63

The degree of freedom is 1, so we check our table for χ2 and see that we are well above the critical value of 3.84. Thus, the difference is significant.

10

10

Table 2color of peas

observed expected (O - E) A2 / Eround and yellow 315 312.75 0.01618705wrinkled and yellow 101 104.25 0.101318945round and green 108 104.25 0.134892086wrinkled and green 32 34.75 0.217625899

sum 556 556 0.470023981

a = .05, df = 3, critical value is 7.82 so accept the null hypothesis that the type and color is in the proportion 9:3:3:1

11

11

Mann-Whitney

12

12

Since our W=5>4, we do not reject the null hypothesis and find that the difference is not statistically significant.

13

13

14

14

15

15

16

16

17

17

What test should I use? (tables)

Parametric tests (data is normally distributed)

Kind of data Number of groups of participants

Test

Continuous Same group Related / paired t-test

Continuous Two different groups Unrelated / independent t-test

Non-parametric tests (data is not normally distributed)

Kind of data Number of groups of participants

Test

Continuous or ordinal Same group Wilcoxon

Continuous or ordinal Two different groups Mann-Whitney

Categorical Two different groups χ2 (chi-square)

Categorical/ dichotomous Same group McNemar

18

18