Answer Key for Chapter Nine Practice Problem: Example analysis using the Kruskal-Wallis statistic

May 2015

“Is there a statistically significant difference in NSEs’ perception of personal qualities of NNSE who have a strong L1 influence in their English depending on the NSEs’ amount of interaction with NNSE?”

[Note: NSE = Native speaker of English; NNSE = Non-native speaker of English; L1 = first language]

The general research question guiding the analysis of Rebecca Noreen’s data…

Hmmmm….(I use purple when I’m thinking aloud)

1) I have perception ratings collected from participants who have had a low, medium, or high amount of interaction with NNSEs.

2) The confidence ratings are collected using an instrument that yields ordinal-scale data.

3) Each participant falls into only one ‘amount of interaction’ group so there are different people in each of the comparison groups.

4) I want to know if there’s a statistically significant difference in the perception of the three groups defined by amount of interaction with NNSE.

Seems like the appropriate statistic for the Kruskal-Wallis statistic.

What to do? What statistic can be proposed for analyzing the data for the research question?

Null hypothesis: There is no statistically significant difference in NSEs’ perception of NNSEs’ amount of education depending on the NSEs’ amount of interaction with NNSE.

 Alternative hypothesis: There is a statistically significant difference in NSEs’ perception of NNSE amount of education depending on the NSEs’ amount of interaction with NNSE.

NOTE: Kruskal-Wallis supports only a two-tailed alternative hypothesis. If the null hypothesis is rejected, pairwise post hoc analyses of the 3 groups defined by the independent variable can be done using Wilcoxon Rank Sums Test (comparing Groups 1 and 2, Groups 1 and 3, and Groups 2 and 3).

Step 1: State hypotheses

What would you like to establish as the level of probability for interpreting the findings of this study?

Step 2: Set level of probability

Kruskal Wallis because:

◦ Non-parametric analysis should be used because the data were collected using a tool that yields rankable data

◦ Independent variable has more than 2 levels◦ There are different people in each of the levels of the

independent variable◦ The researcher wants to know whether there is a difference

among the groups.

Step 3: Choose the appropriate statistic

Rebecca collected the data. All we have to do is import the dataset.

Download the dataset from the Companion Website (http://www.routledge.com/cw/turner-9780415819947/s1/datasets/) and save it on your computer as a comma separated values (csv) Excel document.

This R command gives you access to your computer documents.

>becca.data = read.csv(file.choose(), header=T)

Step 4: Collect the data

The assumptions for using Kruskal-Wallis are:

the independent variable has three or more levels

the researcher wants to determine whether there is a difference among the groups defined by the independent variable.

the groups defined by the levels of the independent variable

are comprised of different participants

the dependent variable yields rankable data

Step 5: Verify assumptions are met

kruskal.test (educate ~ interact, data =becca.data)

Note: this commands work tookruskal.test(becca.data$educate~becca.data$interact)

Step 6: Calculate the observed value of the Kruskal-Wallis statistic—I’m going to use R...

Kruskal-Wallis rank sum test

data: educate by interact

Kruskal-Wallis chi-squared = 24.6297, df = 2, p-value = 4.485e-06

Note: 4.485e-06 means .000004485 (That is, move the decimal point 6 places to the left.)

The output looks like this:

The exact level of probability for the observed statistic is p-value = .000004485

when exact level of p > alpha accept null hypothesis

when exact level of p < alpha reject null hypothesis & accept alternative hypothesis

Steps 7 & 8: To interpret, we could use a chart of critical values for chi-square, but let’s use the exact probability instead.

I can be 95% there is a statistically significant difference in NSEs’ perception of NNSE amount of education depending on the NSEs’ amount of interaction with NNSE.

Step 9: Make probability statement

Statistical logic led us to accept the null hypothesis. I can be 95% there is a statistically significant difference in NSEs’ perception of NNSE amount of education depending on the NSEs’ amount of interaction with NNSE (Kruskal-Wallis chi-squared = 24.6297, df = 2, p-value = .000004485).

Step 10: Interpret meaningfulness--revisit the research question and determine effect size

Calculate each of the 3 pair-wise comparisons using the Wilcoxon Rank Sum statistic (flipping the order of the two levels to determined which Wilcoxon to report):◦Low interaction group to high interaction group◦Low interaction group to mid interaction group◦Mid interaction group to high interaction group

With R, we can extract the z-score using R and calculate effect size for each comparison (since the n-size is sufficiently large for conversion of the Wilcoxon outcomes to z scores).

Calculating effect size

low =subset (becca.data, becca.data$interact=="1")

mid =subset (becca.data, becca.data$interact=="2")

high =subset (becca.data, becca.data$interact=="3")

First, create 3 separate datasets, once for each interaction group....

wilcox.test(low$educate,mid$educate, exact = F)W = 370*, p-value = 0.09734

(Then flip the two groups to determine which value of W to report )

wilcox.test(mid$educate,low$educate, exact = F)W = 591, p-value = 0.09734

*Report W for comparison of low to mid as W = 370.

Compare low to mid

wilcox.test(low$educate,high$educate, exact = F)

W = 164*, p-value = 4.179e-06 [.000004179]

(Then flip the two groups to determine which value of W to report )

wilcox.test(high$educate,low$educate, exact = F)W = 797, p-value = 4.179e-06 [.000004179]

*Report W for comparison of low to high as W = 164

Compare low to high

wilcox.test(mid$educate,high$educate, exact = F)

W = 237*, p-value = 0.000402

(Then flip the two groups to determine which value of W to report )

wilcox.test(high$educate,mid$educate, exact = F)

W = 724, p-value = 0.000402

*Report W for comparison of mid to high as W = 237

Compare mid to high

study.model= wilcox.test(low$educate,mid$educate, exact = F)

z=qnorm(study.model$p.value/2)

effect.size = abs(z)/sqrt(62)

effect size = 0.2105489

Use these commands to calculate effect size for the low to mid comparison...

study.model =wilcox.test(low$educate,high$educate, exact = F)

z = qnorm(study.model$p.value/2)

effect.size = abs(z)/sqrt(62)

effect size = 0.5844904

Use these commands to calculate effect size for low to high comparison...

study.model = wilcox.test(mid$educate,high$educate, exact = F)

z=qnorm(study.model$p.value/2)

effect.size = abs(z)/(62)

effect size = 0.4494233

Use these commands to calculate effect size for mid to high comparison...

The research shows a significant difference in NSEs’ perception of NNSE amount of education depending on the NSEs’ amount of interaction with NNSE (Kruskal-Wallis chi-squared = 24.6297, df = 2, p-value = .000004485).

The Wilcoxon Rank Sum statistic was used to determine that the comparison of the low and high interaction group (W = 164, p = .000004179) and the mid and high interaction group (W = 237*, p-value = 0.000402) are statistically significant. The effect sizes for these comparisons (.58 and .44, respectively) indicate that the differences are moderate to very strong. While the difference between the low and mid interaction group (W =370, p. = 0.09734) does not show a significant difference, the effect size is moderate (.21).

A complete report of the statistical outcomes...