tauchi – tampere unit for computer-human interaction experimental research in it analysis –...

TAUCHI – Tampere Unit for Computer-Human Interaction

Experimental Research in IT

Analysis – practical part

Hanna Venesvirta


Experimental task: select an object as fast as possible.

Depend variable: selection time.

One independent variable (diameter of an object) with three levels (diameter: 25, 30, and 40 mm).

All the participants made the same task: within subject design.

one-way within subject ANOVA.

Example analysis etc.


participant no.Diameter: 25 mm Diameter: 30 mm Diameter: 40 mm1 2490,625 1240,166667 1154,52 6462 1852,4 26033 1006,571429 738,25 747,254 1163,833333 860,25 806,14285715 1890,233396 1918,6 1225,6666676 3399,714286 1237,8 1386,1428577 1091,5 873,875 8748 2180,4 1634,571429 1879,7142869 1614,285714 949,125 970,875

10 1663,125 1441,857143 178211 1598,666667 1066,142857 1277,42857112 1082,375 1159,75 1253,87513 1424,5 1141,571429 1328,62514 1211,5 2521,428571 119715 2542,333333 1703,166667 1128,37516 1471,6 1861,2 134917 1462,571429 1089,625 101718 1048 1073,428571 1036,519 1289 1857,2 1175,33333320 1711,833333 899,25 916,8571429

Mean 1890,233396 1355,982917 1255,464286S.E.M. 276,2519175 105,3038123 95,3892171

0

500

1000

1500

2000

2500

3000

3500

Diameter: 25 mm Diameter: 30 mm Diameter: 40 mmTi

me

(ms)

Mean selection times with S.E.M.s

Difference in the means!

Is it significant?


What is needed for a graph?


Error bars to a graph?

(Applies to MS Office 2007 & 2010)

- Make the graph- Select the data series from the graph -> ”chart tools” is available- Select ”layout” – ”error bars” – ”more error bars options”- Here we use only ”positive” error bars – select ”plus”- ”Error amount” == ”custom” – select ”specify value” – select from the sheet the values you have calculated for the deviation (e.g., S.E.M.s) for ”positive error value”

(MS Office 2013? E.g. http://www.officetooltips.com/excel/tips/adding_error_bars.html )


Data to SPSS?

Three ways (at least):

1) Input by hand Might be painful and cause errors

2) Copy-and-paste (from, e.g., Excel)Paste variable names to ”variable view” and data to ”data view” Note: SPSS is picky with variable names – avoid at least spaces and umlauts


Data to SPSS?

3) Open data from, e.g., Excel

Open – data – browse to data location – select the file type you are using

Note: for the SPSS analysis you use only the data matrix (so no means or deviations etc. are used here)

participant no.Diameter: 25 mm Diameter: 30 mm Diameter: 40 mm1 2490,625 1240,166667 1154,52 6462 1852,4 26033 1006,571429 738,25 747,254 1163,833333 860,25 806,14285715 1890,233396 1918,6 1225,6666676 3399,714286 1237,8 1386,1428577 1091,5 873,875 8748 2180,4 1634,571429 1879,7142869 1614,285714 949,125 970,875

10 1663,125 1441,857143 178211 1598,666667 1066,142857 1277,42857112 1082,375 1159,75 1253,87513 1424,5 1141,571429 1328,62514 1211,5 2521,428571 119715 2542,333333 1703,166667 1128,37516 1471,6 1861,2 134917 1462,571429 1089,625 101718 1048 1073,428571 1036,519 1289 1857,2 1175,33333320 1711,833333 899,25 916,8571429


Parametric tests


From the output, find table ”tests of within-subjects effects”


- So there is a difference, but where?-> pairwise comparisons

Comparing 25 mm to 30 mm, 25 mm to 40 mm, and 30 mm to 40 mm -> 3 comparisons

Two ways: 1) via the GLM menu (showed first)

- calculates automatically the Bonferroni correction but does not tell the t-values

2) via pairwise t-tests-menu (2nd)- gives t-values - remember to calculate the B-correction by hand (α/no. of comparisons)


size

From the output, find ”pairwise comparisons” under ”estimated mariginal

means”:1=25 mm2=30 mm3=40 mm

Check in which order your columns were and you will find out where the difference is.


Paired Samples Test

Paired Differences t df Sig. (2-tailed)

Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the

Difference

Lower Upper

Pair 1Diameter25mm -

Diameter30mm

534,25047932330

8200

1188,6194789491

00000

265,78339543105

6930

-

22,040560576190

714

1090,5415192228

07000

2,010 19 ,059


Diameter40mm

634,76911027568

9700

942,43528288740

0500

210,73493569304

7280

193,69582076596

8620

1075,8423997854

10600

3,012 19 ,007


Diameter40mm

100,51863095238

1600

456,37710097444

7500

102,04902211531

5080

-

113,07242706381

1220

314,10968896857

4230

,985 19 ,337

From the output, find ”paired samples test”


Reporting parametric ANOVA

One way within subjects ANOVA with object diameter as a factor revealedstatistically significant effect of the object diameter, F(2, 38) = 5.57, p < 0.01.


Reporting pairwise comparisons (1)

Post hoc pairwise comparisons showed that the selection time was significantly faster when the object diameter was 40 mm than when the diameter was 25 mm, MD = 634.77, p < 0.05. Other pairwise comparisons were not statistically significant.

”MD” stands for ”Mean Difference”

1=25 mm2=30 mm3=40 mm

Check in which order your columns were and you will find out where the difference is.


Paired Samples Test

Paired Differences t df Sig. (2-tailed)

Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the

Difference

Lower Upper


Diameter30mm

534,25047932330

8200

1188,6194789491

00000

265,78339543105

6930

-

22,040560576190

714

1090,5415192228

07000

2,010 19 ,059


Diameter40mm

634,76911027568

9700

942,43528288740

0500

210,73493569304

7280

193,69582076596

8620

1075,8423997854

10600

3,012 19 ,007


Diameter40mm

100,51863095238

1600

456,37710097444

7500

102,04902211531

5080

-

113,07242706381

1220

314,10968896857

4230

,985 19 ,337

Post hoc pairwise comparisons showed that the selection time was significantlyfaster when the object diameter was 40 mm than when the diameter was 25 mm,t(19) = 3.012, p < 0.05. Other pairwise comparisons were notstatistically significant.

Reporting pairwise comparisons (2)


Non-parametric one way within subjects analysis of variance:

Friedman test


NPar Tests

Descriptive Statistics

N Mean Std. Deviation Minimum Maximum

Diameter25mm 20 1890,2334 1235,43613 1006,57 6462,00

Diameter30mm 20 1355,9829 470,93297 738,25 2521,43

Diameter40mm 20 1255,4643 426,59355 747,25 2603,00

Friedman TestRanks

Mean Rank

Diameter25mm 2,60

Diameter30mm 1,75

Diameter40mm 1,65

Test Statisticsa

N 20

Chi-Square 10,900

df 2

Asymp. Sig. ,004

a. Friedman Test

From the output, find ”test statistics”


Post hocs for Friedman:

Wilcoxon (Repeated Measures) Signed-Rank test


Wilcoxon Signed Ranks Test

Ranks

N Mean Rank Sum of Ranks

Diameter30mm -

Diameter25mm

Negative Ranks 14a 11,64 163,00

Positive Ranks 6b 7,83 47,00

Ties 0c

Total 20

Diameter40mm -

Diameter25mm

Negative Ranks 18d 11,00 198,00

Positive Ranks 2e 6,00 12,00

Ties 0f

Total 20

Diameter40mm -

Diameter30mm

Negative Ranks 9g 12,44 112,00

Positive Ranks 11h 8,91 98,00

Ties 0i

Total 20

a. Diameter30mm < Diameter25mm

b. Diameter30mm > Diameter25mm

c. Diameter30mm = Diameter25mm

d. Diameter40mm < Diameter25mm

e. Diameter40mm > Diameter25mm

f. Diameter40mm = Diameter25mm

g. Diameter40mm < Diameter30mm

h. Diameter40mm > Diameter30mm

i. Diameter40mm = Diameter30mm

Test Statisticsa

Diameter30mm -

Diameter25mm

Diameter40mm -

Diameter25mm

Diameter40mm -

Diameter30mm

Z -2,165b -3,472b -,261b

Asymp. Sig. (2-tailed) ,030 ,001 ,794

a. Wilcoxon Signed Ranks Test

b. Based on positive ranks.

From the output, find ”test statistics”


Test Statisticsa

N 20

Chi-Square 10,900

df 2

Asymp. Sig. ,004

a. Friedman Test

Reporting Friedman’s test

Friedman's test showed that there was statistically significant effect of object diameter, Χ²(2) = 10.90, p < 0.01


Reporting Wilcoxon test

Post hoc pairwise comparisons with Wilcoxon signed-rank tests showed that the selection time was significantly faster when the object diameter was 40 mmthan when the diameter was 25 mm, Z = -3.47, p < 0.01. Other pairwise comparisons were not statistically significant.

Test Statisticsa

Diameter30mm - Diameter25mm



Z -2,165b -3,472b -,261b

Asymp. Sig. (2-tailed) ,030 ,001 ,794

a. Wilcoxon Signed Ranks Test

b. Based on positive ranks.

Note: doubleclick the table (twice) and you will see more accurate p-value (on this case p = 0.000517 so it is significant in 0.01 level as 0.01/3=0.0033..).

tauchi – tampere unit for computer-human interaction experimental research in it analysis –...

Documents