11

Instructor: Vincent Duffy, Ph.D.Instructor: Vincent Duffy, Ph.D.

Associate Professor of IEAssociate Professor of IE

Lab 1 – Part 1 Review - Lifelong Learning Lab 1 – Part 1 Review - Lifelong Learning

Fri. Feb. 2, 2007Fri. Feb. 2, 2007

IE 486 Work Analysis & Design IIIE 486 Work Analysis & Design II

22

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis 1. research ideas/question -what is the purpose? 2. what background support? - literature review 3. theoretical basis for analyzing question/hypothesis? 4. applicability-practical contribution? 5. theoretical contribution? 6. appropriate methodologies for carrying out study?

(determining variables, data collection, method used to test hypotheses, validity of measures and reliability)

7. appropriate statistical analyses and assumptions? 8. presentation of results: what do they really mean? 9. conclusions drawn: are they reasonable? 10. future work/research directions: any possibilities?

33

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

1. research ideas/question -what is the purpose?

Determine if computers are social actors across cultures

Possible because previous studies showed…humans interact with computers the way they behave with humans - U.S.

Questions – is this generalizable across cultures…maybe, sometimes…(conditional)

Differences in culture; individualistic vs. collectivist behavior; reciprocity

44

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

2. what background support? - literature review

Tested repeatedly in U.S. – social-psychology literature, some HCI studies demonstrated resuts that contributed to the Computers as Social Actors paradigm

Nass, 1996; 1999….; colleagues also…including Reeves (1996), Moon (1999), Fogg (1997)…

Reciprocity? (supported?)…literature back to 1960.

55

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

3. theoretical basis for analyzing question/hypothesis?

Reciprocity? (supported?)…literature back to 1960.Computers as Social Actors Paradigm

Method for identifying new hypotheses… Go to social-psychology literature Re-write rule for human-human interaction to see If it

applies to HCI Design an experiment to test Used on 100 social rules in the past

66

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

4. applicability-practical contribution?

Computer user interface design… More specifically…products designed for multiple

markets/countries (marketing and branding and perception that

influences behavior in different culture can impact, ultimately, market share).

Impacting how designers consider potential users (more broad perspective needed for ‘who is the user’.)

People informed in the scientific method can contribute to future product development…

77

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

5. theoretical contribution? Could broaden scope of where existing HCI studies can

be applied Also could highlight limitations in relation to past

expected generalizability Could challenge us to re-think how broadly we should

(or shouldn’t generalize) how to apply the results See discussion… especially paragraph 2… This research extends the Computers as Social Actors

in two ways… Verify previous studies, or extend generalizability of

previous findings…(relating to understanding of human-human interaction differences across culture)

88

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

6. appropriate methodologies for carrying out study? (determining variables, data collection, method used to test hypotheses, validity of measures and reliability)

2x2 between subjects test… (vs. within subjects test). Variables – reliable? Valid? Measures? (is that the

way?)

99

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

7. appropriate statistical analyses and assumptions? Confidence to ignore uncontrolled factors??

Results are clear (differences shown in graph) results of F test are significant… meaning…p value significant and below some alpha value

(likely .05 since many are showing <.01 and words describing the relationships as significant.

Threshold is the level at which …alpha level is likelihood of (type I error) making an error – 5% chance that the reported (significant) relationship is reported incorrectly/in error.

2 graphs…one for each experiment Values of hypothesis testing…

1010

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

8. presentation of results: what do they really mean?T test, F-test. Post hoc test…why? To interpret the ‘signficant differences’

reported from the F tests.

Additional items were mentioned earlier in the ‘elevator’ description.

1111

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

9. conclusions drawn: are they reasonable?

1212

10 ways to look at a research paper 10 ways to look at a research paper systematically for critical analysissystematically for critical analysis

10. future work/research directions: any possibilities?