AVI/Psych 358/IE 340: Human Factors

Data GatheringOctober 6, 2008

2

Overview (1/2)

Interviews

Ethnography

3

Interviews (1/2)

1. Unstructured - are not directed by a script, are more conversational. Rich but not replicable.

- Example: ‘tell me about yourself’

2. Structured - are tightly scripted (formalized, limited set of questions). Replicable but may lack richness.

- Example: questionnaires, surveys

3. Semi-structured - guided by a script but interesting issues can be explored in more depth. Allows new questions to be brought into interview as a result of what interviewee says. Can provide a good balance between richness and replicability.

4

Interviews (2/2)

• Must avoid:− Leading questions that make assumptions – these questions

suggest a particular answer, imply that a certain kind of answer is more/less correct−Ex: ‘What do you enjoy about this product?’ leading

vs.

‘How do you feel about this product’ not leading

− Unconscious biases− Interviewer biases− Interviewee biases

• Structured interviews tend to avoid these issues, but still must be aware of them.

5

Interview questions (1/2)

• Two types:1. Closed questions: direct and focused; have a predetermined

answer format - yes/no answers

- short and factual answers

2. Open questions: do not have a predetermined format

- whatever the interviewee wants to say in response

• Can use both types of questions within the same interview

• Avoid: leading questions, unconscious biases

6

Closed questions

• Easier to analyze Must avoid:

− Long questions− Compound sentences - split them into two− Jargon and language that the interviewee may not understand− Unconscious biases – e.g., gender stereotypes

7

Running the interview (1/2)

• Introduction – introduce yourself, explain the goals of the interview, reassure about the ethical issues, ask to record, present any informed consent form.

• Warm-up – make first questions easy and non-threatening.

8

Running the interview (2/2)

• Main body – present questions in a logical order• A cool-off period – include a few easy questions to defuse

tension at the end• Closure – thank interviewee, signal the end,

e.g., switch recorder off.

9

Enriching the interview process

• Props - devices for prompting interviewee (e.g., a

prototype, hypothetical scenario)

Focus groups

10

Ethnography (1/4)

Ethnography is a philosophy with a set of techniques that include participant observation and interviews

Ethnographers immerse themselves in the culture that they study Must fully immerse themselves – otherwise their

presence will likely cause people to act differently than they normally do (negates the point of doing an ethnographic study)

11

Ethnography (2/4)

A researcher’s degree of participation can vary along a scale from ‘outside’ to ‘inside’

Analyzing video and data logs - can be time-consuming Collections of comments, incidents, and artifacts are

made

12

Ethnography (3/4)

• Co-operation of people being observed is required• Informants are useful• Data analysis is continuous

13

Ethnography (4/4)

• Questions get refined as understanding grows

• Reports usually contain examples and episodes

14

Choosing and combining techniques

• Technique chosen based upon:– The focus of the study– The participants involved– The nature of the technique– The resources available

15

Summary (1/2)

• Three main data gathering methods: – Interviews– Questionnaires– Observation

• Four key issues of data gathering– Goals– Triangulation– Participant relationship– Pilot

16

Summary (2/2)

Breakdown of data gathering methods/techniques:• Interviews may be:

– Structured– Unstructured– Semi-structured

• Questionnaires may be on paper, online or telephone• Observation may be:

– direct OR indirect– in the field OR in controlled setting

• Techniques can be combined depending upon:– Study focus– Participants – Nature of technique– Available resources

17

Review

Data recording approaches:

• Hand-written notes• User diaries• System logs• Video• Audio• Photographs

Data collection techniques:

• Think-aloud• Online questionnaires• Semi-structured interviews• Structured interviews• Focus groups• Ethnography

18

Interview in-class activity (1/2)

Organize yourselves into dyads Interviewer Interviewee

Data collection technique: Hand-written notes based on semi-structured interview questions

19

Interview in-class activity (2/2)

1. How do you use Google?

2. How can search using Google be improved?