observations contextual inquiry

CSCI 4163/6904, summer 2011

Quiz

Multiple choice Answer individually - pass in Then class discussion

11. Questionnaires...

a. are a quick and easy way to gather information

b. are only good for collecting qualitative data

c. are only good for collecting quantitative data

d. require thought and careful planning

2. If you want a high completion rate, you would administer the questionnaire:

a. in person b. via (snail) mail c. via email d. using a web-based form

3. 3. Before sending out a questionnaire, you should make sure that:

a. questions are worded clearly

b. the questionnaire can be completed within the desired amount of time

c. you can analyze the results

d. all of the above

4. Which of the following is not an important aspect of a successful questionnaire?

a. include only closed questions

b. know what statistics you are going to run in advance

c. do a pilot questionnaire

d. include only understandable, clear questions

5. Interviews are:

a) Verbally asking participants questions

b) Hearing their point of view in their own words

c) Both A & B

d) Neither A nor B

6. Which type of interview allows you to probe participants’ responses?A) Structured

B) Unstructured

C) Semi-Structured

D) Unstructured and Semi-Structured

7. Which type of interview allows you to quantitatively compare responses?A) Structured

B) Unstructured

C) Semi-Structured

D) Unstructured and Semi-Structured

8. Active listening is…

A) Making a conscious effort to hear the words a person is saying

B) Making an effort to understand the complete message being sent

C) Paying attention to the other person very carefully

D) All of the above

Exercise

Research topic: Rituals of on-line information seeking behaviour

Questions: What types of information sources are regularly checked? Is it a push or pull paradigm? What is the frequency? What prompts a session to begin? How long does a session usually last? What brings a session to an end (time? Information found? All sources seen?)? Is there multi-tasking?

Observation

Watching people, programs, events, communities, etc.

Used to: Provide information about real-life

situations and circumstances Assess what is happening

Valuable because you cannot rely on participants’ willingness and ability to furnish information

When is observation useful? When you want direct information When you are trying to understand

an ongoing behaviour or process When there is physical evidence,

products, or outcomes that can be readily seen

When other data collection methods seem inappropriate

Observations

Advantages Disadvantages Most direct measure

of behavior Provides direct

information Easy to complete Saves time (?) Can be used in

natural or experimental settings

May require training Observer’s presence

may create artificial situation

Potential for bias Potential to overlook

meaningful aspects Potential for

misinterpretation Difficult to analyze

(If unobtrusive…)

Can see things in their natural context Can see things that may escape

conscious awareness, things that are not seen by others

Can discover things that may have been taken for granted

Can learn about things that people might not be willing to talk about

Low potential for generating observer effects

Major limitations

Potential for bias Observer bias Cultural bias (during observation and

interpretation) Reliability

Ease of categorization Often used in combination with other

methods to provide a more thorough account

Types of observation

Observing what does not happen may be as important as observing what does happen

Planning

Determine who/what to observe Determine what aspects will be observed

(characteristics, attributes, behaviours, etc.)

Determine when/where observations will be made

Develop the observation record sheet Pilot test the observation record sheet Train the observers, practice Collect information, analyze and interpret

Observations need to be credible Observation guide Recording sheet Checklist Field notes Pictures Video Some combination of the above

Ecological validity

Is what you are observing representative of usual behaviours?

Unobtrusive? Task? Setting? Tools?

If unobtrusive…

Can be hard to understand why….

Contextual inquiry

Interviewees are interviewed in their context, when doing their tasks, with as little interference from the interviewer as possible.

Allows probing of “why?” Can be real-time or record

interesting actions for later discussion

“Typical” 4 phased approach Traditional interview

Get an overview, establish trust, start recording Switch to a master-apprentice relationship

Tell them what you want to observe Make sure to establish when ok to interrupt Observe, ask questions

Take notes Balance need to understand with impact of

interruptions Summarization

Go over observations and your understanding with participant

Make sure that you go it right

Other ways of providing context If natural observation not possible,

can ask them to demonstrate specific tasks of interest

Can provide task scenarios and ask them to perform

“Think aloud” aloud protocols

Other ways of getting observational data Logging Screen recording (check out

Camtasia) Trace data

Homework

Advertising Diary exericise for Thurs Assigned reading w/ questions for

Thurs Read McGrath’s Methodology Matters

(submit questions) for Tues