STRUCTURED INTERVIEWS AND INSTRUMENT DESIGN PART I

Lecture 7

“Continuum” of Interview Situations

2

Unstructured Structured

Open-ended interviews

Interview Guides, still open-ended

Identical Instrument (questions, etc) given to all participants

3

What is a Survey?

To survey: “act of looking or seeing, observing”

Research Surveys Qualitative interviews, focus groups Specific, systematic, quantitative data-collection

instruments

4

5

Example Surveys

US Census

General Social Survey

Online Surveys

Structured Interview Formats6

Face-to-face, self-administered (email, mail, etc), phone surveys

Typically either cross-sectional or longitudinal

Often a balance between (1) standardization (2) ability to reach sample (3) available resources (4) type of questions you are asking

Structured Interviewing7

Standardization of Wording and Format in an Interview Instrument

8

Must have a logic for each question Must have a logic for the question responses (if provided) Must have a logic for the sequencing of questions

Must have clear wording

Must have clear formatting

Must take into account the sample population that will actually take/use the survey instrument Culture, language, interpretive ambiguity

Question Logic9

Avoid asking unnecessary questions Example: Survey on computer usage

Socio-demographic questions (age, gender)? Risk behavior questions (drug use, etc)?

Closed or Open-ended?

Closed versus Open Questions10

(Q1a) What do you spend most of your time doing on the Internet?

(Q1b) Which of the following things do you do most of the time on the Internet?

(a) Email(b) Browse the web(c) Play games(d) Other

Question Logic11

Avoid redundant questions unless you have a clear reason Exception: Sometimes it is a good idea to ask

two questions that tap same concept (first as a scale, then as a categorical decision)

Example (ranks): First ask respondent to rate 1-10 how much they agree

with several statements. Next, ask respondents to rank the statements by how

much they agree with them (i.e., 5 statements, rank them 1-5).

Question Response Logic: Scales versus Categories

12

As a basic rule, metric scales with more range are better than binary or categorical responses (all other things being equal).

Example: Age Are you 18 or older (yes/no)? How old are you (in years)? _________

With general scales and Likert-scales, consider having no “middle” category (neutral, no opinion).

13

Question Response Logic: Categorical Responses

Responses must be mutually exclusive Example (bad): Where do you live?

Berkeley, San Fran, Bay Area, Other

Responses must be exhaustive Example (bad): What kind of computer do you

have? PC, Mac

Use ‘Don’t Know’, ‘Other’, ‘Not Applicable’ when absolutely necessary

14

Question Sequence

Static order for questions needs to have some rationale/logic Grouping similar items together Scattering similar items throughout survey

Personal demographic questions work best at end of survey (response rate and completion)

Randomization for all respondents

15

Clear Wording / Leading Questions

Clear Wording Example:

“What ISP do you use?” “If you have Internet service at home, what company or

service provider do you use for Internet access?”

Leading Questions Example:

“Don’t you think we should support our troops in Iraq?” “How strongly do you agree or disagree with the

following question: ‘We should support our troops in Iraq’”

16

Clear Formatting, Logic

Not all questions apply to everyone Example:

“How much do you spend on gas heat each month?”

Branching is a possible solution Example:

“Do you have gas heat? If yes, go to next question. If not, skip to question #3.

Condense when possible to avoid unnecessary branching. Example:

“How much do you spend on gas heat each month? (write 0 if you do not have gas heat)

17

Know your sample population

Regional language and terminology

Cultural differences

How you conduct survey can influence your valid sample Door to door? Registered telephone directory? Internet-based survey?

18

Replication and Using Existing Survey Instruments

ALWAYS a good idea to find other surveys that are used in your area of interest.

Especially with large, funded surveys the questions may have been tested for reliability.

Allows for comparisons between different samples if the question wording is the same.

If a question or set of questions is accepted as a good operationalization of the concept you are interested in, you don’t want to reinvent it unless you really intend to argue that your measure is more appropriate.