Questionnaires

AD700 College of Advancing Studies

20 October, 2004 Brendan Rapple

Use Questionnaires or Not?

• Perhaps “better” responses by phone -- but phone interviewing is very time-consuming.

• On other hand, people tend to be more truthful with anonymous questionnaires

• Generally cheaper than one-on-one interviews

• Perhaps a mix of questionnaire and interviewing?

Mailed Questionnaires

• Explain purpose of questionnaire very clearly and precisely.

• Explain any potential use the results will have for the respondents

themselves.

• Stress voluntary nature - respondents are doing a favor to the

researcher.

• Use letter-head -- official.

• Confidentiality is clearly promised.

• It has a clear, physically separate expression of thanks.

Whatever Format . . . . . .

• Important to specify a deadline for response.

• Follow-up letter (e-mail) often advisable

• Always essential for PILOT STUDY or PRE-TEST

Questions Should Be

– Woven together

– Flow smoothly

– Avoid confusion

A good questionnaire forms an integrated whole.

Different Understandings

• Everyone should see/understand the exact same

question – no ambiguity.

• But people from different backgrounds, with diverse

frames of reference, may have different perceptions of

the same question.

Avoid Ambiguity

• Do you favor governmental involvement in health care?

• “What is your income?” __________________ do you mean:

weeklymonthlyannualpretaxafter taxfrom salary or from all sources

Another Example:

(A blouse manufacturer wants to ascertain what type of sleeves teenage females prefer in their blouses)

1. Do you like short sleeve blouses?YES__ NO__

2. Do you like long sleeve blouses?YES__ NO__

3. Do you like sleeveless blouses?YES__ NO__

Another Example

• “Do you jog regularly?”

Problem, of course, lies with REGULARLY

•

Avoid jargon, slang, abbreviations

• Plumbers talk about “snakes”

• Psychologists about “oedipus complex”

• Lawyers about “mens rea”

“NATO” usually means North Atlantic Treaty Organization

But some respondents might take it to mean:

--National Auto Tourist Organization

--Native Alaskan Trade Orbit

--North African Tea Office

Types of Questions

• Open-Ended Questions:

– What is your age?

– What is the total turnover in your company?

– Which of the four seasons do you prefer?

– How would you spend a a $1,000,000 lottery win?

Disadvantages of Open-ended Questions:

• Variation in answers make coding/scoring difficult

Advantages of Open-ended Questions

• Do not impose researcher's opinion on respondent

• Can lead to a very precise answer

Forced-Choice Questions

• How many books do you read each year? Please check as appropriate below:

___ none ___ 1 to 5___ 6 to 10___ 11 to 20___ more than 20

• What do you recall about the frequency of snowfall in Boston during the winter of 1994?

___ It snowed almost every day___ It snowed about once a week___ It snowed about once every two weeks

___ It snowed about once a month

Importance of Wording• Take a very simple question:

How many drinks do you have each day? (Check one of the following)

____5 or more

____4____3____2____1____none

Better to Ask

• Are your daily drinking habits reasonably consistent --i.e. do you take about the same number of alcoholic drinks each day?

_____YES

_____NO (if you mark "NO," skip the following question).

Checking Respondents' Consistency

Q. 4. Check one of the following:_x_ I believe that manufacturing should be

increasingly computerized, even if layoffs ensue.

___ Preserving the jobs of workers is more important than computerizing manufacturing.

Later on, the Researcher Might Ask:

Q. 30. Check one of the following:___ I support G.M.'s increasing use of robotics,

despite the resulting massive layoffs.

_x_ The government should force G.M. to curtail computerization to ensure a reduction in

layoffs.

Avoid Emotional Language

• “What do you think about a policy to pay murderous terrorists who threaten to steal the freedoms of peace-loving people?”

Problematic Words: murderous

freedomsstealpeace

What is your present marital status?

1 never married 2 married 3 divorced 4 separated 5 widowed

Arrangement Is Important

A Better Arrangement Would Be:

• What is your present marital status? (circle number)

1. never married2. married3. divorced4. separated5. widowed

Danger of Overlapping

• Check How Many Children are in Your Family:

a. 0-1 children

b. 1-2 children

c. 2 or more children

Avoid Prestige Bias

• “Most doctors say that cigarette smoke causes lung disease for those near a smoker”. Do you agree?”

• “Do you support the president’s policy regarding Zanozui?”

Avoid Double-Barreled Questions:

• "Does your company have pension and health insurance benefits?"

Avoid loaded questions:

• "Should the mayor spend even more tax money trying to keep the streets in top shape?"

• "Should the mayor fix the pot-holed and dangerous streets in our city?"

Ranking Answers (Likert Scale Format):

• Women should automatically receive three months maternity leave in your company

strongly agree partly agree disagree strongly agree disagree

___ ___ ___ ___ ___

How often do you attend church?

Never Yearly Monthly Weekly Daily

____ ___ ___ ___ ___

Visual Analog Scales BIPOLAR scales

I SEE MYSELF AS:

Extremely Extremely interested in uninterested

politics ______:______:______:______:______ in politics