rmk chapter 4: survey research l types of surveys (q & i) types of surveys (q & i) types of...

RMK Chapter 4: Survey ResearchRMK Chapter 4: Survey Research

Types of Surveys (Q & I) Types of Questions Decisions about Question Content Decisions about Response Format Decisions about Question Wording Decisions about Question Placement Conducting Interviews Advantages / Disadvantages of Methods

Types of SurveysTypes of Surveys

Questionnaires and InterviewQuestionnaires and Interviewss

QuestionnairesQuestionnaires

Group administrationGroup administrationGroup administrationGroup administration

Mail administrationMail administrationMail administrationMail administration

Household drop offHousehold drop offHousehold drop offHousehold drop off

Web-basedWeb-basedWeb-basedWeb-based

InterviewsInterviews

PersonalPersonalPersonalPersonal

TelephoneTelephoneTelephoneTelephone

Types of Types of QuestionsQuestions

Type of QuestionsType of Questions

UnstructuredUnstructured» Open ended / short comment box / transcript Open ended / short comment box / transcript

of interview / narrativeof interview / narrative

StructuredStructured» Details in following slidesDetails in following slides

Structured QuestionsStructured Questions

DichotomousDichotomousDichotomousDichotomous

____Yes____Yes

____No____No

MaleMale FemaleFemale

NominalNominalNominalNominal

Occupational Class:Occupational Class:

1 = Truck driver1 = Truck driver2 = Lawyer2 = Lawyer3 = etc.3 = etc.

OrdinalOrdinalOrdinalOrdinal

Rank the candidates in order of preference fromRank the candidates in order of preference frombest to worst...best to worst...

___ Bob Dole___ Bob Dole___ Bill Clinton___ Bill Clinton___ Newt Gingrich___ Newt Gingrich___ Al Gore___ Al Gore

IntervalIntervalIntervalInterval

Likert Response ScaleLikert Response Scale

11 22 33 44 55stronglystrongly disagreedisagree neutralneutral agreeagree stronglystronglydisagreedisagree agreeagree

Semantic DifferentialSemantic Differential

veryvery some-some- neitherneither some-some- veryverymuchmuch whatwhat whatwhat muchmuch

interestinginteresting boringboring

simplesimple complexcomplex

uncaringuncaring caringcaring

usefuluseful uselessuseless

etc.etc.

Guttman (Cumulative) ScaleGuttman (Cumulative) Scale

___Are you willing to permit immigrants to live in your country?___Are you willing to permit immigrants to live in your country?

___Are you willing to permit immigrants to live in your community?___Are you willing to permit immigrants to live in your community?

___Are you willing to permit immigrants to live in your neighborhood?___Are you willing to permit immigrants to live in your neighborhood?

___Would you be willing to have an immigrant live next door to you?___Would you be willing to have an immigrant live next door to you?

___Would you let your child marry an immigrant?___Would you let your child marry an immigrant?

Filter or Contingency QuestionsFilter or Contingency Questions

Have you ever smoked marijuana?Have you ever smoked marijuana?

yesyesnono

Have you ever smoked marijuana?Have you ever smoked marijuana?

yesyesnono

If If yesyes, about how many times have , about how many times have you smoked marijuana?you smoked marijuana?

onceonce2 to 5 times2 to 5 times6 to 10 times6 to 10 times11 to 20 times11 to 20 timesmore than 20 timesmore than 20 times

Try to Try to avoidavoid having more than three having more than three levels (two jumps) for any question.levels (two jumps) for any question.

If only two levels, use If only two levels, use graphicgraphic to jump to jump (for example, arrow and box).(for example, arrow and box).

If all responses can’t fit on the page, If all responses can’t fit on the page, then jump to a then jump to a new pagenew page..

Electronic advantages (web pages)Electronic advantages (web pages)– Can be programs to move forward in a Can be programs to move forward in a

specific way based on response.specific way based on response.

Decisions About Decisions About Question ContentQuestion Content

Decisions About Question ContentDecisions About Question Content

ExampleExample– Do you Do you needneed the age of each child or just the age of each child or just

the number of children under 16?the number of children under 16? ExampleExample

– Do you Do you needneed to ask income or can you to ask income or can you estimate?estimate?

Is the question necessary / useful?Is the question necessary / useful?Is the question necessary / useful?Is the question necessary / useful?

Examples: Bad QuestionExamples: Bad Question– What are your feelings towards What are your feelings towards African-African-

Americans and Hispanic-AmericansAmericans and Hispanic-Americans??– What do you think of proposed changes in What do you think of proposed changes in

benefits and hoursbenefits and hours??

Are several questions needed?Are several questions needed?Are several questions needed?Are several questions needed?

Don’t combine two issues in one...Don’t combine two issues in one...

If you ask about earnings, might not If you ask about earnings, might not mention all incomemention all income

If ask if they’re in favor of public TV, ask If ask if they’re in favor of public TV, ask for whom (might not mention it’s for whom (might not mention it’s because of because of Sesame StreetSesame Street for the kids) for the kids)

Did you get all the info desired...Did you get all the info desired...

If you ask research participants about If you ask research participants about attitudes towards a specific religion, can attitudes towards a specific religion, can you interpret the results without finding you interpret the results without finding out about their attitudes towards out about their attitudes towards religion?religion?

Do you need more context to interpret answer...Do you need more context to interpret answer...

If they say they support public TV …If they say they support public TV …– do they watch it?do they watch it?– would they be willing to have their tax would they be willing to have their tax

dollars spent on it?dollars spent on it?

Need more info to determine the intensity...Need more info to determine the intensity...

ExamplesExamples– ‘‘Do you think Dean Rusk acted correctly in Do you think Dean Rusk acted correctly in

the Bay of Pigs crisis?’ -- can’t say if they the Bay of Pigs crisis?’ -- can’t say if they don’t know who he is.don’t know who he is.

– In TV surveys, ‘regular’ watchers can be In TV surveys, ‘regular’ watchers can be asked about program content, ‘never watch’ asked about program content, ‘never watch’ should not be asked.should not be asked.

Do respondents have the needed info?Do respondents have the needed info?Do respondents have the needed info?Do respondents have the needed info?

How well did you like the book? How well did you like the book?

Versus...Versus... Did you recommend the book to others?Did you recommend the book to others? Did you look for other books by that Did you look for other books by that

author?author?

Does question need to be more specific?Does question need to be more specific?Does question need to be more specific?Does question need to be more specific?

For instance, seasonal specificity.For instance, seasonal specificity.– If you ask what they usually watch and it’s If you ask what they usually watch and it’s

summer, you’ll get a different answer than in summer, you’ll get a different answer than in the winter.the winter.

Is question sufficiently general?Is question sufficiently general?Is question sufficiently general?Is question sufficiently general?

Ask about the benefits of a tax cut Ask about the benefits of a tax cut without asking about the disadvantageswithout asking about the disadvantages

Ask about the disadvantages of Ask about the disadvantages of eliminating welfare without asking about eliminating welfare without asking about the benefitsthe benefits

Is question biased or loaded?Is question biased or loaded?Is question biased or loaded?Is question biased or loaded?

To find out age or income, give brackets.To find out age or income, give brackets.– [18-22], [23-27], etc.[18-22], [23-27], etc.

To find out about charitable To find out about charitable contributions, ask how much “people you contributions, ask how much “people you know” typically give.know” typically give.

To find out what magazines they read, To find out what magazines they read, offer to buy used ones.offer to buy used ones.

Will respondent answer truthfully?Will respondent answer truthfully?Will respondent answer truthfully?Will respondent answer truthfully?

Decisions About the Decisions About the Response FormatResponse Format

Should the Response Format be...Should the Response Format be... Single option vs. multi-optionSingle option vs. multi-option StructuredStructured

Dichotomous (two possibilities)Dichotomous (two possibilities) Multiple choiceMultiple choice Fill-in-the-blankFill-in-the-blank Scale - interval (e.g., Likert response format, Scale - interval (e.g., Likert response format,

semantic differential, Guttmann scale)semantic differential, Guttmann scale) Checklist – Checklist – see next slide…see next slide…

UnstructuredUnstructured Open ended / free answer / narrativeOpen ended / free answer / narrative

If Checklist...If Checklist...

are all alternatives covered?are all alternatives covered? is it of reasonable length?is it of reasonable length? is the wording impartial?is the wording impartial? is the form of the response easy, is the form of the response easy,

uniform?uniform? If its possible that there may be some If its possible that there may be some

other alternatives, provide “other” option other alternatives, provide “other” option and allow open ended response.and allow open ended response.

Decisions About Decisions About Question WordingQuestion Wording

Decisions About Question WordingDecisions About Question Wording

Questions about ‘nationality’ or ‘marital Questions about ‘nationality’ or ‘marital status are too vague.status are too vague.

Questions about ‘mass media’ are too Questions about ‘mass media’ are too general.general.

What kind of headache remedy do you What kind of headache remedy do you use?use?– Want brand name?Want brand name?– Want type? (Aspirin, pill, or capsule?)Want type? (Aspirin, pill, or capsule?)

Can the question be misunderstood?Can the question be misunderstood?Can the question be misunderstood?Can the question be misunderstood?

If you ask what social class someone’s in, If you ask what social class someone’s in, you assume that they know what social you assume that they know what social class is and that they think of themselves class is and that they think of themselves as being in one.as being in one.

Check assumption in a previous question.Check assumption in a previous question.

What assumptions does question make?What assumptions does question make?What assumptions does question make?What assumptions does question make?

Do you think Congress will cut taxes?Do you think Congress will cut taxes? Do you think Congress will successfully Do you think Congress will successfully

resist tax cuts?resist tax cuts? Neither of these specifies a time frame.Neither of these specifies a time frame.

Is the time frame specified?Is the time frame specified?Is the time frame specified?Is the time frame specified?

Are working conditions satisfactory or not Are working conditions satisfactory or not satisfactory in the plant where you work?satisfactory in the plant where you work?

» ObjectiveObjective Do you feel that working conditions are satisfactory Do you feel that working conditions are satisfactory

or not satisfactory in the plant where you work?or not satisfactory in the plant where you work?» FeelingFeeling

Are you personally satisfied with working conditions Are you personally satisfied with working conditions in the plant where you work?in the plant where you work?

» Personal satisfactionPersonal satisfaction

How personal is the wording?How personal is the wording?Ensure Appropriate Context for the studyEnsure Appropriate Context for the study

How did you feel about being in the war? How did you feel about being in the war? - May be too direct- May be too direct

How well did the equipment hold up in the How well did the equipment hold up in the field?field?

How well were new recruits trained?How well were new recruits trained?

Is the wording too direct?Is the wording too direct?Is the wording too direct?Is the wording too direct?

Does question contain difficult or unclear Does question contain difficult or unclear terminology?terminology?

Does the question make each alternative Does the question make each alternative explicit?explicit?

Is the wording objectionable?Is the wording objectionable? Is the wording loaded or slanted?Is the wording loaded or slanted?

Some Additional Issues...Some Additional Issues...Some Additional Issues...Some Additional Issues...

Question Placement Question Placement and Sequenceand Sequence

Decisions About PlacementDecisions About Placement

Is the answer influenced by prior Is the answer influenced by prior questions?questions?

Does question come too early or too Does question come too early or too late to arouse interest?late to arouse interest?

Does the question receive sufficient Does the question receive sufficient attention?attention?

The Opening QuestionsThe Opening Questions These are particularly true in the case of an These are particularly true in the case of an

interview.interview.– Should be easy to answerShould be easy to answer– Should not be sensitive materialShould not be sensitive material– Should get the respondent “rolling”Should get the respondent “rolling”

In a survey, considerIn a survey, consider– putting basic demographic questions towards putting basic demographic questions towards

the end of a survey because these are easy to the end of a survey because these are easy to complete and thus it may overcome “survey complete and thus it may overcome “survey fatigue.”fatigue.”

Sensitive QuestionsSensitive Questions

Only after trust is developedOnly after trust is developed Should make sense in that section of Should make sense in that section of

the survey (not “out of left field”)the survey (not “out of left field”) Precede with warm-up questionsPrecede with warm-up questions

A Checklist of Considerations A Checklist of Considerations (slide 1)(slide 1)

Start with easy, nonthreatening Start with easy, nonthreatening questions.questions.

Put more difficult, threatening questions Put more difficult, threatening questions near end.near end.

Never start mail survey with an open-Never start mail survey with an open-ended question.ended question.

Put demographics at end (unless Put demographics at end (unless needed to screen).needed to screen).

A Checklist of Considerations A Checklist of Considerations (slide 2)(slide 2)

Avoid demographics at beginning.Avoid demographics at beginning. For historical demographics, follow For historical demographics, follow

chronological order.chronological order. Ask about one topic at a time.Ask about one topic at a time. When switching topics, use a transition.When switching topics, use a transition. Reduce response set.Reduce response set. For filter or contingency questions, For filter or contingency questions,

make a flowchart.make a flowchart.

Important RuleImportant Rule

Keep questionnaire Keep questionnaire as short as as short as possible!possible!

InterviewsInterviews

The Role of the InterviewerThe Role of the Interviewer

Locate and enlist cooperation of Locate and enlist cooperation of respondents.respondents.

Motivate respondents to do good job.Motivate respondents to do good job. Clarify any confusion/concerns.Clarify any confusion/concerns. Observe quality of responses.Observe quality of responses. Conduct a good interview.Conduct a good interview.

Training the Interviewers Training the Interviewers (slide 1)(slide 1)

Describe the entire study.Describe the entire study. State who is sponsor of research.State who is sponsor of research. Teach enough about survey research Teach enough about survey research

(should respect method and be (should respect method and be motivated).motivated).

Explain the sampling logic and process.Explain the sampling logic and process. Explain interviewer bias.Explain interviewer bias. ““Walk through” interview.Walk through” interview.

Training the Interviewers Training the Interviewers (slide 2)(slide 2)

Explain selection procedures, includingExplain selection procedures, including– Reading mapsReading maps– Identifying householdsIdentifying households– Identify respondentsIdentify respondents

Rehearse interviewRehearse interview Explain supervisionExplain supervision Explain schedulingExplain scheduling

The Interviewer’s KitThe Interviewer’s Kit

Official-looking 3-ring notebookOfficial-looking 3-ring notebook MapsMaps Sufficient copies of surveySufficient copies of survey Official identificationOfficial identification A cover letterA cover letter A phone number to callA phone number to call

The InterviewThe Interview

Opening remarksOpening remarks Asking the questionsAsking the questions Obtaining adequate answersObtaining adequate answers Recording the responseRecording the response Concluding the interviewConcluding the interview

More Details on Following SlidesMore Details on Following Slides

Opening RemarksOpening Remarks

Gaining entry Gaining entry ---- appearance, appearance, confidence, non-threateningconfidence, non-threatening

Doorstep technique Doorstep technique ---- brief, suggest brief, suggest don’t askdon’t ask

IntroductionIntroduction -- -- name, organization, name, organization, credentialscredentials

Explaining the study Explaining the study ---- general, stress general, stress confidentialityconfidentiality

Asking the QuestionsAsking the Questions

Use questionnaire carefully, but Use questionnaire carefully, but informally.informally.

Ask questions exactly as writtenAsk questions exactly as written Follow the order given.Follow the order given. Ask every question.Ask every question. Don’t finish sentences.Don’t finish sentences. Don’t leave any questions blank.Don’t leave any questions blank.

Obtaining Adequate Responses - The Obtaining Adequate Responses - The ProbeProbe

Silent probe - a pauseSilent probe - a pause Overt encouragementOvert encouragement ElaborationElaboration

– How do you mean? Anything else?How do you mean? Anything else? Ask for clarificationAsk for clarification RepetitionRepetition

Recording the ResponseRecording the Response

Record responses immediatelyRecord responses immediately Include all probes (write them in)Include all probes (write them in) Use abbreviations where possible (for Use abbreviations where possible (for

exampleexample– R = respondent; DK = don’t know)R = respondent; DK = don’t know)

Concluding the InterviewConcluding the Interview

Thank respondent.Thank respondent. Don’t be brusque or hasty.Don’t be brusque or hasty. Immediately after leaving -- write down Immediately after leaving -- write down

any notes about how the interview went.any notes about how the interview went.

Advantages and Advantages and Disadvantages of Survey Disadvantages of Survey

Methods Methods (general assessment)(general assessment)

Visual PresentationsVisual Presentations Possible?Possible?

QuestionnaireQuestionnaire

GroupGroup MailMail Drop-OffDrop-Off

InterviewInterview

PersonalPersonal PhonePhone

Visual PresentationsVisual Presentations Possible?Possible?

YesYes YesYes YesYes NoNoYesYes

InterviewInterview

Long Response CategoriesLong Response Categories Possible?Possible?

InterviewInterview

Long Response CategoriesLong Response Categories Possible?Possible?

YesYes YesYes ?? NoNoYesYes

InterviewInterview

PrivacyPrivacy??

InterviewInterview

PrivacyPrivacy??

YesYes NoNo NoNo ??NoNo

InterviewInterview

FlexibleFlexible??

InterviewInterview

FlexibleFlexible??

NoNo NoNo YesYes YesYesNoNo

InterviewInterview

Open-ended QuestionsOpen-ended Questions Feasible?Feasible?

InterviewInterview

Open-ended QuestionsOpen-ended Questions Feasible?Feasible?

NoNo NoNo YesYes YesYesNoNo

InterviewInterview

Reading and WritingReading and Writing Needed?Needed?

InterviewInterview

Reading & WritingReading & Writing Needed?Needed?

YesYes YesYes NoNo NoNo??

InterviewInterview

Can You Can You Judge QualityJudge Quality of Response?of Response?

InterviewInterview

Can You Can You Judge QualityJudge Quality of Response?of Response?

NoNo ?? YesYes ??YesYes

InterviewInterview

High High Response RatesResponse Rates??

InterviewInterview

High High Response RatesResponse Rates??

NoNo YesYes YesYes NoNoYesYes

InterviewInterview

ExplainExplain Study in Person? Study in Person?

InterviewInterview

ExplainExplain Study in Person? Study in Person?

NoNo YesYes YesYes ??YesYes

InterviewInterview

Low Low CostCost??

InterviewInterview

Low Low CostCost??

YesYes NoNo NoNo NoNoYesYes

InterviewInterview

Staff and FacilitiesStaff and Facilities Low?Low?

InterviewInterview

Staff and FacilitiesStaff and Facilities Low?Low?

YesYes NoNo NoNo NoNoYesYes

InterviewInterview

Access to Access to Dispersed SamplesDispersed Samples??

InterviewInterview

Access to Access to Dispersed SamplesDispersed Samples??

YesYes NoNo NoNo NoNoNoNo

InterviewInterview

Respondent Has Respondent Has TimeTime to Formulate to Formulate Answers?Answers?

InterviewInterview

Respondent Has Respondent Has TimeTime to Formulate to Formulate Answers?Answers?

YesYes YesYes NoNo NoNoNoNo

InterviewInterview

PersonalPersonal Contact? Contact?

InterviewInterview

PersonalPersonal Contact? Contact?

NoNo YesYes YesYes NoNoYesYes

InterviewInterview

Long SurveyLong Survey Feasible?Feasible?

InterviewInterview

Long SurveyLong Survey Feasible?Feasible?

NoNo NoNo YesYes NoNoNoNo

InterviewInterview

QuickQuick Turnaround? Turnaround?

InterviewInterview

QuickQuick Turnaround? Turnaround?

YesYes NoNo NoNo YesYesNoNo

InterviewInterview

rmk chapter 4: survey research l types of surveys (q & i) types of surveys (q & i) types of...

question contentif

question contentdecisions

question wordingdecisions

question contentexamples

question contentexampledo

question necessary useful

contingency questionshave

response formatdecisions

Documents

webs and q-howe dualities in types bcd - arxiv ·...

jeopardy fiber types character -istics testingmisc. other...

forms & surveys · forms, and self-grading online quizzes...

q-1 : define “interview.” discuss types of q-2 : discuss

jeopardy types of clouds group appearance periodic table...

conducting wildlife surveys - texas a&m...

jeopardy types of chemical reactions chemical reactions rate...

jeopardy psychological tests intelligence testing types of...

jeopardy types of forces motion of objects motion of objects...

pavement condition surveys lecture 4. instructional...

jeopardy poem types poetry terms more poetry terms name that...

jeopardy design principles data types field properties...

class and statutory surveys. 2 surveys ► different types...

types of surveys

ecology coniferous & deciduous forest forest misc deciduous...

surveys objectives 1.to understand why surveys are popular...

web surveys question types for questionnaire design

guidelines: architectural historic resource surveys and...

2: types of studies1. 2 in chapter 2: 2.1 surveys 2.2...

different types of web-based surveys - selasturkiye