Tailored Design MethodTailored Design Method

Social Exchange TheorySocial Exchange Theory

Asking questions is a social exchange Asking questions is a social exchange that is facilitated whenthat is facilitated when• Rewards are perceived as highRewards are perceived as high• Costs are perceived as lowCosts are perceived as low• Trust is establishedTrust is established

Rewards for RespondingRewards for Responding

Show positive regardShow positive regard Thank the respondentThank the respondent Ask for adviceAsk for advice Support group valuesSupport group values Provide incentivesProvide incentives Make the questionnaire interestingMake the questionnaire interesting Inform respondents that time to respond is Inform respondents that time to respond is

limitedlimited

Reduce Cost of RespondingReduce Cost of Responding

Avoid demanding languageAvoid demanding language Avoid using language that respondents will Avoid using language that respondents will

not understandnot understand Include a stamped return envelopeInclude a stamped return envelope Make the questionnaire appear short and Make the questionnaire appear short and

easy to respond toeasy to respond to Do not ask for personal information that is Do not ask for personal information that is

not critical to the design of the studynot critical to the design of the study

Establish TrustEstablish Trust

Provide a token of appreciationProvide a token of appreciation Secure sponsorship Secure sponsorship Put forth enough effort in Put forth enough effort in

constructing the survey to make the constructing the survey to make the task appear importanttask appear important

Designing Survey to Reduce Four Designing Survey to Reduce Four Common ErrorsCommon Errors

Sampling ErrorSampling Error Coverage ErrorCoverage Error Measurement ErrorMeasurement Error Non-response ErrorNon-response Error

Research-based Findings Research-based Findings

Multiple contacts are needed to Multiple contacts are needed to achieve a high response rate.achieve a high response rate.

Incentives increase response rates.Incentives increase response rates. Surveys that appear easy to Surveys that appear easy to

complete yield higher response complete yield higher response rates.rates.

Writing QuestionsWriting Questions

The goal in writing questions is to The goal in writing questions is to construct items that respondents will construct items that respondents will be willing and able to answer. be willing and able to answer.

Classification of Item TypesClassification of Item Types

Format of the itemFormat of the item Content of the itemContent of the item Topic of the itemsTopic of the items

Format of the ItemFormat of the Item

Open vs. ClosedOpen vs. ClosedMiddle position offered vs. omittedMiddle position offered vs. omittedFiltered vs. unfilteredFiltered vs. unfilteredBalanced vs. unbalanceBalanced vs. unbalanceUnbalance with/without substantive Unbalance with/without substantive

counter argumentcounter argument

Example Open QuestionsExample Open Questions

What should be done to improve the What should be done to improve the attendance policy in this school?attendance policy in this school?

What factors explain the increased What factors explain the increased level of violence in schools?level of violence in schools?

How many times during the last How many times during the last week did you eat in a restaurant with week did you eat in a restaurant with table service?table service?

Closed Questions with Ordered Closed Questions with Ordered ChoicesChoices

The attendance policy in this school The attendance policy in this school needs to be made more stringent?needs to be made more stringent?• Strongly agreeStrongly agree• AgreeAgree• Neither agree nor disagreeNeither agree nor disagree• DisagreeDisagree• Strongly disagreeStrongly disagree

Closed Question with Ordered Closed Question with Ordered ChoicesChoices

In general, how consistently do you In general, how consistently do you think teachers in this building apply think teachers in this building apply the attendance policy?the attendance policy?• Not at all consistentlyNot at all consistently• Somewhat consistentlySomewhat consistently• Very consistentlyVery consistently

Closed Question with Ordered Closed Question with Ordered ChoicesChoices

How often do you send students to the How often do you send students to the office because they are late to your class?office because they are late to your class?• Every dayEvery day• Three or four times a weekThree or four times a week• One or two times a weekOne or two times a week• Two or three times a monthTwo or three times a month• About once a monthAbout once a month

Closed Questions with Closed Questions with Unordered ChoicesUnordered Choices

Which of the following actions would improve the Which of the following actions would improve the attendance policy in our school?attendance policy in our school?a. Uniform enforcement of the attendance policy a. Uniform enforcement of the attendance policy

by all teachers in the building by all teachers in the buildingb. Strong parental notification policies for b. Strong parental notification policies for

tardinesstardinessc. Consistent enforcement of the attendance c. Consistent enforcement of the attendance

policy by administratorspolicy by administratorsd. More liberal definitions of tardinessd. More liberal definitions of tardinesse. Stricter punishments for tardinesse. Stricter punishments for tardiness

Partially Closed QuestionPartially Closed Question Why do you think the school had a low rate of Why do you think the school had a low rate of

attendance at the last parent meeting?attendance at the last parent meeting?• The meeting conflicted with some other eventThe meeting conflicted with some other event• The meeting was scheduled at a time when many The meeting was scheduled at a time when many

parents were workingparents were working• There was no student program at the meetingThere was no student program at the meeting• The topic of the meetings was of low interest to parentsThe topic of the meetings was of low interest to parents• Parents who work all day are unwilling to attend evening Parents who work all day are unwilling to attend evening

meetingsmeetings• Other reason ______________________________Other reason ______________________________

Middle PositionMiddle Position

Omitted FormOmitted Form In your opinion should penalties for using In your opinion should penalties for using

marijuana be more strict or less strict than marijuana be more strict or less strict than they are now?they are now?

Offered FormOffered Form In your opinion should penalties for using In your opinion should penalties for using

marijuana be more strict, less strict, or marijuana be more strict, less strict, or about the same as they are now?about the same as they are now?

Filtered QuestionsFiltered Questions

Filtered questions provide the Filtered questions provide the respondent with an opportunity to respondent with an opportunity to indicate that he/she does not have indicate that he/she does not have the information needed to respond.the information needed to respond.

Example of a Filtered ItemExample of a Filtered Item

Some teachers believe that out-of- Some teachers believe that out-of- school suspension is the only way to school suspension is the only way to control some types of student control some types of student behavior; other teachers believe in-behavior; other teachers believe in-school-suspension works better while school-suspension works better while some teachers have not thought some teachers have not thought about it enough to have formed an about it enough to have formed an opinion. What is your opinion?opinion. What is your opinion?

Fully Filtered ItemFully Filtered Item

I am going to ask your opinion about I am going to ask your opinion about punishment for marijuana use. You may punishment for marijuana use. You may not have an opinion and that is fine. Just not have an opinion and that is fine. Just say I don’t have an opinion.say I don’t have an opinion.

Do you favor or oppose stricter penalties Do you favor or oppose stricter penalties for marijuana use or have you not formed for marijuana use or have you not formed an opinion about penalties for marijuana an opinion about penalties for marijuana use? use?

Examples of Balanced QuestionExamples of Balanced Question

Do you favor or do you oppose the Do you favor or do you oppose the State Board of Education’s recent State Board of Education’s recent decision to increase the number of decision to increase the number of science credits required for high science credits required for high school graduation?school graduation?

Example of Unbalanced ItemExample of Unbalanced Item

Do you favor the Lawrence School Do you favor the Lawrence School Board’s current recommendation to Board’s current recommendation to close Riverside School.close Riverside School.

Balanced with Counter Balanced with Counter ArgumentArgument

Some people believe that free and Some people believe that free and appropriate public education should be appropriate public education should be interpreted to mean tax dollars can be interpreted to mean tax dollars can be used to send children to private schools used to send children to private schools others disagree because they believe others disagree because they believe private schools foster separatist and elitist private schools foster separatist and elitist attitudes. Do you agree or disagree that attitudes. Do you agree or disagree that free and appropriate public education free and appropriate public education means tax dollars can be used to send means tax dollars can be used to send children to private schools? children to private schools?

Classification of QuestionsClassification of Questions

Content MeasuredContent Measured• KnowledgeKnowledge• Attitudes Attitudes • BehaviorBehavior

Nature of the Topic Measured Nature of the Topic Measured by the Itemby the Item

Socially desirable/undesirableSocially desirable/undesirable ThreateningThreatening

Socially Desirable TopicsSocially Desirable Topics

Topics are considered socially Topics are considered socially desirable if respondents believe they desirable if respondents believe they will be viewed positively if they will be viewed positively if they respond in a particular way.respond in a particular way.

A socially desirable attitude for A socially desirable attitude for teachers is “I try to meet the needs teachers is “I try to meet the needs of all of my students.”of all of my students.”

Socially Undesirable TopicsSocially Undesirable Topics

Attitudes or behaviors individuals do not Attitudes or behaviors individuals do not want acknowledged even though they want acknowledged even though they engage in the behavior or hold the engage in the behavior or hold the attitude.attitude.

Attitudes or behaviors individuals do not Attitudes or behaviors individuals do not hold or engage in. They also don’t want hold or engage in. They also don’t want people to think they do.people to think they do.

Threatening TopicsThreatening Topics

Questions are threatening if the Questions are threatening if the respondent thinks there is a “right” respondent thinks there is a “right” answer and they don’t know what it is or answer and they don’t know what it is or they know what it is and don’t endorse or they know what it is and don’t endorse or practice it.practice it.

A threatening item for teachers might be: A threatening item for teachers might be: “All students should be taught together in “All students should be taught together in the same classrooms regardless of ability.”the same classrooms regardless of ability.”

Social Exchange TheorySocial Exchange Theory

Asking questions is a social exchange Asking questions is a social exchange that is facilitated whenthat is facilitated when• Rewards are perceived as highRewards are perceived as high• Costs are perceived as lowCosts are perceived as low• Trust is establishedTrust is established

Designing Survey to Reduce Four Designing Survey to Reduce Four Common ErrorsCommon Errors

Sampling ErrorSampling Error Coverage ErrorCoverage Error Measurement ErrorMeasurement Error Non-response ErrorNon-response Error

Research-based Findings Research-based Findings

Multiple contacts are needed to Multiple contacts are needed to achieve a high response rate.achieve a high response rate.

Incentives increase response rates.Incentives increase response rates. Surveys that appear easy to Surveys that appear easy to

complete yield higher response complete yield higher response rates.rates.