Download - Creating an Effective Survey Instrument
![Page 1: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/1.jpg)
CREATING AN EFFECTIVE SURVEY INSTRUMENT
SUZANNE SANNWALD
ASHFORD UNIVERSITY
INSTRUCTIONAL DESIGN & DELIVERY
EDU 652
DR. KATHY ZIENTEK
MAY 4, 2013
![Page 2: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/2.jpg)
CREATING AN
EFFECTIVE ONLINE SURVEY
INSTRUMENT
![Page 3: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/3.jpg)
1. Online Survey Benefits2. Survey Process3. Survey Goals: What, Who, Why & How4. Online Survey Design Hints5. Designing Questions
1. Writing Effective Questions
2. Survey Question Types
3. Question Sequencing
6. Feedback7. Results Processing
Overview
![Page 4: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/4.jpg)
Online Survey Benefits
Participation can be self-select, if openly available on a Web page
Participation can be targeted when distributed by e-mail
Quick turnaround, although potentially lower response rates
Cost efficient, particularly with large samples
Optional anonymity conducive to questions of a sensitive nature
(Schoniau & Fricker, 2002, p. xiv, viii-xx)
![Page 5: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/5.jpg)
What are the basic steps of survey creation?
Click icon to add pictureWhen creating online surveys, “the basic steps in the process
should be the same as with any other survey
medium” (Schoniau & Fricker, 2002, p. 6).
![Page 6: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/6.jpg)
Survey Process
Plan optimal survey timing considering time necessary for entire process
University of Wisconsin-Madison (2010) defines the survey process as follows:
An additional step is to apply findingsApply findings
(University of Wisconsin-Madison, 2010, p. 4)
![Page 7: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/7.jpg)
Survey Goals: What
What is the purpose of the survey? What do you want/need to learn?” According to Barlow (2010), surveys often
have a goal of gathering information about: “Feelings, attitudes, reactions to products or
services” “Anticipated reactions to new kinds of products
or services” “Current levels of behaviors/activities for
planning of or to gauge impact of programs” (p. 3).
![Page 8: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/8.jpg)
Survey Goals: Who
Who is your target population? Who will you use as your sample
(subgroup of your target population)? Sample Size Calculator
Reasons to use a sample: If the target population is too large or
widely dispersed To allow for more reminders and follow-up
contact Consider any relevant privacy issues
(Barlow, 2010, p. 4)(University of Wisconsin-Madison, 2010, p. 5, 17)
![Page 9: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/9.jpg)
Survey Goals: Why & How Why do you need the information about
the particular target population? How will the results help you or others? How will you use the results? How can you apply your findings?
Will you use information to change current or future service delivery?
Will you re-evaluate the impact of any changes?
(Johnson, n.d., p. 3)
![Page 10: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/10.jpg)
How else can we improve response rates?
Click icon to add pictureFrom the start, improve response rate by giving respondents
“clear indication of intent of survey and how information will
be used”(Barlow, 2010, p. 9).
![Page 11: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/11.jpg)
Online Survey Design Hints
Ensure and communicate protection of privacy, as appropriate
Provide indication of survey progress Limit questions per screen Limit use of graphics, understanding graphics may cause
bias Limit use of complex or matrix questions Restrict and validate responses when possible to reduce
response errors Add and test all skip logic to streamline survey Only require responses when necessary Ease navigation with format (e.g., spacing, fonts, shading) Thoroughly test survey, including technical compatibility(Schoniau & Fricker, 2002, p. 41-53)
![Page 12: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/12.jpg)
How can we improve reliability and validity?
Click icon to add pictureStrive for reliability and validity:• “Reliability is the extent to
which repeatedly measuring the same property produces the same result” and
• “Validity is the extent to which the survey question measures the property it is supposed to measure”
(University of Wisconsin-Madison, 2010, p. 6).
![Page 13: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/13.jpg)
Writing Effective Questions
Survey goals should drive and shape questions According to Johnson (n.d.), surveys reflect a
solely personal expression and should thus avoid: “Asking secondhand information” “Asking hypothetical questions” “Asking about causation” “Asking for solutions to complex issues” (p. 14)
Language used should be familiar to respondents Do not use abbreviations Avoid technical language and jargon
![Page 14: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/14.jpg)
Writing Effective Questions
“Questions are
effective only if all learners understand them and can answer
them the way you
intended”
(Horton, 2012, p. 243)
Focus on one issue per question Use simple, direct, neutral language
Keep both questions and choices short Avoid difficult, ambiguous, or leading language Avoid questions in the negative or with double
negatives Include clear instructions
Specify conditions such as time frames (e.g., within the past week)
Specify tasks required (e.g., select one, select all that apply)
Emphasize important word (e.g., “not”) Use parallelism with answer choices
Same length Equivalent grammar
(Barlow, 2010, p. 5, 7)(Horton, 2012, p. 244-
259)(Johnson, n.d., p. 11-
13)
![Page 15: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/15.jpg)
What types of questions are there?
Click icon to add pictureTo pick the right question
type, “consider what kind of information
you are trying to gather” (Horton, 2012, p. 220).
![Page 16: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/16.jpg)
Survey Question Types
May take more time for respondents to answer
Takes more time to process
May provide new or unique insight
Responses may be hard to interpret and analyze
Quick response time
Respondents are more limited in responses
Quick processing time
Insights are limited response options
Open-Ended “Fill in the blank” Responses
Closed or Structured Responses
(University of Wisconsin-Madison, 2010, p. 9)
![Page 17: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/17.jpg)
Survey Question Types: Closed
Examples: Multiple Choice True/False Checklists Attitude scales
Options should be exclusive (non-overlapping)
Options should be exhaustive Include an “Other” option when appropriate
For frequency, use specific number ranges versus “often, seldom, never”(Barlow, 2010, p. 5-6)
![Page 18: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/18.jpg)
Question Sequencing
Order questions for a logical flow Start with an easy question Start with questions that reflect the study topic
and/or pique interest Group similar questions For questions requiring recall, use the order in
which they happened Sequence questions about details before asking
for an overall assessment End with personal and/or demographic questions Enable forward and backward navigation through
questions (Johnson, n.d., p. 18)(University of Wisconsin-Madison, 2010, p.
11)
![Page 19: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/19.jpg)
Feedback
During Survey: Provide indication of survey progress Provide clear error/warning messages Provide an avenue for respondents to report
problems. Incomplete Surveys:
Follow-up with reminders to complete surveys Completed Surveys:
Send thank you messages for survey completion Consider providing incentives for survey completion Update respondents with survey results Plan outreach to respondents based on survey
results (Schoniau & Fricker, 2002, p. 45, 47, 52-53)
![Page 20: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/20.jpg)
Results Processing: By Survey Consider audience and purpose of result
sharing Define demographic characteristics of
sample Report survey response rate Determine handling of incomplete
surveys Provide general summary of results Attach survey as appendix
(Barlow, 2010, p. 13)
![Page 21: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/21.jpg)
Results Processing: By Question Quantitative Processing:
For open-ended responses Report key themes and their frequency
Qualitative Processing For closed responses Report frequency, percentages, and by
categories Find ways to visually represent
responses
(Barlow, 2010, p. 12)
![Page 22: Creating an Effective Survey Instrument](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c6d7964a79599e578b45c3/html5/thumbnails/22.jpg)
References
Barlow, P. (2010). Effective survey design and analysis [Presentation]. Retrieved from University of Wisconsin-La Crosse: http://www.uwlax.edu/catl/Presentations/T4TW/Effective%20Survey%20Design%20and%20Analysis+Worksheet%20-%20June%202010.pdf
Horton, W. (2012). E-Learning by design (2nd ed.). San Francisco, CA: Wiley.
Johnson, K. (n.d.). An introduction to effective survey design and administration [Presentation]. Retrieved from Penn State University Survey Research Center: http://studentaffairs.psu.edu/assessment/pdf/KurtJohnsonPresentationFA11.pdf
University of Wisconsin-Madison Office of Quality Improvement. (2010). Survey fundamentals: A guide to designing and implementing surveys. Retrieved from http://oqi.wisc.edu/resourcelibrary/uploads/resources/Survey%20Guide%20v%202.0.pdf
Schoniau, M., Fricker, R. D., & Elliott, M. N. (2002). Conducting research surveys via e-mail and the Web. [Adobe Digital Editions version]. Retrieved from http://site.ebrary.com/