Download - Conducting Surveys at Cornell University
Conducting Surveys atCornell University
Marin Clarkberg, Associate Director Institutional Research and Planning
Yasamin Miller, DirectorSurvey Research Institute
Who We Are
Institutional Research and Planning (IRP)• Within the Division of Budget and Planning;
goal to inform institutional decision-making• Administers regular, large-scale surveys to
students and other University constituencies
Survey Research Institute (SRI)• Full service survey enterprise at Cornell• Designed, hosted, and analyzed hundreds of
surveys for non-profit, government, corporate, and Cornell clients
Rationales for surveys
Why do a survey?• Increasing call to have “real data” and to
assess processes and outcomes• Looks easy (and inexpensive… and maybe
even fun)
Why not do a survey?1. Lots of data is already available2. Survey fatigue3. Not as easy at it might seem at first4. Serious limitations on what survey data can
actually tell you about processes and outcomes
Why you shouldn’t:1. Data already exists
Everyone has some data…• Has existing data been thoroughly
analyzed and understood?
In addition, the University archives an enormous amount data about students (and other constituencies)• Students’ academic records• IRP Surveys, other surveys
Why you shouldn’t:2. Survey fatigue
Surveys are more common• IRP surveying regularly since 2000• CIT-hosted surveys, e.g. WebSurveyor:
oOver 1600 surveys in 2005-2006
Survey response rates are down• Senior Survey response rates:
61% in 1998; 50% in 2002; 45% in 2006
Student (staff, faculty) time is a university resource
Why you shouldn’t:3. There is such a thing as survey expertise
Question design• Bad questions give you bad data
Instrument design• Respondents often bail out of
unsatisfactory surveys
Sample design• Samples are often adequate to the task• Sampling saves all kinds of resources
Why you shouldn’t:4. Survey research has major limitations
Ability to generalize with survey data a function of response patterns• Respondents may differ in important
ways from nonrespondents
Surveys cannot demonstrate causation
What are the alternatives to surveys?
Ransack existing sources of data• Multiple sources of data help
“triangulate”• If nothing else, you can learn more about
what you don’t know (and thus what remains to be learned)
Look at alternate modes of collecting data• Some questions better addressed with
observation, focus groups, interviews, etc.
So when is a survey appropriate?
Existing data is well-understood• Unanswered questions clearly identified
Scope of survey will minimize the imposition on respondents• As short as possible, asked to as few as possible
Appropriate survey expertise in involved in designing the study• Allow ample time for coordination, consultation,
design and pre-testing
Shared understanding of study limitations• Start small and manage expectations
How to Survey
1. Develop a reasonable timeline2. Define your research questions3. Design the survey instrument
and sampling plan4. Develop a data security plan5. Notify and secure approvals6. Data analysis and reporting
1. Develop a reasonable timeline
Putting questions on the web, collecting responses, and even data analyses are the easy parts
Having established a clarity of task, four to six months is not an unreasonable amount to develop and pre-test a survey instrument
4. Develop a data security plan
Anonymous DataIdentities of respondents are never captured• Fewer data security concerns• Impossible to know who responded (or how often)• Impossible to link survey data with other data
sources
Confidential DataIdentities of respondents are kept, but secured
• Necessary to have very secure file storage• Possible to link survey data to other data of interest
5. Notify and Secure Approvals
IRP Survey CalendarNotification courtesy
Institutional Review BoardIf a project is “research” (i.e. develops or contributes to generalizable knowledge), it needs to be reviewed and approved by the IRB.
Data StewardsStudents: the Office of the Registrar
Student and Academic ServicesSAS-Research Group
Resources on campus
Find the help you need• Institutional Research and Planning (IRP)
o Serves the Universityo Stewards of much existing student survey datao Available to consult with institutional studies
(sampling plans, instrument design and review)
• Survey Research Institute (SRI)o Comprehensive survey services from initial planning to
data analysis and reportingo IRP uses SRI for survey hosting and administration
• Institutional Review Board for Human Participants
o First and last authority on mandated review requirements and processes – ask them.