i. what is a survey?
DESCRIPTION
I. What is a survey?. What is a survey?. A tool to collect information from or about people to describe, compare, or explain their knowlege, feelings, values, and behaviour . Source: F i nk , A. (2009 ) , How to conduct surveys, SAGE. What is a survey?. - PowerPoint PPT PresentationTRANSCRIPT
I. What is a survey?
What is a survey?
A tool to collect information from or about people to describe, compare, or explain their knowlege, feelings, values, and behaviour .
Source: Fink, A. (2009), How to conduct surveys, SAGE.
What is a survey?
Mostly a quantitative research tool ….
• Questionnaires sent by post
• Online questionnaires
• Face-to-face structured interviews
• Telephone interviews
Doing a survey – main stepsDesign survey
process
Develop questions
Test questions/tool and train
interviewers
Collect data
Analyze data
II. Survey design
Target group• Who do I want to interview? (who should be in and who out)
• Do I want to interview the whole target population or its sample?
Tracer studyTarget population: graduates
Sample: all graduates (of selected fields) or their sample
Timing • When is the best time to contact your target population?
• How much time is needed to collect responses?
Tracer study
Timing: at least 6-9 months after graduation
Urban Rural Urban Rural Urban Rural1 month 6 months 12 months
12 717 13
1913
18
13
26
17
28
22
26 49
24
39
2339
3525 23 21 20 14
Other (military/parental leave/studying/disabled)UnemployedTraining/probationWorking for familyEmigrationUnregistered employeeEntrepreneurRegistered employee
Source: The ETF Transition Study in the Kyrgyz Republic 2011/2012
Timing (Repeated) Cross-sectional design
Panel design
- population of graduates asked only once
- population of graduates asked repeatedly at different time points
Method of data collection
Self-administered interviews (e.g. web-based, paper-pencil )
Face-to-face interviews
Telephone interviews
Face-to-face interviews
• High response rate
• Possibility to clarify ambiguous answers/provide follow-up information
- More time needed
- More resources needed (most expensive)
Telephone interviews
• High response rate, but lower than in case of face-to-face
- Possibility of sample bias (not everybody has a phone)
• Less time consuming (in comparison with face-to-face interviews
• Less expensive (in comparison with face-to-face interviews)
Self-administered interviews
• Saves money and time
- Lower response rate
- Possible sample bias (web-based interviews)
• People tend to be more truthful while responding
Doing a survey – testing
Once a questionnaire is ready, it must be tested in the field!
- Do people understand to the questions?
- Do people complete the questionnaire?
- Do they often drop-out? - How long does it take to fill the questionnaire? - Can interviewers administer the questionnaire properly?
- Are all procedures standardised?
Doing a survey – testing- Try to anticipate the actual circumstances in which the survey will be conducted
- Choose respondents similar to the ones who will eventually complete the survey
- Enlist as many people in the trial as reasonable, considering also your resources - Focus on the clarity of the questions and the general format of the survey
Doing a survey – things to consider
• Reliability
• Costs
• Usefulness and credibility of results
• Comparability
• Validity
Doing a survey – things to consider
• Reliability
A reliable survey provides consistent information (i.e. repeated measurement yields the same results).
…… make sure that each question means the same thing to everyone.
Doing a survey – things to consider
• Validity
A valid survey provides accurate information.
Do our questions measure what they are supposed to measure?
How were the respondents selected?
Can we generalise our results? …………
III. How to increase a response rate?
What is a response rate?
The proportion of people who return the questionnaire out of the totalnumber of people to whom a questionnaire was distributed.
Example: If you distribute 100 questionnaires and you get back 80 of them,your response rate is 80%.
Response rate matters!
• If we get responses only from a small share of respondents →
- results can be biased and not representative
- results are not significant and useful
What to do?
1. Study preparation
Think about study feasibility!
Think how to reach your respondents/graduates!
- Do you know their addresses? Or their parents’ addresses?
- Do you know their telephone numbers?
- Can you contact them via internet?
- Do you need to use a snowball technique?
What to do?
2. Designing a questionnaire
- Add a pre-notification letter before contacting the respondents
- Add a cover letter while distributing the questionnaires
Personalised requests and communication are important!
- Make sure that respondents see the value of the survey
What to do?
2. Designing a questionnaire
- Limit the number of questions to those which are relevant
- Add clear instructions
- Use clear and simple language
- Add ‘thank you’ at the end of the questionnaire
What to do?
3. Fieldwork
Reminders!
e.g. postal survey – send reminders after 2 and 5 weeks
e.g. face-to-face/telephone interviews – contact respondents at least 3 times at different hour/day
- Contact people several times- Time all contacts- Use different strategies
Follow up of respondents is an important part of the survey fieldwork and can influence the quality of your results!
IV. How to ask a good question?
Doing a survey – preparing a questionnaire• What information is needed?
• Can such information be obtained?
• Format of questions: open vs. closed, scales etc.
• Length of the questionnaire
• The order of questions
• Piloting
Questionnaire – basic rules
• Introduction – provide a context of the survey
• Assure anonymity
• Simple format
• Clean and consistent lay-out
Questionnaire – questions order
• Begin with questions reflecting the subject of the study
• Ask people to recall things in a chronological order
• Group items that are of a similar topic
• Personal and demographic questions should be placed at the end
Questionnaire – length
• Too long questionnaire – negative effect on a response rate
• Too short questionnaire – limited possibilities to respond our objectives
Be efficient and ask questions that are relevant and needed!
Doing a survey – format of questions
Closed questionsExample: How would you rate your overall career perspectives over the next years?1. Very good2. Good3. Poor 4. Very poor
Open questionsExample: What are your overall career perspectives over the next years?------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Doing a survey – format of questions
Scales
1 2 3 4
Students always like to go to the school
Students are interested in the field of study
Students are satisfied with teaching and learning services
Students are present during the entire school day
Attendance to the school is regular during the school year
Thinking about students' motivation, please rate the following statements:(Likert) 1=strongly agree, 2=agree, 3=disagree, 4=strongly disagree
What is a good question?
• Avoid double negative questions
• Avoid asking two things in one question
• A question should have a meaning
• A question should be concrete
• A standard language should be used
• Avoid biased words and phrases
What is a good question?
• How bad is your teacher?
• How would you rate the quality of your teachers and teaching materials?
• Would you say that the students are not motivated to study?
Not concrete and biased
Two questions in one
Double negative question
What is a good question?
• Please, specify the expenditure on ICT:
• How teachers and students evaluate the quality of food provided at school?
• Do you think that math teachers are better than those teaching languages?
Abbrevition used – not clear
Two questions in one
Biased question
Doing a survey – scalesNumber of response categories depends on the survey purposes (usually between 3 to 5).
Example: Do you agree or disagree with the statement that students always like to go to school?
1=agree 2=neither agree nor disagree
3=disagree
1=strongly agree 2=agree 3=disagree 4=strongly disagree
1=strongly agree 2=agree 3=neither agree nor disagree
4=disagree 5=strongly disagree
Doing a survey – scales
Number of response categories must be balanced.
Example: Do you agree or disagree with the statement that students always like to go to school?
1=strongly agree, 2=agree, 3=disagree, 4=strongly disagree
1=strongly agree, 2=agree, 3=disagree
Doing a survey – closed questions/scales
Number of response categories for each question must be exhaustive.
Example: What is your current marital status?
1. Never married 2. Married / Living together
3. Divorced / Separated 3. Other
4. Widowed
Doing a survey – closed questions/scales
Always specify whether one answer or multiple answers are possible.
Example: What is your current marital status?
1. Never married 2. Married / Living together3. Other
Example: Why did you leave school before graduation?
1. Studies too difficult 2. I wanted to work 3. I needed to work4. I got married5. Other
References
A large part of the presentation was based on - Fink, A. (2009), How to conduct surveys, SAGE.