ADVANCED RESEARCH TECHNIQUES
QUESTIONNAIRES
CEMAL ESKİCİ
QUESTIONNAIRES Introduction Questionnaires in language research
What are questionnaires What do they measure Advantages and disadvantages of
questionnaires Constructing the questionnaires
General features The main parts of a questionnaire Questionnaire content and multi-item scales Close-ended questionnaire items Open-ended questions
QUESTIONNAIRES
Introduction One of the most common methods of data
collection in research is to use various kinds of questionnaires. (Dörnyei, 2003)
A survey is a system for collecting information to describe, compare, or explain knowledge, attitudes, and practices or behaviour. (Fink, 1995)
The essential point in research is that it cannot be built on poorly collected data. (Gillham, 2004)
QUESTIONNAIRES
What are questionnaires?
Brown (cited in Dörnyei, 2003)
expresses that questionnaires are any
written instruments that presents the
respondents with a series of questions or
statements to which they are to react
either by writing out their answers or
selecting from among existing answers.
QUESTIONNAIRESWhat are questionnaires?
Dörnyei (2003) states that although the term
‘questionnaire’ is familiar, it’s hard to provide a
precise definition for it. He reasons that many
of the questionnaires do not contain any or
many real questions that end with a question
mark.
QUESTIONNAIRESWhat are questionnaires?
He adds that the other reason is
that the rubric of ‘questionnaire’ has
been used by researchers in at least two
broad senses:
Interview schedules
Self-administered pencil and paper
questionnaires
QUESTIONNAIRES
What are questionnaires?
Aiken (cited in Dörnyei, 2003)
explains that questionnaires are often
referred to as ‘forms, opinnionaires,
tests, batteries, checklists, scales,
surveys, schedules, studies, profiles,
indexes, indicators, or even simply
sheets.’ (p. 5)
QUESTIONNAIRES
What do questionnaires measure? Factual questions- classification
questions or subject descriptors are used to find out about who the respondents are.
Behavioral questions are used to find out what the respondents are doing or have done in the past.
Attitudinal questions are used to find out what people think. (Dörnyei, 2003)
QUESTIONNAIRES
AdvantagesThe main attraction of questionnaires is their unprecedented efficiency in terms of: researcher time researcher effort financial resources. (Dörnyei,2003)
QUESTIONNAIRES
AdvantagesQuestionnaires are: easy to conduct extremely versatile uniquely capable of gathering a
large amount of information quickly in a form that is readily processible. (Dörnyei,2003)
QUESTIONNAIRES
Advantages Low cost in time and money. Easy to get information from a lot of
people very quickly. Respondents can complete the
questionnaire when it suits them. Analysis of answers to closed
questions is straightforward.
QUESTIONNAIRES
Advantages Less pressure for an immediate
response. Respondents’ anonymity. Lack of interviewer bias. Standardization of questions. Can provide suggestive data for
testing an hypothesis. (Gillham, 2004)
QUESTIONNAIRES
Disadvantages Simplicity and superficiality of
answers Unreliable and unmotivared
respondents Respondent literacy problems Little or no opportunity to correct
the respondents’ mistakes Social desirability (or prestige) bias Self-deception
QUESTIONNAIRES
Disadvantages Acquiescence bias Halo effect Fatigue effects Problems of data quality
(completeness and accuracy). Typically low response rate unless
sample «captive».
QUESTIONNAIRES
Disadvantages Questionnaire development is often
poor. Seek information just by asking
questions. Assumes respondents have answers
available in an organized fashion.
QUESTIONNAIRES
Disadvantages Lack of control over order and
context of answering questions. Question wording can have a major
effect on answers. People talk more easily than they
write. Respondent uncertainty as to what
happens to data. (Dörnyei, 2003; Gillham, 2004)
Constructing the questionnairesThe steps required to design and administer a questionnaire include:
Defining the objectives of the survey
Determining the sampling group
Deciding on the general features of the
questionnaire
Writing effective items/questions and drawing
up an item pool (Dörnyei.2003)
QUESTIONNAIRES
QUESTIONNAIRES
Constructing the questionnaires
Selecting and sequencing the items
Writing appropriate instructions and
examples
Piloting the questionnaire
Conducting item analysis
QUESTIONNAIRES
General featuresLength
Dörnyei (2003) suggests that to
be effective a questionnaire should not
be more than 4 pages and it shouldn’t
take more than 30 minutes for the
respondents to answer all the
questions.
QUESTIONNAIRES
General featuresLayout: An attractive and proffessional design involves: Booklet format Appropriate density Orderly layout Paper quality Sequence marking
QUESTIONNAIRES
General featuresSensitive topics and anonymity
Dörnyei (2003) claims that these are not only confined to explicitly illegal or embarrassing subjects but also include basic demographic items such as age and marital status.
He also states that to diffuse sensitive items we make the questionnaire anonymous.
QUESTIONNAIRES
General featuresSensitive topics and anonymity
Oppenheim (cited in Dörnyei, 2003) suggests that to overcome the issues concerning sensitive topics and anonymity, something along the following line should be displayed prominently on the front of the questionnaire:
«The contents of this form are absolutely confidential. Information identifying the respondent will not be diclosed under any circumstances.»
QUESTIONNAIRES
The main parts of a questionnaire Title Instructions
1. General instructions2. Specific instructions
Questionnaire items Additional information Final «Thank you!»
QUESTIONNAIRES
Questionnaire content and multi-item scalesAppropriate sampling of the content«The temptation is always to cover too much, to ask everything that might turn out to be interesting. This must be resisted.» (Moser & Kalton, cited in Dörnyei, 2003 p.32) The research problem should be clarified. What critical concepts need to be
addressed by the questionnaire should be identified.
QUESTIONNAIRESQuestionnaire content and multi-item scalesUsing multi-item scales Close-ended questionnaire items
Rating scales Likert scalesEx. strongly agree strongly disagree;
51
QUESTIONNAIRES
Close-ended questionnaire items Semantic differential scalesEx. Listening comprehension tasks are:difficult__:__:__:_x_:__:__:__ easyuseless__:__:__:__:__:_x_:__ useful Numerical rating scales True- False items
QUESTIONNAIRES
Close-ended questionnaire itemsMultiple choice items Response options should be short. Responses should be put in a natural
order or in random or alphabetical order.
All the options should be grammatically correct with respect to the stem.
Using negative expressions in both the stem and the response options should be avoided.
QUESTIONNAIRES
Close-ended questionnaire items Rank order items Numeric items Checklists
QUESTIONNAIRES
Open-ended questionsDisadvantages They take up more time and restrict
the range of topics the questionnaire can contain.
They are difficult to code in reliable manner.
QUESTIONNAIRES
Open-ended questions
Specific open questions Clarification questions Sentence completion items Short answer questions
QUESTIONNAIRESReferences
Aiken, L. (1997). Questionnaires and inventories: Surveying opinions and assessing personality. New York: John WileyBrown, J. D. (2001). Using surveys in language
programs. Cambridge, UK: Cambridge University Press
Dörnyei, Z. (2003). Questionnaires in second language research. Mahwah, New Jersey: Lawrence Erlbaum Associates
Fink, A. (1995). How to ask surveys questions. Thousand Oaks, California: Sage
QUESTIONNAIRES
Gillham, B. (2004). Developing a questionnaire. London: ContinuumHopkins, K.D., Stanley, J.C. & Hopkins, B.R.(1990). Educational and psychological
measurement and evaluation (7TH Ed.). Englewood Cliffs, NJ: Prentice HallLow, G.(1999).What respondent do with questionnaires: Accounting for incongruity
and fluidity. Applied Linguistics, 20, 503-533
QUESTIONNAIRES
Moser, C. A. & Kalton, G. (1971). Survey methods in social investigation. London: HeinemannNewell, R. (1993). Questionnaires. In N. Gilbert (Ed.) Researching social life. (pp 94-115) London: SageOppenheim, A. N. (1992). Questionnaire design, interviewing and attitude measurement (New Edition). London: Pinter
QUESTIONNAIRES
Robinson, J.P., Shaver, P.R. &Wrightson, L.S. (1991). Criteria for scale selection and evaluation. In J.P. Robinson, P.R. Shaver
& L.S. Wrightson (Eds.) Measures of personality and social psychological attitudes (pp1-16). San Diego, CA: Academic PressRobson, C. (1993). Real world research: A source for social scientists and practitioner-researchers. Oxford: Blackwell
QUESTIONNAIRES
Sanchez, M. E. (1992). Effects of questionnaire design on the quality of survey design. Public Opinion Quarterly, 56,216-217Wilson, N., McClean, S. (1994). Questionnaire design. Newtownabbey, Northern Ireland: University of Ulster
QUESTIONNAIRES
How would you rate this presentation?
Good Very Good Excellent
Thank you for your attention.
Cemal ESKİCİ