market and social research part 7

Questionnaire designLearning outcomes:• To understand the process of creating a

questionnaire• To understand some of the problems in

devising a questionnaire• To appreciate some of the different

question formats that can be used

in a questionnaire

Oh no! Not another questionnaire to fill in!

Study Skills and Learning Resources - http://www.griot.org

A questionnaire is a tool for gathering information and recording responses from informants in the light of the survey objectives.

It is generally much misused in research and industry and a researcher should not underestimate the time or care required to prepare one.

Questionnaires don’t take hours to produce – they take days or longer!


The role of the questionnaire is to ensure that all parties involved in the research process obtain what they need/want. Parties involved in the questionnaire can include the client, the researcher, the interviewer, the respondent and the data processor. For a satisfactory research programme there must be clear communication between all these parties.

Client

Researcher

Interviewer and respondent

Data processor Questionnaire Study Skills and Learning

Resources - http://www.griot.org

Questionnaires may be used in person, on the phone or as self-completion so this choice changes the design and appearance.

Each word must be written (and delivered) extremely carefully, scales may be used.

The questionnaire should be piloted.


Factors that affect respondent answers include:

Retroactive inhibition, Proactive inhibition Emotional factorsRepressionTelescoping

These, and other aspects, have implications -so carefully developed materials are important


Retroactive inhibition - new information inhibits recall of previously memorised information

Proactive inhibition - previously memorised information inhibits the memorisation of new things

Emotional Factors - positive emotion can help, negative emotion can inhibit

Repression - respondents avoid the recall of information with unpleasant associations

Telescoping effect - remembering an event as occurring in the less distant past than it actually did


It is critical to be clear about exactly what information is to be sought: the researcher should always be asking, ‘How will this information be used? How will it contribute towards meeting the terms of reference?’ How are you going to measure or rank the information?


The initial considerations should focus on:• the research objectives• what information is needed• what to measure and type of response format required• issues to be included, and question content• Wording of the questions• Question sequence and question layout• Pretesting (piloting) and correction of problems• the profile and number of the informants• distribution and administration• the method of editing, coding and analysis to

be used: how, precisely, will you analyse your data?• the respondent's ability/willingness to access

factual information• reminder/monitoring/follow up procedure to be adopted


Before asking the questions:

• provide information; tell your respondent who you are, why the questionnaire is being distributed and what you intend to achieve

• stress the confidentiality of all views received• outline what if anything is in it for the respondent• consider the format of the questions, and the instructions you

provide. Instructions should be clear and concise, separated from the questions, and with a clear difference of text style

• consider what people know: you can only ask people what they know, not what they think other people know• consider what terms/vocabulary your informants are familiar with: what is everyday language to you may not be to them


Question content:

• Is each question relevant?

• Is each question generating information, not data?

• Will the informant understand the question?

• Is the question really asking what you think it is asking: one question, no ambiguity?

• Can the respondent answer the question? Do they have access to appropriate factual information?

• Will the respondent answer the question?


Question phrasing:

• use the simplest form possible• categories should be comprehensive and make

sense: they should be mutually exclusive• instructions should be clear and concise, separated

from the questions and with a clear difference of text style

• only ask people what they know, not what they think other people know

• keep instructions simple and consistent• group questions in appropriate ways


Question phrasing (contd.):

• pitch the questions appropriately for your informants• make sure there is no ambiguity• ensure there is no bias, or leading questions• make sure that the question asks for the types of

responses which are being sought• never assume the informant will understand jargon

or technical language• do not ask redundant questions• be careful of such words as often, regularly,

frequently, etc• be aware of the fact that the above points will apply to covering letters as well as questionnaires


Layout:

•questionnaires should be attractive to look at, brief (or at least: no longer than is necessary), easy to understand and reasonably quick to complete, with instructions clearly provided• filter questions should be provided: signposting and directing should be easy to understand and follow•group questions in sensible ways and in clear sections, which should be clearly differentiated from each other.


Layout (contd):

•sufficient space should be provide for informants to indicate responses or to write answers•a consistent style should be followed throughout the questionnaire• it is usual to have response boxes on the right, not the left, of the page


Popular styles of response format:

• open

• dichotomous (two options only)

• multiple-choice single response

• multiple choice multiple response

• scaled


Open ended questions

Answer is recorded in the respondent’s own words but coding is slow andMisinterpretation is possible

Three key open-ended questions which are useful:

1.What is good? 2.What is bad? 3.What

improvements could be made

Open ended questions (whilst still needing piloting) are quicker to construct that closed questions BUT REQUIRE CONSIDERABLE TIME AND ABILITY TO ANALYSE


Dichotomous questions - e.g:

What gender are you?

Male � Female �or

Are you a full time student at the

Uni. Of Herts?

Yes � No �


Multiple choice - single response:

What is your favourite colour (tick one box only):

White � Blue �Blue � Green �Red � Purple �Brown � Grey �Yellow � Other �


Multiple choice - multiple response:

Which of the following factors affected your choice to become a student at the University of Hertfordshire? Tick as many as apply:

Need for increased qualification �Desire for letters after name �No other option �Required by employer �Seen as stepping stone to promotion �Girlfriend/ boyfriend said it was a good

thing to do �Other �


Scaled - e.g. Consider the following statements and then, for each statement, circle the number which corresponds most closely with your own views, where:

1 = strongly agree

2 = agree

3 = neither agree nor disagree

4 = disagree

5 = strongly agree


Caroline is great 1 2 3 4 5

Ruth is dreadful 1 2 3 4 5

The University is good 1 2 3 4 5

Canteen food is rubbish 1 2 3 4 5

Students work hard 1 2 3 4 5

Lecturers are lazy 1 2 3 4 5


Question sequence - what factors should we consider?

• Filtering questions at the beginning if appropriate

• easy, non-threatening questions at the beginning

• questions of interest to informants early on

• a logical order should be followed, so that informants can see where the questionnaire is leading

• linking sections between different parts of the questionnaire: headings to indicate focus to informants

• personal, difficult, or demanding questions should be left until the end


Question layout - questionnaires should:

• be easy to read

• be aesthetically pleasing

• allow for easy recording of responses

• allow for clear instructions, to both informants and researchers

• be easy to follow


Pilot, revision…..pilot, revision…..pilot, revision….. Etc etc etc

• How often should you pilot? Until the questionnaire works.

• Who should you pilot it on? Informants of a similar type to the final informants

• What else do you pilot? Any covering letters, enclosures, or instructions the informants will receive


A good questionnaire is one which has exactly the right number of

questions, in the right form and at the right level, to generate the

required information - not data - to meet the terms of

reference/objectives of the research project.


Key elements of the questionnaire include:

• planning what to measure

• question content - issues to be included

• wording of the questions

• types of response format

• question sequence

• question layout

• identification of survey objectives

• pretesting (piloting) and correcting problems


market and social research part 7

Education

learning resources http

recall of information

important study skills

research process

research objectives

client researcher interviewer

questions question sequence

question content wording