questionnaire design dr. abdulaziz ali almezam, md, msc, abcm family and community medicine...

Questionnaire DesignQuestionnaire Design

Dr. Abdulaziz Ali Almezam, MD, MSC, ABCMDr. Abdulaziz Ali Almezam, MD, MSC, ABCM Family and Community Medicine DepartmentFamily and Community Medicine Department

Collage of Medicine, King Saud UniversityCollage of Medicine, King Saud University


Dr. Abdulaziz Ali Almezam, MD, MSC, ABCMDr. Abdulaziz Ali Almezam, MD, MSC, ABCM Family and Community Medicine DepartmentFamily and Community Medicine Department

Collage of Medicine, King Saud UniversityCollage of Medicine, King Saud University

nnnn


What Is A QuestionnaireWhat Is A Questionnaire??

A tool for collecting information toA tool for collecting information to describe, compare, or explain knowledge, describe, compare, or explain knowledge, attitudes, behaviorsattitudes, behaviors,,

and/or socio-demographic characteristicsand/or socio-demographic characteristics on a particular target groupon a particular target group..

Well designed questionnairesWell designed questionnaires

A highly structured to allow the sameA highly structured to allow the sametypes of information to be collected from types of information to be collected from a large number of people in the same a large number of people in the same way and for data to be analyzed way and for data to be analyzed quantitatively and systematically. quantitatively and systematically.

Well designed questionnairesWell designed questionnaires

Questionnaires are best used for collecting Questionnaires are best used for collecting factual data and appropriate questionnaire factual data and appropriate questionnaire design is essential to ensure that wedesign is essential to ensure that weobtain valid responses to our questionsobtain valid responses to our questions

What can questionnaires What can questionnaires measuremeasure??

- Questionnaires are quite flexible in what - Questionnaires are quite flexible in what they can measure, however they are not they can measure, however they are not equally suited to measure all types of data. equally suited to measure all types of data. - We can classify data in two ways:- We can classify data in two ways: Subjective vs. Objective Subjective vs. Objective Quantitative vs. Qualitative. Quantitative vs. Qualitative.

Objectives in designing Objectives in designing questionnairesquestionnaires

There are two main objectives in designingThere are two main objectives in designinga questionnaire: a questionnaire: To maximize the proportion of subjects To maximize the proportion of subjects

answering our questionnaire - that is, the answering our questionnaire - that is, the response rateresponse rate. .

To obtain To obtain accurate relevant accurate relevant information information for our survey. for our survey.

Response Rate and RespondentsResponse Rate and Respondents

To maximize our response rate, we have to To maximize our response rate, we have to consider carefully how we administer the consider carefully how we administer the questionnaire, establish rapport, explain questionnaire, establish rapport, explain the purpose of the survey, and remind the purpose of the survey, and remind those who have not responded. The length those who have not responded. The length of the questionnaire should be appropriateof the questionnaire should be appropriate . .

Obtaining accuracy & relevancyObtaining accuracy & relevancy

In order to obtain accurate and relevant In order to obtain accurate and relevant information, we have to give some thought information, we have to give some thought to what questions we ask, how we ask to what questions we ask, how we ask them, the order we ask them in, and the them, the order we ask them in, and the general layout of the questionnairegeneral layout of the questionnaire

Obtaining accuracy & relevancyObtaining accuracy & relevancy

The length of the questionnaire shouldThe length of the questionnaire should be appropriatebe appropriate . .

In order to obtain accurate relevant In order to obtain accurate relevant information, we have to give some thought information, we have to give some thought to what questions we ask, how we ask to what questions we ask, how we ask them, the order we ask them in, and the them, the order we ask them in, and the general layout of the questionnairegeneral layout of the questionnaire

potential types of informationpotential types of information::

There are three potential types of information: There are three potential types of information: Information we are primarily interested in Information we are primarily interested in

that is, that is, dependent variablesdependent variables. . Information which might explain the Information which might explain the

dependent variables-that is, dependent variables-that is, independent independent variablesvariables. .

Other factors related to both dependent and Other factors related to both dependent and independent factors which may distort the independent factors which may distort the results and have to be adjusted for - that is, results and have to be adjusted for - that is, confounding variablesconfounding variables. .

The steps required to designThe steps required to designa questionnairea questionnaire??

The steps required to design and The steps required to design and

administer a questionnaire include: administer a questionnaire include: Defining the Objectives of the survey. Defining the Objectives of the survey. Determining the Sampling Group.Determining the Sampling Group. Literature Review. Literature Review. Writing the Questionnaire. Writing the Questionnaire. Administering the Questionnaire. Administering the Questionnaire. Data Entry and analysis.Data Entry and analysis. Interpretation of the Results.Interpretation of the Results.

Several points that must by Several points that must by consideredconsidered?? ::

11..Clarity:Clarity: This is probably the area that causes the This is probably the area that causes the greatest source of mistakes in questionnaires. greatest source of mistakes in questionnaires. Questions must be clear, succinct (brief), and Questions must be clear, succinct (brief), and unambiguousunambiguous . .

The goal is to eliminate the chance that the question The goal is to eliminate the chance that the question will mean different things to different peoplewill mean different things to different people..If the designers fails to do this, then essentially If the designers fails to do this, then essentially participants will be answering different questionsparticipants will be answering different questions . .

Example of ClarityExample of Clarity::such as:such as:Very OftenVery Often OftenOften SometimesSometimes RarelyRarely

NeverNever

It is better to quantify the choices, such as:It is better to quantify the choices, such as:Every Day or MoreEvery Day or More

2-62-6 Times a WeekTimes a Week About Once a WeekAbout Once a Week About Once a MonthAbout Once a Month

NeverNever

22 . .Leading QuestionsLeading Questions::

A leading question is one that forces or implies a certain A leading question is one that forces or implies a certain type of answer. It is easy to make this mistake not in type of answer. It is easy to make this mistake not in the question, but in the choice of answersthe question, but in the choice of answers..

A closed format question must supply answers that not A closed format question must supply answers that not only cover the whole range of responses, but that are only cover the whole range of responses, but that are also equally distributed throughout the rangealso equally distributed throughout the range..All answers should be equally likelyAll answers should be equally likely . .

Example of a Leading QuestionsExample of a Leading Questions::

An obvious, nearly comical, example would be a question An obvious, nearly comical, example would be a question that supplied these answer choicesthat supplied these answer choices::

SuperbSuperb ExcellentExcellent GreatGreat GoodGood FairFair

Not so GreatNot so Great

33..PhrasingPhrasing

Most adjectives, verbs, and nouns in English have Most adjectives, verbs, and nouns in English have either a positive or negative either a positive or negative

connotation(meaning)connotation(meaning) . .Two words may have equivalent meaning, yet one Two words may have equivalent meaning, yet one

may be a compliment and the other an insultmay be a compliment and the other an insult . .Consider the two words "child-like" and "childish", Consider the two words "child-like" and "childish", which have virtually identical meaning. Child-which have virtually identical meaning. Child-like is an affectionate term that can be applied like is an affectionate term that can be applied to both men and women, and young and old, yet to both men and women, and young and old, yet

no one wishes to be thought of as childishno one wishes to be thought of as childish..

44 . .Embarrassing QuestionsEmbarrassing Questions::

Embarrassing questions dealing with personal or Embarrassing questions dealing with personal or private matters should be avoided. Your data private matters should be avoided. Your data is only as good as the trust and care that is only as good as the trust and care that your respondents give you. If you make them your respondents give you. If you make them feel uncomfortable, you will lose their trust. feel uncomfortable, you will lose their trust. Do not ask embarrassing questionsDo not ask embarrassing questions..

55..Hypothetical QuestionsHypothetical Questions

Hypothetical are based, at best, on conjecture(Thinking) Hypothetical are based, at best, on conjecture(Thinking) and, at worst, on fantasyand, at worst, on fantasy . .Simple question such asSimple question such as : :If you were governor, what would you do to stop crimeIf you were governor, what would you do to stop crime??

This forces the respondent to give thought to This forces the respondent to give thought to something he may have never considered. This something he may have never considered. This does not produce clear and consistent data does not produce clear and consistent data

representing real opinionrepresenting real opinion . .Do not ask hypothetical questionsDo not ask hypothetical questions..

66 . .Prestige BiasPrestige Bias : :

Prestige bias is the tendency for respondents to Prestige bias is the tendency for respondents to answer in a way that make them feel better. answer in a way that make them feel better. People may not lie directly, but may try to put People may not lie directly, but may try to put

a better light on themselvesa better light on themselves . .Are You SmokerAre You Smoker??

Questionnaire General FormatQuestionnaire General Format

Self administered (mailed or personal Self administered (mailed or personal contact).contact).

In person (face-to-face).In person (face-to-face). Telephone interviews.Telephone interviews.

Questionnaire General FormatQuestionnaire General Format

The exact method of administration also The exact method of administration also depends on who the respondents are? depends on who the respondents are?

For example, university lecturers may be For example, university lecturers may be more appropriately surveyed by email; more appropriately surveyed by email; older people by telephone interviews;older people by telephone interviews;

train passengers by face to face train passengers by face to face interviews.interviews.

Self administered questionnairesSelf administered questionnaires

May be sent byMay be sent by PostPost EmailEmail

Electronically onlineElectronically online

Interview administered Interview administered questionnairesquestionnaires

May be byMay be by TelephoneTelephone face to faceface to face

The Questions or ItemsThe Questions or Items

Are the focus on any survey or Are the focus on any survey or questionnaire.questionnaire.

It is crucial to know how to ask the It is crucial to know how to ask the questions in written and spoken form.questions in written and spoken form.

The way you ask the questions determines The way you ask the questions determines the answers.the answers.

Questions ContextQuestions Context

Standardize the interviewer.Standardize the interviewer. Standardize the response format.Standardize the response format. Ask questions in a social, cultural, and economic Ask questions in a social, cultural, and economic

context.context. Keep confidentiality and voluntary participation.Keep confidentiality and voluntary participation. Include a letter of introduction or presentation.Include a letter of introduction or presentation.

Questions FormatQuestions Format

Open questions – more information but Open questions – more information but difficult to codify, enter, and analyze.difficult to codify, enter, and analyze.

Closed questions – less information but Closed questions – less information but easy to codify, enter, and analyze.easy to codify, enter, and analyze.

Advantages of open formatAdvantages of open format

Allows exploration of the range of possible Allows exploration of the range of possible themes arising from an issue. themes arising from an issue.

Can be used even if a comprehensive Can be used even if a comprehensive range of alternative choices cannot be range of alternative choices cannot be compiled (explained). compiled (explained).

Advantages of Closed-thatAdvantages of Closed-thatis, forced choice-formatis, forced choice-format

Easy and quick to fill in. Easy and quick to fill in. Minimize discrimination against the less literate Minimize discrimination against the less literate

(in self administered questionnaire) or the less (in self administered questionnaire) or the less articulate(explain) (in interview questionnaire). articulate(explain) (in interview questionnaire).

Easy to code, record, and analyze results Easy to code, record, and analyze results quantitatively. quantitatively.

Easy to report results.Easy to report results.

Types of closed (forced choice) Types of closed (forced choice) formatformat

1. Choice of categories:1. Choice of categories:Fro example, “What is your marital status”?Fro example, “What is your marital status”? SingleSingle MarriedMarried Divorced Divorced WidowedWidowed


2. Likert style scale:2. Likert style scale: For example, “Statistics is an interesting For example, “Statistics is an interesting

subject”.subject”. Strongly disagree Strongly disagree DisagreeDisagree Cannot Cannot

decide decide Agree Agree Strongly agree Strongly agree


3. Differential Scales3. Differential Scales For example, “How would you rate the presentation”?For example, “How would you rate the presentation”? Extremely interesting>>> 1 2 3 4 5 6 7 Extremely interesting>>> 1 2 3 4 5 6 7 8 9 10 >>>>Extremely dull8 9 10 >>>>Extremely dull It may also be diagrammatic.It may also be diagrammatic.


4. Checklists4. ChecklistsFor example, “Circle the clinical specialties you areFor example, “Circle the clinical specialties you areparticularly interested in”.particularly interested in”.General medicineGeneral medicine OBS/GYNOBS/GYN General SurgeryGeneral Surgery OrthopedicsOrthopedics OphthalmologyOphthalmology Accident and EmergencyAccident and Emergency Pediatrics Pediatrics General practiceGeneral practice


5. Ranking5. RankingFor example, “Please rank your interests in the followingFor example, “Please rank your interests in the followingspecialties”.specialties”.(1 = most interesting >>>>>8 = least interesting)(1 = most interesting >>>>>8 = least interesting) General medicineGeneral medicine OBS/GYNOBS/GYN General SurgeryGeneral Surgery OrthopaedicsOrthopaedics OphthalmologyOphthalmology Accident and EmergencyAccident and Emergency Pediatrics Pediatrics General practiceGeneral practice

How to obtain valid informationHow to obtain valid information??

Ask purposeful questions.Ask purposeful questions. Ask concrete questions.Ask concrete questions. Use time periods based on importance of the Use time periods based on importance of the

question.question. Use conventional language.Use conventional language. Use complete sentences.Use complete sentences. Avoid abbreviationsAvoid abbreviations..

How to obtain valid informationHow to obtain valid information……

Review questions with experts and potential Review questions with experts and potential respondents.respondents.

Use shorter questions.Use shorter questions. Avoid two-edged questions.Avoid two-edged questions. Avoid negative questions.Avoid negative questions. Adopt/adapt questions used successfully in Adopt/adapt questions used successfully in

other questionnairesother questionnaires

General rules for constructing General rules for constructing good questions in a questionnairegood questions in a questionnaire


- Use short and simple sentences- Use short and simple sentences- Ask for only one piece of information at a time - Ask for only one piece of information at a time - Avoid negatives if possible - Avoid negatives if possible - Ask precise questions - Ask precise questions


- Ensure those you ask have the necessary - Ensure those you ask have the necessary knowledge knowledge - Avoid unnecessary details - Avoid unnecessary details - Avoid Sensitive issues - Avoid Sensitive issues - Minimize bias - Minimize bias

Piloting and evaluation of Piloting and evaluation of questionnairesquestionnaires

- Given the complexity of designing a questionnaire, it is- Given the complexity of designing a questionnaire, it isimpossible even for the experts to get it right the firstimpossible even for the experts to get it right the firsttime round. time round. - Questionnaires must be pretested - that is, piloted - on- Questionnaires must be pretested - that is, piloted - ona small sample of people characteristic of those in the a small sample of people characteristic of those in the survey. survey. - In a small survey, there might be only pretesting of the - In a small survey, there might be only pretesting of the drafted questionnaire. drafted questionnaire.

Why we should "pilot" it Why we should "pilot" it (i.e. test it)(i.e. test it) ? ?

- - To test how long it takes to complete To test how long it takes to complete

- To check that the questions are not ambiguous - To check that the questions are not ambiguous

- To check that the instructions are clear - To check that the instructions are clear

- To allow you to eliminate questions that do not - To allow you to eliminate questions that do not yield usable data yield usable data

Ideal Pilot StudyIdeal Pilot Study

- It should be piloted on a group similar to the - It should be piloted on a group similar to the one that will form the population of your study. one that will form the population of your study.

- It is difficult to give an exact number for the - It is difficult to give an exact number for the pilot group, but as a rule of thumb, try to pilot pilot group, but as a rule of thumb, try to pilot on about on about

5-10% of your final sample number. 5-10% of your final sample number. - The results from the pilot study, however, - The results from the pilot study, however,

should not be included with your final results.should not be included with your final results.

THANK YOUTHANK YOU