copy of developing instrument data collection (1) 2 mei
TRANSCRIPT
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
1/36
Developing Instrument Data
Collection (1)
Kumboyono
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
2/36
Choosing an instrument
Appropriateness
Validity
Responsiveness
Precision
Interpretability
Acceptability
Reliability
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
3/36
Choosing an instrument
Appropriateness
Is the content of the instrument appropriate to thequestions which the study is intended to address?
Reliability
Does the instrument produce results that are reproducibleand internally consistent?
Validity
Does the instrument measure what it claims to measure?
Responsiveness
Does the instrument detect changes over time that matterto patients?
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
4/36
Choosing an instrument (2)
Precision
How precise are the scores of the instrument?
Interpretability
How interpretable are the scores of the instrument? Acceptability
Is the instrument acceptable to patients?
Feasibility
Is the instrument easy to administer and process?
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
5/36
Overview of Lecture
What are questionnaires? Why use them?
Ideal properties of a questionnaire
Planning a questionnaire (general principles)
(Choosing/adapting a data collection method, Questions and response categories, Use
of existing data collection instruments)
Questionnaire completion issues
Piloting
Conclusions
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
6/36
What are questionnaires?Why use them?
Why use them? Measures of outcome (i.e. in a trial)
Audit/Quality assurance (patients and clinicians)
Assessment of health care needs/ preferences
Assessment of individual patient care/satisfaction with that care
List goes on!
A Quantitativesurvey method
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
7/36
Planning a questionnaire
What to ask?
Who to ask? When to ask?
How to ask?
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
8/36
Questionnaire design steps
1. Literature Review and initial planning
2. Choose the most appropriate method of data collection (e.g.self-completed questionnaires vs. interviews)
3. Design or adapt data collection instruments (Layout, Presentation,Types of instruments)
4. Conduct pilot work and refine methods and instruments
5. Design and select sample
6. Conduct data collection (often termed field work)
7. Process data8. Analyse data
9. Report findings
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
9/36
Initial Planning & Literature Review
So where to start?
List topics of interest - If research question is already known!
(why ask?)
Review existing literature on your topic, collate existing
questions and scales that can be used (What worked? What
didnt? Dont re-invent the wheel!)
First opportunity to think about - timing, frequency, content,
wording, layout & administration mode
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
10/36
Choosing the data collection
method (administration methods) Self-completion
Self completion postal survey (retrospectively)
Self completion concurrent survey
Computer-assisted (internet based?) Paper based
Interviewer administration
Face-to-face (paper/computer assisted) Telephone interviews
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
11/36
Advantages/ Disadvantages
Postal Administration ?
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
12/36
Advantages/ Disadvantages
Postal
Advs:
Cheap
Easy to administer (large volume of participants can be accessed
easily) Privacy for sensitive/ stigmatised questions to be explored (more or
less likely to divulge without someone present?)?
Participants can verify responses over time (checking)
Dis: Missing data?
Low response rates?
Misunderstanding of questions (meaning and flow)?
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
13/36
Design/adapt data collection instruments(Questionnaire layout)
Introduction
Anonymity & confidentiality
Deadlines
Feedback (potentially)
Instructions
Return envelope (reply paid)
Personalisation
Covering
letter?
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
14/36
Design/adapt data collection instruments(Presentation issues)
Length
Shorter more likely to be completed
Dont need to know, dont ask!
Visual Impact
Paper colour & quality
Font size
Layout
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
15/36
General principles of questionnaire design
(1)
Ensure all questions and sub questions are clearly labelled
(1,2a,2b,3,4a, etc)
Start with easier (not too sensitive initially!)
Use a logical order but if a long questionnaire start with most
important first
Put demographic questions at the end
Ask about behaviour before attitudes
Ask about past behaviour before current behaviour
Ask general questions first, then more specific
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
16/36
General principles of questionnaire design
(2)
Use funnelling-(start broad then narrowdown)
Use filtering
Use familiar words (avoid technical jargon)
Avoid vague terms/abbreviations
Avoid ambiguity
Keep questions short (if possible)
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
17/36
General principles of questionnaire design
(3)
Avoid double negatives
Avoid manipulating responses (by conveying
an assumption)
Avoid leading questions
Use hypothetical questions carefully
Avoid bias (mention more later on this)
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
18/36
Types of questions
Behavioural
Factual
Knowledge
Attitude
Primary consideration: Response formats
2 basic kinds - Open Response
- Closed Response
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
19/36
Open and closed question responses
Open are just that, left open
- e.g. How do you feel about your health today?
Closed questions have pre-coded answers
- Do you feel well today? Yes No
- Today my health is excellent:(Tick)
Agree Neither agree or disagree Disagree
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
20/36
Open format
Open questions useful when.
replies unknown
too many/complex to pre-code
good for search for explanations to answers
Open questions are NOT useful when
need to be coded somehow possibility of
bias introduced difficult to analyse and generalise
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
21/36
Closed format
Closed format are useful because.
Easy and quick to fill in
Easy to code, record and analyse
Closed Format are not useful because..
need cover all possible responses(comprehensive)
usually mutually exclusive no room ambiguity
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
22/36
Examples of response categories
Yes/no
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
23/36
Examples of response categories
Scales (1-10)
Height/ weight etc
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
24/36
Examples of response categories
Frequency, duration, distress
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
25/36
Format of response categories
Circling or tick boxes
Skip & branch
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
26/36
Design/adapt data collection
instruments (Types of instrument)
Can use data collection instruments (with already defined
response categories) and just adapt them for your work.
Existing data collection instruments tend to be specific for a
particular health related topic.
For e.g.
- disease specific
- site or region (of the body!) specific
- dimension specific- generic
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
27/36
Design/adapt data collection
instruments (Types of instrument)
Disease specific
The Gastrointestinal Symptom Rating Scale (GSRS)
The Arthritis Impact Measurement Scales (AIMS)
Hospital Anxiety & Depression Scale (HADS)
(overleaf)
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
28/36
HADS
(Hospital Anxiety &Depression Scale)
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
29/36
HADS
(Hospital Anxiety &Depression Scale)
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
30/36
Visual
analogue scale(EQ5d)
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
31/36
The ICIQ -Internal
Consultation onIncontinence
Modular
Questionnaire
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
32/36
Questionnaire completion issues
Under-reporting
Over-reporting (exaggeration)
Social desirability bias
Observer bias (in interview situation only ?)
Memory errors (time frame consideration? Too soon,
cloud judgement? Too long forgotten?) Ill-considered responses (possibly inaccurate on
reflection?)
Skewed/misleadingresponses
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
33/36
Piloting and Evaluation
A pilot project - Serving as a tentative model for future experiment or development(serving or leading as guide)
(Ref: Teijlingen Van E et al 2001)
Piloting1. Development
& testing ofresearch
instruments
2. Assement ofthe feasibility of
a full scalesurvey
3. Identification oflogistical problems
with the questionnaire
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
34/36
Pilot and evaluate
Prepare first draft and pre-test
Distribute questionnaire to small sample
Evaluate responses & redraft
Points to check: (adapted from Bowling 2002)
The questions measure what is intended
The wording is understood by all and mean a similar thing to all
Instructions are easily understood/followed
Do the responses suggest all relevant issues have been covered? Any questions systematically/frequently missed etc
Basically test for bugs
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
35/36
Conclusions
When it comes to questionnaire design
No universal recommendations, just issues
to be aware of, consider and balance out Take into account aims of study, population
and resources available
Principal objective is to collect data that isRELIABLE, VALID & UNBIASED
-
8/22/2019 Copy of Developing Instrument Data Collection (1) 2 Mei
36/36
References
Oppenheim AN. Questionnaire design, interviewing and attitudemeasurement. 2nd ed. London: Pinter; 1992.
McColl et al (2001). Design and use of questionnaires: a review of bestpractice applicable to surveys of health service staff and patients.
HTA: 5 (31). Bowling A. Research methods in health: Investigating health and
health services, 2nd edition. Open University Press, 2002.
Teijlingen Van E, Rennie AM, Hundley V, Graham W, The importance ofconducting and reporting pilot studies: the example of the ScottishBirths Survey. Journal of Advanced Nursing 2001; 343; 289-295
www.natcen.ac.uk
www.statistics.gov.uk
http://www.natcen.ac.uk/http://www.statistics.gov.uk/http://www.statistics.gov.uk/http://www.natcen.ac.uk/