Data Collection

Prof. Dr. Ali ŞEN

Objectives

After completion of this session the students should be able to:

• Identify major types of data collection methods • Perform a data collection plan • Compare different type of Self-Reports• Discuss observational methods• Differentiate between validity and reliability

OutlinesPart 1 :Measurement & data collection

• Major types of data collection methods • Developing a data collection plan • Implementing a data collection plan • Research examples

Cont. Part 11 Self-Reports

• Unstructured & Semi-Structured Instruments • Questionnaires Versus interviews: • An assessment • Response Biases

cont.Part 3 :Observational methods

• Observational methods: Unstructured observations

Structured observations

Cont, :Part 4. Assessing data quality

• Measurement • Reliability of measuring instruments • Validity • Other criteria for assessing quantitative

measure • Assessment of qualitative data

Introduction

• Data collection means gathering information to address those critical evaluation questions that you have identified earlier in the evaluation process.

• To plan data collection, you must think about the questions to be answered and the information sources available.

Cont,

• you must begin to think ahead about how the• information could be organized, analyzed,

interpreted and then reported to various audiences

• There are many methods available to gather information, and a wide variety of information sources.

Pre-Data Collection Steps

1. Clearly define the goals and objectives of the data collection

2. Reach understanding and agreement on operational definitions and methodology for

the data collection plan3. Ensure data collection (and measurement) repeatability,

reproducibility, accuracy,and stability

What kind of data should be collected?

• The information you collect is the evidence you will have available to answer the evaluation questions.

• Poor evidence is information which cannot be trusted, is limited, or simply is not relevant to the questions asked.

• Good evidence is information that comes from reliable sources

and through trustworthy methods that address important questions

There are two general types of information:

• descriptive and judgmental.Descriptive information

• can include the following examples:• Characteristics of the project• Reports of project accomplishments• Current skill or knowledge levels of project personneland the target audience• Amount of participation by the target audience• Policies concerning cost share• Rules regarding research • Types of participants• Demographic data

Judgmental information

• Includes the following examples:• Opinions from experts or consultants• Consumer preferences• Target audience’s beliefs and values• Technical agency personnel’s interpretation of laws• Stakeholders perceived priorities• Stakeholders interpretation of guidelines

What methods should be used to collect data?

The selection of a method for collecting information must balance several concerns including

• Resources available• Credibility• Analysis and reporting• Resources• And the skill of the evaluator

Major types of data collection methods

• Three types of approach have been used most frequently by nurse researcher:

• Self-reports • Observation • Biophysiologic

1. Self-reports

• Self-report data can be gathered either by oral interview or by written questionnaire

• The self-report method is strong with respect to its directness & flexibility *If we want to know what people think, feel, or believe (qualitative data) the most direct means of gathering the information is to ask them about it.

Disadvantage of self-report

• Verbal report instruments share a number of weaknesses. The most serious issue is the question of the validity & accuracy of self-report:

• How can we really be sure that respondents feel or act the way they say they do?

• How can we trust the information that respondents provide, particularly if the questions could potentially require them to reveal an unpopular position on a controversial issues?

Developing a data collection plan

• Identify types of data needed for the study• Select the types of measures to measure each variable• Select and/or develop instruments• Secure written permission to use each instrument • Pilot test researcher-developed instrument & revise plan• Develop data collection forms and procedures• Implement data collection plan

1-Identify types of data needed for the study

1-Testing hypothesis or answering research questions 2-Describe characteristics of sample Demographics - age, gender, ethnic origin, education

background, marital status Health-related variables - health habits, diet, exercise, illness,

length of illness, 3- Control for extraneous variables Measure as many as possible Intrinsic and extrinsic factors (variables) May want to see if main effects also apply to

Cont. 4- Analyze data 5-Interpretation of results Consider results that do not fit desired outcomes Identify what data might explain undesirable Obtain administrative data Records of subject ID=s Data on number of times subjects recruited or approached,

dates of data collection, length of time for data collection, location, time of

day, and who collected data Who, what, when, where, and how of data collection

2 Select the types of measures to measure each variable

• Each variable must be measured - quantified• Highly structured measurement takes time and testing to

develop, yet gives data easier to analyze both descriptively and inferentially

• Objective measurement of phenomenon • Better to use multiple methods to measure variables - self-

reports, observations, and physiologic measures• Need to consider ethics, cost, time, staff, and stress on

pt./family &/or agency personnel

3-Select and/or develop instruments Identify existing instruments Fit with conceptual definition of variable Quality of instrument - validity & reliabilityValidity:

Will the information collection methods you have designed produce information that measures what you say you are measuring? Be sure that the information you collect is relevant to the evaluation questions you are intending to answer.

Reliability:

Will the evaluation process you have designed consistently measure what you want it to measure? If you use multiple interviews, settings, or observers, will they consistently measure

the same thing each time? If you design an instrument, will people interpret your questions the same way each time?

• Resources - costs• Instrument use & scoring• Data collectors salary• Subject compensaton• Availability & familiarity• Researcher expertise• Equipment• Norms - comparability• Established norms for instrument - provide comparison group• Replication - use same instruments• Populations appropriateness• Reading level & writing ability• Cultural , ethnic origin• Gender biased• Translations for non-English speaking subjects

Cont.

4 Secure written permission to use each instrument

• Look for employer and write to author at

place of employment• Find most recent publication to identify

current employer• Request a copy of the instrument and

information on scoring, procedures, validity, and reliability

Develop instrument, pilot test researcher-developed instrument

• Revise it plan Instrument based on theoretical framework of study

• Pilot test on small scale & evaluate before administration large group

• Determine amount of time to complete instrument

Areas problematic for subjects

• Difficult to understand & reading level too high• Offensive terminology• Sequencing of questions• Training of data collectors• How well instrument discriminates among subjects

- detects differences in subjects• Revise instrument based on pilot

6 Develop data collection forms and procedures

• Forms• Screening potential subjects• Consent & assent forms• Explanations to potential subjects for people referring

subjects• Advertisements to recruit subjects• Records for tracking contacts with subjects• Mailing lists and logs for receipts•

• Procedures• Specific conditions for data collection• Specific procedures and sequencing for

experiments• Standard information for subject's questions• Procedures for risks if they occur• List of all materials needed• Interview guidelines, instruments, observation

directions

Cont.

7 Implement data collection plan

• Select who will collect data• Researcher or neutral agent• Staff• Experience• Background similar to subject• Unremarkable appearance - dress, make-up, jewelry• Personality - pleasant, sociable, non-judgmental,

non-threatening• Available to collect data for the entire study period

Training data collectors

• Includes general principles of data collection & specifics for study

• Data collection training manual• Review manual, forms, and procedures • Demonstrate data collection and return

demonstration with videotape• Periodic observation of data collection to verify

procedures are followed•

Self-reports• Approaches to collect self report data• Unstructured• Semi- Unstructured• Self report techniques • Interviews • verbal communication between research and subject;

commonly used in exploratory and descriptive studies Unstructured interview• 1- subject's world view; open-ended questions with

probes and prompting; qualitative; audiotape

• Semi-structured interview - • 1 subject; list of topics or questions for discussion with

additional probes, aka topic guide; uses both open• and close-ended questions; taped • Structured interview –• 1 subject; specific questions asked in• Consistent order using the same words each time;

no variation from questions and no explanation of unclear questions

Focus group interview • 5 to 15 subjects in a group;• Interviewer/moderator asks open-ended

questions; efficient yet some individuals inhibited by others in the group

• Life history - anthropologic; chronologic changes; sequence of events

• Critical incident - subject describes an event (incident) from his viewpoint r/t research topic

• Diary - subject's daily log describing own experiences r/t the research topic

• Instruments - interview schedule with questions and space to record answers; tape recordings of face-to-face or telephone interviews

• Interviewer training - pilot testing; familiar with content and situations encountered with interviewing; practice maintaining unbiased verbal and nonverbal communication

• Gathering unstructured self report data • good listener good questioner; make subject

relaxed and open; maintain focus on the topic; summarize and close on a positive note

• Evaluate unstructured approach data• Explore new topics and experiences; time

consuming and demanding

Structured self report instrument Types of closed-ended questions 1- Dichotomous items – 2 opposing choices; yes/no or true/false 2- Multiple choice -- >2 choices from which subject

selects 1 option 3- Rank-order - subject places options in rank or

order of importance based on tool directions 4-Forced-choice - 2 polar alternative statements;

similar to dichotomous, but in the form of sentences rather than singl words

5- Rating - Likert scale; semantic differential scale• Psychological tests• Personality inventories – mmpi, Edwards

personal preference• Projective techniques – Rorschach inkblot

test; Thematic Apperception Test (tell a story based on a picture)

•

• Administering self-report instruments• Interviews• Put subject at ease• Punctual, friendly, courteous, respectful• Privacy – obtain written consent• Check tape recorder function before each interview; bring

extra batteries and tapes• Remain neutral• Use interview guide for consistency

Advantages of interview

• Better response rates• Wider range of subjects • Can clarify questions • Greater depth of questions • Decreases missing information • Order of questions controlled • Adds supplementary observational data

Questionnaires

• Group or individuals – easy with group• Personal contact with subjects or mailed• Want response rate of at least 60% to avoid response bias• Cover letter stating completion and return of questionnaire

indicates• consent of subject• Can drop off and pick up in person or mail• Mailing with follow-up plan – questionnaire, postcard,

questionnaire,• postcard• Advantages - Cheaper, anonymity, no interviewer bias

• Response biases of subjects• Social desirability – chooses answer most

socially acceptable• Response set – items influence the subjects

response to other items• Extreme responses – selects response on

either extreme end of poles

• Acquiescence response • yeah-sayers – always agrees with statement; use

both positive and negative statements to counterbalance this response

• Nay-Sayers – always disagrees with statement; use both positive and negative statements to

counterbalance this response

Tips for wording the question

• Clearly stated• Bias decreased• Sensitive information – respect subject• Worded so that subjects can respond to

questions• Short sentences – ask only 1 question at a

time; positive statements

Cont.• Avoid leading questions; focus on research

objectives and questions• Use closed-ended questions with socially

unacceptable topics• Do not assume subjects are well informed about

topic• Impersonal wording of items without "I"• Mutually exclusive item choices

Observational methodsGathering data through visual, auditory, tactile and other senses• Phenomenon• Characteristics of individuals• Verbal & nonverbal communication behavior• Activities• Skill attainment and performance• Environmental characteristics• Steps in observation• Decide what to observe• Determine how to conduct observation to ensure every variable observed

in same manner Orient and provide written instructions to observers• Validate accuracy and consistency of observation techniques

Observational Methods • Unstructured observations • No planning – spontaneous• Observer freedom• Risk loss of objectivity• Requires excellent memory & interpersonal skills• Structured observations• Define what is to be observed• Define how observations will be made, recorded, and

coded• Recording structured observations•

• Participant observations – observer & subject interact to some degree duration observation

• Overt – subject aware of observation• Covert – subject not aware of observation• Non-participant observations – observer and subject do not interact; can be overt or covert• Timing of observations• Time sampling – making an observation for a specified

length of time at pre-determined intervals (eg observe for 5 minutes every 15 minutes or observe for 5 minutes 3 randomly selected times each hour)

• • Event sampling – making an observation whenever an event

occurs

111-Biophysiologic Methods

PurposesBasic physiology with relevance for nursing careWays that nursing actions or medical interventions affect patient

health outcomesEvaluation of specific nursing procedures or interventions testing

a hypothesisImproving measurement and recording of biophysiologic data

collected by RNCorrelation of physiologic function in patient with health

problems

Major types

• In vivo

• Measurements performed directly with in or on living organisms themselves

• May use complex instrumentation system with computers• May be simple – thermometer, pulse oximeter, stethoscope• In vitro

• Measurements performed outside the organism’s body Specimens collected and tested outside body

• Blood chemistries, microbiologic, cytologic specimens …………………………….SCAN

Considerations for physiologic measurements

Will measure yield good information? Does it fit researchquestions/hypothesis and variables of study?What other methods of measurement could be used?

Invasive &noninvasiveEquipment and supply costs; reliability of equipment and

complexity of operationTraining of personnel

Thank You!

Prof. Dr. Ali ŞEN