research design what? design: a plan to conduct the research, test the hypotheses or answer the...
TRANSCRIPT
RESEARCH DESIGN
WHAT?
• DESIGN: A PLAN TO CONDUCT THE RESEARCH, TEST THE HYPOTHESES OR ANSWER THE RESEARCH QUESTIONS.
RESEARCH DESIGN
WHY?
• RESEARCHER WANTS TO CONTROL THE QUALITY OF THE RESEARCH BY KEEPING CONDITIONS UNIFORM.
• WANTS TO AVOID BIAS AND ENSURE REPLICATION.
RESEARCH DESIGN
HOW?
• QUESTION OFTEN LEADS TO DESIGN.
• SOME QUESTIONS ARE NOT RESEARCH ABLE.
• WHAT KINDS OF VARIABLES DO NOT LEND THEMSELVES TO INTERVENTIONS OR EXPERIMENTS?
RESEARCH DESIGN
OTHER FACTORS LEADING TO
DESIGN SELECTION
1. ACCURACY: ALL ASPECTS OF STUDY SYSTEMATICALLY & LOGICALLY FLOW FROM PROBLEM STATEMENT.
2. FEASIBILITY: CAN PROJECT BE DONE? SUBJECTS, DATA, COSTS, TIME, FACILITY & EQUIPMENT, ETHICS & RESEARCHER EXPERIENCE?
RESEARCH DESIGN
3. CONTROL: HOLD CONDITIONS OF STUDY CONSTANT, ESTABLISH SPECIFIC SAMPLING CRITERIA.
• CONTROL RULES OUT EXTRANEOUS OR MEDIATING VARIABLES: ONES THAT INTERFERE WITH INDEPENDENT VARIABLE, e.g. AGE (MATURATION EFFECT), OTC MEDS IN ANTIBIOTIC STUDY.
RESEARCH DESIGN
HOW TO CONTROL FOR
MEDIATING VARIABLES
1) HOMOGENOUS SAMPLING - GENERALIZE FINDINGS ONLY TO THIS POPULATION
RESEARCH DESIGN
HOW TO CONTROL FOR
MEDIATING VARIABLES
2) USE CONSISTENT DATA COLLECTION TECHNIQUES: COOKBOOK MEASURES, RATER RELIABILITY, EQUIPMENT. ALL CALIBRATED.
RESEARCH DESIGN
HOW TO CONTROL FOR
MEDIATING VARIABLES
3) MANIPULATE INDEPENDENT VARIABLE: CONTROL GROUP HAS NO MANIPULATION, BUT “USUAL CONDITIONS” REMAIN CONSTANT.
RESEARCH DESIGN
HOW TO CONTROL FOR
MEDIATING VARIABLES
• 4) RANDOMIZATION IN SELECTION OF SUBJECTS AND ALLOCATION OF TREATMENT (INDEPENDENT VARIABLE).
RESEARCH DESIGN
NON-EXPERIMENTAL DESIGNS
• DO NOT MANIPULATE INDEPENDENT VARIABLE & DO NOT HAVE EITHER AN EXPERIMENTAL OR CONTROL GROUP.
• USUALLY CALLED THE STUDY GROUP.
• DESIGN HAS TO FIT THE QUESTION.
RESEARCH DESIGN
EXPERIMENTAL DESIGNS
• MANIPULATE THE INDEPENDENT VARIABLE & ALWAYS HAVE A CONTROL AND AN EXPERIMENTAL GROUP.
RESEARCH DESIGN
ORDER OF RIGOUR OF DESIGNS
1. NON-EXPERIMENTAL
A. EXPLORATORY DESIGN
B. CORRELATIONAL DESIGN
2. QUASI-EXPERIMENTAL
3. EXPERIMENTAL
RESEARCH DESIGN
1. NON-EXPERIMENTAL
A. EXPLORATORY DESIGN
• LITTLE LITERATURE EXISTS. RESEARCHER WANTS TO DESCRIBE OR CATEGORIZE THE PHENOMENA. NO CONTROL OF EXTRANEOUS VARIABLES.
RESEARCH DESIGN
1. NON-EXPERIMENTAL
B. CORRELATIONAL DESIGN
• MORE LITERATURE EXISTS. RESEARCHER WANTS TO LOOK AT RELATIONSHIP BETWEEN TWO VARIABLES. SOME CONTROL OF EXTRANEOUS VARIABLES.
RESEARCH DESIGN
2. QUASI-EXPERIMENTAL
• MANIPULATE INDEPENDENT VARIABLE, BUT OFTEN MISSING CONTROL GROUP OR BASELINE MEASUREMENT.
RESEARCH DESIGN
4. EXPERIMENTAL
• MANIPULATE INDEPENDENT VARIABLE. RANDOM SELECTION & ALLOCATION (CONTROL GROUP). BASELINE MEASURES. STRICT CONTROLS.
RESEARCH DESIGN
NONEXPERIMENTAL DESIGNS
• USED TO DESCRIBE PHENOMENA. NO MANIPULATION OF INDEPENDENT VARIABLE. EXPLORE RELATIONSHIPS OF DIFFERENCES. CAN LEAD TO MORE RIGOROUS DESIGNS.
• ALSO REQUIRES CLEAR CONCISE PROBLEM STATEMENT.
RESEARCH DESIGN
1. NON-EXPERIMENTAL
A. DESCRIPTIVE/EXPLORATORY SURVEYS
• INTERESTED IN OPINIONS, ATTITUDES & FACTS. HOW MANY PEOPLE WILL VOTE FOR WHICH PARTY? WHAT SHOULD THE BASIC ENTRY LEVEL PREPARATION OF SOCIAL WORKERS BE?
RESEARCH DESIGN
A. DESCRIPTIVE/EXPLORATORY SURVEYS
METHODS:
INCLUDE EITHER QUESTIONNAIRE (MAILED, COMPUTER) OR INTERVIEW (PHONE, IN-PERSON).
RESEARCH DESIGN
PROS OF DESCRIPTIVE/ EXPLORATORY SURVEYS
• LENDS ITSELF TO RANDOM SELECTION & SO ACCURACY CAN BE MEASURED.
• INEXPENSIVE.• CAN GATHER LOT OF INFORMATION.
RESEARCH DESIGN
CONS OF DESCRIPTIVE EXPLORATORY SURVEYS
• DESCRIPTION OF VARIABLES
• OBTAINS BREATH NOT DEPTH OF INFORMATION. MAY BE EXPENSIVE.
• REQUIRES RESEARCH EXPERTISE IN SAMPLING TECHNIQUES, QUESTIONNAIRE CONSTRUCTION, INTERVIEWING & DATA ANALYSIS.
RESEARCH DESIGN
2. NON-EXPERIMENTAL
B. CROSS-SECTIONAL STUDIES
• COLLECT DATA AT ONLY ONE POINT IN TIME. CAN BE DONE ON MORE THAN ONE GROUP. OFTEN A SURVEY.
RESEARCH DESIGN
PROS OF CROSS-SECTIONAL STUDIES
• LESS EXPENSIVE
• RESULTS READILY AVAILABLE
• NO MATURATION, MORTALITY THREATS.
RESEARCH DESIGN
CONS OF CROSS-SECTIONAL STUDIES
• CAN’T TELL IF INDEPENDENT VARIABLE PRECEDED DEPENDENT VARIABLE.
RESEARCH DESIGN
1. NON-EXPERIMENTAL
C. LONGITUDINAL STUDY
• DATA FROM SAME GROUP IS COLLECTED AT DIFFERENT POINTS IN TIME. FOLLOW COHORT. E.G. FRAMINGHAM STUDY. CANADA POPULATION HEALTH STUDY.
• CAN BE RETROSPECTIVE. USUALLY IS PROSPECTIVE.
RESEARCH DESIGN
PROS OF LONGITUDINAL STUDY
• APPROPRIATE WHEN STUDYING CHRONIC DISEASES OR CONDITIONS WITH LATENT ATTRIBUTES.
• EACH SUBJECT ACTS AS OWN CONTROL
• CAN SPOT EARLY TRENDS.
RESEARCH DESIGN
CONS OF LONGITUDINAL STUDY
• EXPENSIVE
• LOSS TO FOLLOW-UP MORTALITY)
• TESTING EFFECT
RESEARCH DESIGN
1. NON-EXPERIMENTAL
D. CROSS-SECTIONAL STUDIES
• COLLECT DATA AT ONLY ONE POINT IN TIME. CAN BE DONE ON MORE THAN ONE GROUP. OFTEN A SURVEY.
RESEARCH DESIGN
1. NONEXPERIMENTAL DESIGNS
E. CORRELATIONAL DESIGN (EX POST FACTO)
• NOT INTERESTED IN TESTING WHETHER ONE VARIABLE CAUSES ANOTHER OR HOW DIFFERENT ONE VARIABLE IS FROM ANOTHER.
RESEARCH DESIGN
• ASKS: AS ONE VARIABLES VARIES, WHAT HAPPENS TO ANOTHER VARIABLE?
• IN WHAT DIRECTION DOES CHANGE OCCUR? HOW DOES MATERNAL ANXIETY EFFECT CHILDREN’S ANXIETY? DOES MATERNAL ANXIETY RELATE TO OR IN CHILDREN’S ANXIETY?
RESEARCH DESIGN
PROS OF CORRELATION DESIGNS
• FLEXIBILE WHEN INVESTIGATING COMPLEX RELATIONSHIPS.
• EFFICIENT & ALLOWS COLLECTION OF LARGE AMOUNTS OF DATA.
RESEARCH DESIGN
CONS OF CORRELATIONAL STUDIES
• NO MANIPULATION
• NO RANDOM SELECTION, DEALS WITH PRE-EXISTING GROUPS.
• NO CAUSALITY.
RESEARCH DESIGN
1 NON-EXPERIMENTAL
F. COMPARATIVE DESIGN OR CASE CONTROL STUDY
• HYPOTHESIZE ABOUT A RELATIONSHIP & TEST TO SEE IF IT EXISTS. DO NOT RANDOMLY ALLOCATE SUBJECTS TO GROUPS. DO NOT MANIPULATE.
RESEARCH DESIGN
E. COMPARATIVE DESIGN OR CASE CONTROL STUDY
• FIND SUBJECTS WITH DISEASE. MATCH WITH OTHER SUBJECTS WITHOUT DISEASE. COMPARE EXPOSURE BACK IN TIME.
• USUALLY RETROSPECTIVE.
RESEARCH DESIGN
PROS OF CASE CONTROL
• EFFICIENT
• HIGH LEVEL OF CONTROL.
RESEARCH DESIGN
CONS OF CASE CONTROL
• DO NOT RANDOMLY ALLOCATE
• DO NOT MANIPULATE.
• NO CAUSALITY, CAN NOT RULE OUT COMPETING HYPOTHESES.
RESEARCH DESIGN
COMPARATIVE DESIGN OR CASE CONTROL STUDY
• GROUP A O1
• ---------------------------------------------
• GROUP B O1
RESEARCH DESIGN
2. QUASI-EXPERIMENTAL DESIGNS
• STILL INTERESTED IN CAUSE AND EFFECT; HOWEVER BECAUSE THEY ARE LESS RIGOROUS, THEY ARE PRONE TO BIASES OR THREATS TO INTERNAL VALIDITY.
RESEARCH DESIGN
2. QUASI-EXPERIMENTAL DESIGNS
• ALWAYS HAVE MANIPULATION.
• USUALLY LACKS RANDOM ALLOCATION.
• MAY LACK CONTROL GROUP.
RESEARCH DESIGN
2. QUASI-EXPERIMENTAL DESIGNS
A. NON-EQUIVALENT CONTROL GROUP DESIGN
• NONEQUIVALENT BECAUSE GROUPS WERE NOT FORMED BY RANDOM ALLOCATION. HAS MANIPULATION. CAN HAVE PRETEST AND/OR POST-TEST.
RESEARCH DESIGN
• OFTEN USED WITH TWO HOSPITAL WARDS WITH PATIENTS WITH SIMILAR DIAGNOSES.
• ONE DOESN’T ABANDON A DESIGN. ONE ANTICIPATES THE THREATS AND PLANS HOW TO ADDRESS THEM.
RESEARCH DESIGN
• A. NON-EQUIVALENT CONTROL GROUP DESIGN (POST-TEST ONLY )
• GROUP A X1 O1
• -----------------------------
• GROUP B X1 O1
RESEARCH DESIGN
A. NON-EQUIVALENT CONTROL GROUP DESIGN (PRETEST-POSTTEST)
• GROUP A O1 X O2
• -----------------------------
• GROUP B O1 O2
RESEARCH DESIGN
A. NON-EQUIVALENT CONTROL GROUP DESIGN
THREATS
• SELECTION (EQUIVALENCE AT BASELINE),
• MATURATION,
• TESTING
• MORTALITY:
RESEARCH DESIGN
QUASI-EXPERIMENTAL DESIGNS:
3. TIME SERIES
NON EQUIVALENT OR NO GROUPS. MEASURE PRE-TEST SEVERAL TIMES THEN DO MANIPULATION. AND DO SEVERAL POST-TESTS
RESEARCH DESIGN
3. TIME SERIES
GROUP A O1 O2 O3 O4 X O5 O6 O7 O8
RESEARCH DESIGN
3. TIME SERIES
THREATS
• MATURATION EFFECT: DEPENDENT VARIABLE MAY CHANGE NATURALLY OVER TIME.
• MEASUREMENT: INTERVENTION FADE OVER TIME?
• SELECTION.
• MORTALITY.
RESEARCH DESIGN
PROS OF QUASI-EXPERIMENTAL DESIGNS
• PRACTICAL • FEASIBLE: WHEN YOU CAN’T
ETHICALLY ASSIGN PEOPLE TO NOXIOUS INTERVENTION: CANCER.
• GENERALIZABLE
RESEARCH DESIGN
CONS OF QUASI-EXPERIMENTAL DESIGNS
• LACK OF RIGOR
• INCREASED THREATS TO INTERNAL VALIDITY
RESEARCH DESIGN
3. EXPERIMENTAL DESIGNS
OFTEN LAY PEOPLE THINK THAT WHEN A STATISTICAL ASSOCIATION IS FOUND, THAT MEANS CAUSALITY. THERE ARE SIGNIFICANTLY MORE BLUED-EYED MURDERERS THAN BROWN-EYED. ALL BLUE EYED PEOPLE KILL?
RESEARCH DESIGN
3. EXPERIMENTAL DESIGNS CAUSALITY1. STRENGTH OF ASSOCIATION
2. DOSE-RESPONSE RELATIONSHIP
3. CONSISTENCY OF ASOCIATION
4.TEMPORALLY CORRECT ASSOCIATION
5. SPECIFICITY OF ASSOCIATION
6.COHERENCE WITH EXISTING KNOWLEDGE
1
RESEARCH DESIGN
EXPERIMENTAL DESIGN HAS 3 CHACTERISTICS
• 1) RANDOM ASSIGNMENT (major difference between quasi & experimental),
• 2) CONTROL, AND • 3) MANIPULATION OF INDEPENDENT
VARIABLE.
RESEARCH DESIGN• 3. EXPERIMENTAL DESIGNS• i . RANDOM ASSIGNMENT: SUBJECTS
ASSIGNED TO GROUP PURELY ON BASIS OF CHANCE.
• EACH SUBJECT HAS EQUAL OR KNOWN PROBABILITY OF BEING ASSIGNED TO ANY GROUP.
• ESSENTIAL TO ELIMINATE SYSTEMATIC BIAS THAT MAY EFFECT DEPENDENT VARIABLE.
RESEARCH DESIGN
i . RANDOM ASSIGNMENT: HELPS TO ASSURE THAT INTERVENING VARIABLES ARE EQUALLY DISTRIBUTED AMONG GROUPS, E.G. AGE, PROGNOSIS.
• TO ENSURE THAT RANDOMIZATION HAS OCCURED, YOU ANALYZE CHARACTERISTICS OF TWO GROUPS AND REPORT THE FINDINGS.
RESEARCH DESIGN
3. EXPERIMENTAL DESIGNS
i . RANDOM ASSIGNMENT
HOW IS RANDOM SELECTION DIFFERENT FROM RANDOM ASSIGNMENT?
RESEARCH DESIGN
3. EXPERIMENTAL DESIGNS
ii. CONTROL
MANIPULATING INDEPENDENT VARIABLE, ADHERING TO PROTOCOLS, AND USING COMPARISON GROUPS.
RESEARCH DESIGN
3. EXPERIMENTAL DESIGNS
iii. MANIPULATION
INDEPENDENT VARIABLE IS THE MANIPULATION. IN HEALTH SCIENCES THIS IS OFTEN AN EDUCATIONAL PROGRAM.
RESEARCH DESIGN
• ANTECEDENT VARIABLE: OCCURS BEFORE STUDY BUT MAY EFFECT DEPENDENT VARIABLE, E.G. SES, GENDER, RACE.
• INTERVENING VARIABLE: OCCURS DURING STUDY & MIGHT EFFECT DEPENDENT VARIABLE. E.G. PROVINCIAL PSYCHIATRIC HOSPITAL CLOSURES.
RESEARCH DESIGN
1. TRUE EXPERIMENT (RANDOMIZED CLINICAL/CONTROL TRIAL RCT).
• SUBJECTS RANDOMLY ASSIGNED TO EXPERIMENTAL/CONTROL GROUP. MAY HAVE PRE-TEST , BUT DEFINITELY HAVE POST-TEST MEASURES.
• WHY A CONTROL GROUP? HOW DO YOU KNOW GROUPS ARE THE SAME?
RESEARCH DESIGN
RANDOMIZED CLINICAL TRIAL
• WHY HAVE A PRE-TEST?
• WHY HAVE A POST-TEST?
• WHY RANDOMLY ASSIGN?
• HOW BIG A PROBLEM IS IT IF PATIENTS ARE NOT RANDOMLY SELECTED?
RESEARCH DESIGN
3. EXPERIMENTAL DESIGNS
ii. SOLOMON FOUR GROUP DESIGN
• BECAUSE YOU WANT TO PRE-TEST & POST-TEST WITH SAME INSTRUMENT IN SHORT PERIOD OF TIME, TESTING EFFECTS ARE AN INTERNAL VALIDITY THREAT. TO OVERCOME THIS, USE A SOLOMON FOUR GROUP DESIGN.
RESEARCH DESIGN
3. EXPERIMENTAL DESIGNS
ii. SOLOMON FOUR GROUP DESIGN
• RANDOMLY ASSIGN TO 4 GROUPS, ONLY GIVE PRE-TEST TO TWO GROUPS, GIVE POST-TEST TO ALL. WHAT DOES THIS ALLOW YOU TO SAY?
RESEARCH DESIGN
3. EXPERIMENTAL DESIGNS
ii. SOLOMON FOUR GROUP DESIGN
GROUP A R O1 X O2
GROUP B R O1 X O2
GROUP C R X O2
GROUP D R X O2
RESEARCH DESIGN
• 3. POSTTEST ONLY CONTROL GROUP DESIGN:
• NO PRE-TEST.
• NEED TO ENSURE THAT GROUPS ARE SIMILAR.
RESEARCH DESIGN
PROS OF EXPERIMENTAL DESIGNS
TEST CAUSE-EFFECT.
RESEARCH DESIGN
CONS OF EXPERIMENTAL DESIGN
• NEED TO KNOW WHICH VARIABLES MAY BE IMPORTANT IN PREDICTING OUTCOMES (DEPENDENT VARIABLE). HEALTH SCIENCES OFTEN DON’T; KNOWLEDGE IS AT EARLIER STAGE OF DEVELOPMENT.
RESEARCH DESIGN
• IN HEALTH SCIENCES, SOME INTERVENTIONS DO NOT ETHICALLY LEND THEMSELVES TO RANDOMIZATION. CAN’T FORCED PEOPLE TO PARTICIPATE (RANDOM SELECTION). WHAT WOULD BE EXAMPLES OF THIS?
RESEARCH DESIGN
NONEQUIVALENT CONTROL GROUP DESIGN (POST-TEST ONLY )
• GROUP A X O
• ---------------------------------
• GROUP B X 0