research design what? design: a plan to conduct the research, test the hypotheses or answer the...

RESEARCH DESIGN

WHAT?

• DESIGN: A PLAN TO CONDUCT THE RESEARCH, TEST THE HYPOTHESES OR ANSWER THE RESEARCH QUESTIONS.

RESEARCH DESIGN

WHY?

• RESEARCHER WANTS TO CONTROL THE QUALITY OF THE RESEARCH BY KEEPING CONDITIONS UNIFORM.

• WANTS TO AVOID BIAS AND ENSURE REPLICATION.

RESEARCH DESIGN

HOW?

• QUESTION OFTEN LEADS TO DESIGN.

• SOME QUESTIONS ARE NOT RESEARCH ABLE.

• WHAT KINDS OF VARIABLES DO NOT LEND THEMSELVES TO INTERVENTIONS OR EXPERIMENTS?

RESEARCH DESIGN

OTHER FACTORS LEADING TO

DESIGN SELECTION

1. ACCURACY: ALL ASPECTS OF STUDY SYSTEMATICALLY & LOGICALLY FLOW FROM PROBLEM STATEMENT.

2. FEASIBILITY: CAN PROJECT BE DONE? SUBJECTS, DATA, COSTS, TIME, FACILITY & EQUIPMENT, ETHICS & RESEARCHER EXPERIENCE?

RESEARCH DESIGN

3. CONTROL: HOLD CONDITIONS OF STUDY CONSTANT, ESTABLISH SPECIFIC SAMPLING CRITERIA.

• CONTROL RULES OUT EXTRANEOUS OR MEDIATING VARIABLES: ONES THAT INTERFERE WITH INDEPENDENT VARIABLE, e.g. AGE (MATURATION EFFECT), OTC MEDS IN ANTIBIOTIC STUDY.

RESEARCH DESIGN

HOW TO CONTROL FOR

MEDIATING VARIABLES

1) HOMOGENOUS SAMPLING - GENERALIZE FINDINGS ONLY TO THIS POPULATION

RESEARCH DESIGN

HOW TO CONTROL FOR

MEDIATING VARIABLES

2) USE CONSISTENT DATA COLLECTION TECHNIQUES: COOKBOOK MEASURES, RATER RELIABILITY, EQUIPMENT. ALL CALIBRATED.

RESEARCH DESIGN

HOW TO CONTROL FOR

MEDIATING VARIABLES

3) MANIPULATE INDEPENDENT VARIABLE: CONTROL GROUP HAS NO MANIPULATION, BUT “USUAL CONDITIONS” REMAIN CONSTANT.

RESEARCH DESIGN

HOW TO CONTROL FOR

MEDIATING VARIABLES

• 4) RANDOMIZATION IN SELECTION OF SUBJECTS AND ALLOCATION OF TREATMENT (INDEPENDENT VARIABLE).

RESEARCH DESIGN

NON-EXPERIMENTAL DESIGNS

• DO NOT MANIPULATE INDEPENDENT VARIABLE & DO NOT HAVE EITHER AN EXPERIMENTAL OR CONTROL GROUP.

• USUALLY CALLED THE STUDY GROUP.

• DESIGN HAS TO FIT THE QUESTION.

RESEARCH DESIGN

EXPERIMENTAL DESIGNS

• MANIPULATE THE INDEPENDENT VARIABLE & ALWAYS HAVE A CONTROL AND AN EXPERIMENTAL GROUP.

RESEARCH DESIGN

ORDER OF RIGOUR OF DESIGNS

1. NON-EXPERIMENTAL

A. EXPLORATORY DESIGN

B. CORRELATIONAL DESIGN

2. QUASI-EXPERIMENTAL

3. EXPERIMENTAL

RESEARCH DESIGN

1. NON-EXPERIMENTAL

A. EXPLORATORY DESIGN

• LITTLE LITERATURE EXISTS. RESEARCHER WANTS TO DESCRIBE OR CATEGORIZE THE PHENOMENA. NO CONTROL OF EXTRANEOUS VARIABLES.

RESEARCH DESIGN

1. NON-EXPERIMENTAL

B. CORRELATIONAL DESIGN

• MORE LITERATURE EXISTS. RESEARCHER WANTS TO LOOK AT RELATIONSHIP BETWEEN TWO VARIABLES. SOME CONTROL OF EXTRANEOUS VARIABLES.

RESEARCH DESIGN

2. QUASI-EXPERIMENTAL

• MANIPULATE INDEPENDENT VARIABLE, BUT OFTEN MISSING CONTROL GROUP OR BASELINE MEASUREMENT.

RESEARCH DESIGN

4. EXPERIMENTAL

• MANIPULATE INDEPENDENT VARIABLE. RANDOM SELECTION & ALLOCATION (CONTROL GROUP). BASELINE MEASURES. STRICT CONTROLS.

RESEARCH DESIGN

NONEXPERIMENTAL DESIGNS

• USED TO DESCRIBE PHENOMENA. NO MANIPULATION OF INDEPENDENT VARIABLE. EXPLORE RELATIONSHIPS OF DIFFERENCES. CAN LEAD TO MORE RIGOROUS DESIGNS.

• ALSO REQUIRES CLEAR CONCISE PROBLEM STATEMENT.

RESEARCH DESIGN

1. NON-EXPERIMENTAL

A. DESCRIPTIVE/EXPLORATORY SURVEYS

• INTERESTED IN OPINIONS, ATTITUDES & FACTS. HOW MANY PEOPLE WILL VOTE FOR WHICH PARTY? WHAT SHOULD THE BASIC ENTRY LEVEL PREPARATION OF SOCIAL WORKERS BE?

RESEARCH DESIGN

A. DESCRIPTIVE/EXPLORATORY SURVEYS

METHODS:

INCLUDE EITHER QUESTIONNAIRE (MAILED, COMPUTER) OR INTERVIEW (PHONE, IN-PERSON).

RESEARCH DESIGN

PROS OF DESCRIPTIVE/ EXPLORATORY SURVEYS

• LENDS ITSELF TO RANDOM SELECTION & SO ACCURACY CAN BE MEASURED.

• INEXPENSIVE.• CAN GATHER LOT OF INFORMATION.

RESEARCH DESIGN

CONS OF DESCRIPTIVE EXPLORATORY SURVEYS

• DESCRIPTION OF VARIABLES

• OBTAINS BREATH NOT DEPTH OF INFORMATION. MAY BE EXPENSIVE.

• REQUIRES RESEARCH EXPERTISE IN SAMPLING TECHNIQUES, QUESTIONNAIRE CONSTRUCTION, INTERVIEWING & DATA ANALYSIS.

RESEARCH DESIGN

2. NON-EXPERIMENTAL

B. CROSS-SECTIONAL STUDIES

• COLLECT DATA AT ONLY ONE POINT IN TIME. CAN BE DONE ON MORE THAN ONE GROUP. OFTEN A SURVEY.

RESEARCH DESIGN

PROS OF CROSS-SECTIONAL STUDIES

• LESS EXPENSIVE

• RESULTS READILY AVAILABLE

• NO MATURATION, MORTALITY THREATS.

RESEARCH DESIGN

CONS OF CROSS-SECTIONAL STUDIES

• CAN’T TELL IF INDEPENDENT VARIABLE PRECEDED DEPENDENT VARIABLE.

RESEARCH DESIGN

1. NON-EXPERIMENTAL

C. LONGITUDINAL STUDY

• DATA FROM SAME GROUP IS COLLECTED AT DIFFERENT POINTS IN TIME. FOLLOW COHORT. E.G. FRAMINGHAM STUDY. CANADA POPULATION HEALTH STUDY.

• CAN BE RETROSPECTIVE. USUALLY IS PROSPECTIVE.

RESEARCH DESIGN

PROS OF LONGITUDINAL STUDY

• APPROPRIATE WHEN STUDYING CHRONIC DISEASES OR CONDITIONS WITH LATENT ATTRIBUTES.

• EACH SUBJECT ACTS AS OWN CONTROL

• CAN SPOT EARLY TRENDS.

RESEARCH DESIGN

CONS OF LONGITUDINAL STUDY

• EXPENSIVE

• LOSS TO FOLLOW-UP MORTALITY)

• TESTING EFFECT

RESEARCH DESIGN

1. NON-EXPERIMENTAL

D. CROSS-SECTIONAL STUDIES

• COLLECT DATA AT ONLY ONE POINT IN TIME. CAN BE DONE ON MORE THAN ONE GROUP. OFTEN A SURVEY.

RESEARCH DESIGN

1. NONEXPERIMENTAL DESIGNS

E. CORRELATIONAL DESIGN (EX POST FACTO)

• NOT INTERESTED IN TESTING WHETHER ONE VARIABLE CAUSES ANOTHER OR HOW DIFFERENT ONE VARIABLE IS FROM ANOTHER.

RESEARCH DESIGN

• ASKS: AS ONE VARIABLES VARIES, WHAT HAPPENS TO ANOTHER VARIABLE?

• IN WHAT DIRECTION DOES CHANGE OCCUR? HOW DOES MATERNAL ANXIETY EFFECT CHILDREN’S ANXIETY? DOES MATERNAL ANXIETY RELATE TO OR IN CHILDREN’S ANXIETY?

RESEARCH DESIGN

PROS OF CORRELATION DESIGNS

• FLEXIBILE WHEN INVESTIGATING COMPLEX RELATIONSHIPS.

• EFFICIENT & ALLOWS COLLECTION OF LARGE AMOUNTS OF DATA.

RESEARCH DESIGN

CONS OF CORRELATIONAL STUDIES

• NO MANIPULATION

• NO RANDOM SELECTION, DEALS WITH PRE-EXISTING GROUPS.

• NO CAUSALITY.

RESEARCH DESIGN

1 NON-EXPERIMENTAL

F. COMPARATIVE DESIGN OR CASE CONTROL STUDY

• HYPOTHESIZE ABOUT A RELATIONSHIP & TEST TO SEE IF IT EXISTS. DO NOT RANDOMLY ALLOCATE SUBJECTS TO GROUPS. DO NOT MANIPULATE.

RESEARCH DESIGN

E. COMPARATIVE DESIGN OR CASE CONTROL STUDY

• FIND SUBJECTS WITH DISEASE. MATCH WITH OTHER SUBJECTS WITHOUT DISEASE. COMPARE EXPOSURE BACK IN TIME.

• USUALLY RETROSPECTIVE.

RESEARCH DESIGN

PROS OF CASE CONTROL

• EFFICIENT

• HIGH LEVEL OF CONTROL.

RESEARCH DESIGN

CONS OF CASE CONTROL

• DO NOT RANDOMLY ALLOCATE

• DO NOT MANIPULATE.

• NO CAUSALITY, CAN NOT RULE OUT COMPETING HYPOTHESES.

RESEARCH DESIGN

COMPARATIVE DESIGN OR CASE CONTROL STUDY

• GROUP A O1

• ---------------------------------------------

• GROUP B O1

RESEARCH DESIGN

2. QUASI-EXPERIMENTAL DESIGNS

• STILL INTERESTED IN CAUSE AND EFFECT; HOWEVER BECAUSE THEY ARE LESS RIGOROUS, THEY ARE PRONE TO BIASES OR THREATS TO INTERNAL VALIDITY.

RESEARCH DESIGN

2. QUASI-EXPERIMENTAL DESIGNS

• ALWAYS HAVE MANIPULATION.

• USUALLY LACKS RANDOM ALLOCATION.

• MAY LACK CONTROL GROUP.

RESEARCH DESIGN

2. QUASI-EXPERIMENTAL DESIGNS

A. NON-EQUIVALENT CONTROL GROUP DESIGN

• NONEQUIVALENT BECAUSE GROUPS WERE NOT FORMED BY RANDOM ALLOCATION. HAS MANIPULATION. CAN HAVE PRETEST AND/OR POST-TEST.

RESEARCH DESIGN

• OFTEN USED WITH TWO HOSPITAL WARDS WITH PATIENTS WITH SIMILAR DIAGNOSES.

• ONE DOESN’T ABANDON A DESIGN. ONE ANTICIPATES THE THREATS AND PLANS HOW TO ADDRESS THEM.

RESEARCH DESIGN

• A. NON-EQUIVALENT CONTROL GROUP DESIGN (POST-TEST ONLY )

• GROUP A X1 O1

• -----------------------------

• GROUP B X1 O1

RESEARCH DESIGN

A. NON-EQUIVALENT CONTROL GROUP DESIGN (PRETEST-POSTTEST)

• GROUP A O1 X O2

• -----------------------------

• GROUP B O1 O2

RESEARCH DESIGN

A. NON-EQUIVALENT CONTROL GROUP DESIGN

THREATS

• SELECTION (EQUIVALENCE AT BASELINE),

• MATURATION,

• TESTING

• MORTALITY:

RESEARCH DESIGN

QUASI-EXPERIMENTAL DESIGNS:

3. TIME SERIES

NON EQUIVALENT OR NO GROUPS. MEASURE PRE-TEST SEVERAL TIMES THEN DO MANIPULATION. AND DO SEVERAL POST-TESTS

RESEARCH DESIGN

3. TIME SERIES

GROUP A O1 O2 O3 O4 X O5 O6 O7 O8

RESEARCH DESIGN

3. TIME SERIES

THREATS

• MATURATION EFFECT: DEPENDENT VARIABLE MAY CHANGE NATURALLY OVER TIME.

• MEASUREMENT: INTERVENTION FADE OVER TIME?

• SELECTION.

• MORTALITY.

RESEARCH DESIGN

PROS OF QUASI-EXPERIMENTAL DESIGNS

• PRACTICAL • FEASIBLE: WHEN YOU CAN’T

ETHICALLY ASSIGN PEOPLE TO NOXIOUS INTERVENTION: CANCER.

• GENERALIZABLE

RESEARCH DESIGN

CONS OF QUASI-EXPERIMENTAL DESIGNS

• LACK OF RIGOR

• INCREASED THREATS TO INTERNAL VALIDITY

RESEARCH DESIGN

3. EXPERIMENTAL DESIGNS

OFTEN LAY PEOPLE THINK THAT WHEN A STATISTICAL ASSOCIATION IS FOUND, THAT MEANS CAUSALITY. THERE ARE SIGNIFICANTLY MORE BLUED-EYED MURDERERS THAN BROWN-EYED. ALL BLUE EYED PEOPLE KILL?

RESEARCH DESIGN

3. EXPERIMENTAL DESIGNS CAUSALITY1. STRENGTH OF ASSOCIATION

2. DOSE-RESPONSE RELATIONSHIP

3. CONSISTENCY OF ASOCIATION

4.TEMPORALLY CORRECT ASSOCIATION

5. SPECIFICITY OF ASSOCIATION

6.COHERENCE WITH EXISTING KNOWLEDGE

1

RESEARCH DESIGN

EXPERIMENTAL DESIGN HAS 3 CHACTERISTICS

• 1) RANDOM ASSIGNMENT (major difference between quasi & experimental),

• 2) CONTROL, AND • 3) MANIPULATION OF INDEPENDENT

VARIABLE.

RESEARCH DESIGN• 3. EXPERIMENTAL DESIGNS• i . RANDOM ASSIGNMENT: SUBJECTS

ASSIGNED TO GROUP PURELY ON BASIS OF CHANCE.

• EACH SUBJECT HAS EQUAL OR KNOWN PROBABILITY OF BEING ASSIGNED TO ANY GROUP.

• ESSENTIAL TO ELIMINATE SYSTEMATIC BIAS THAT MAY EFFECT DEPENDENT VARIABLE.

RESEARCH DESIGN

i . RANDOM ASSIGNMENT: HELPS TO ASSURE THAT INTERVENING VARIABLES ARE EQUALLY DISTRIBUTED AMONG GROUPS, E.G. AGE, PROGNOSIS.

• TO ENSURE THAT RANDOMIZATION HAS OCCURED, YOU ANALYZE CHARACTERISTICS OF TWO GROUPS AND REPORT THE FINDINGS.

RESEARCH DESIGN

3. EXPERIMENTAL DESIGNS

i . RANDOM ASSIGNMENT

HOW IS RANDOM SELECTION DIFFERENT FROM RANDOM ASSIGNMENT?

RESEARCH DESIGN

3. EXPERIMENTAL DESIGNS

ii. CONTROL

MANIPULATING INDEPENDENT VARIABLE, ADHERING TO PROTOCOLS, AND USING COMPARISON GROUPS.

RESEARCH DESIGN

3. EXPERIMENTAL DESIGNS

iii. MANIPULATION

INDEPENDENT VARIABLE IS THE MANIPULATION. IN HEALTH SCIENCES THIS IS OFTEN AN EDUCATIONAL PROGRAM.

RESEARCH DESIGN

• ANTECEDENT VARIABLE: OCCURS BEFORE STUDY BUT MAY EFFECT DEPENDENT VARIABLE, E.G. SES, GENDER, RACE.

• INTERVENING VARIABLE: OCCURS DURING STUDY & MIGHT EFFECT DEPENDENT VARIABLE. E.G. PROVINCIAL PSYCHIATRIC HOSPITAL CLOSURES.

RESEARCH DESIGN

1. TRUE EXPERIMENT (RANDOMIZED CLINICAL/CONTROL TRIAL RCT).

• SUBJECTS RANDOMLY ASSIGNED TO EXPERIMENTAL/CONTROL GROUP. MAY HAVE PRE-TEST , BUT DEFINITELY HAVE POST-TEST MEASURES.

• WHY A CONTROL GROUP? HOW DO YOU KNOW GROUPS ARE THE SAME?

RESEARCH DESIGN

RANDOMIZED CLINICAL TRIAL

• WHY HAVE A PRE-TEST?

• WHY HAVE A POST-TEST?

• WHY RANDOMLY ASSIGN?

• HOW BIG A PROBLEM IS IT IF PATIENTS ARE NOT RANDOMLY SELECTED?

RESEARCH DESIGN

3. EXPERIMENTAL DESIGNS

ii. SOLOMON FOUR GROUP DESIGN

• BECAUSE YOU WANT TO PRE-TEST & POST-TEST WITH SAME INSTRUMENT IN SHORT PERIOD OF TIME, TESTING EFFECTS ARE AN INTERNAL VALIDITY THREAT. TO OVERCOME THIS, USE A SOLOMON FOUR GROUP DESIGN.

RESEARCH DESIGN

3. EXPERIMENTAL DESIGNS

ii. SOLOMON FOUR GROUP DESIGN

• RANDOMLY ASSIGN TO 4 GROUPS, ONLY GIVE PRE-TEST TO TWO GROUPS, GIVE POST-TEST TO ALL. WHAT DOES THIS ALLOW YOU TO SAY?

RESEARCH DESIGN

3. EXPERIMENTAL DESIGNS

ii. SOLOMON FOUR GROUP DESIGN

GROUP A R O1 X O2

GROUP B R O1 X O2

GROUP C R X O2

GROUP D R X O2

RESEARCH DESIGN

• 3. POSTTEST ONLY CONTROL GROUP DESIGN:

• NO PRE-TEST.

• NEED TO ENSURE THAT GROUPS ARE SIMILAR.

RESEARCH DESIGN

PROS OF EXPERIMENTAL DESIGNS

TEST CAUSE-EFFECT.

RESEARCH DESIGN

CONS OF EXPERIMENTAL DESIGN

• NEED TO KNOW WHICH VARIABLES MAY BE IMPORTANT IN PREDICTING OUTCOMES (DEPENDENT VARIABLE). HEALTH SCIENCES OFTEN DON’T; KNOWLEDGE IS AT EARLIER STAGE OF DEVELOPMENT.

RESEARCH DESIGN

• IN HEALTH SCIENCES, SOME INTERVENTIONS DO NOT ETHICALLY LEND THEMSELVES TO RANDOMIZATION. CAN’T FORCED PEOPLE TO PARTICIPATE (RANDOM SELECTION). WHAT WOULD BE EXAMPLES OF THIS?

RESEARCH DESIGN

NONEQUIVALENT CONTROL GROUP DESIGN (POST-TEST ONLY )

• GROUP A X O

• ---------------------------------

• GROUP B X 0