+Experimental

Research Designs:

True Experimental vs. Quasi

Experimental Research Designs

Ms. Sharanjit KaurLecturer

UCON, BFUHS, Faridkot

+

Health care professionals are increasingly expected to adopt an evidence based practice.

Experimental Research generates the Best Evidence to practice EBP

Introduction

+Hierarchy of study designs

7.7.

6.

5

4

33

11

22

6.6.

5.5.

44

+Study designs

in an evidence hierarchy

+Introduction

Experimental Designs are a type of Quantitative research design

Basic difference

Here the researchers are active agent, not passive observers.

+

Experimenters manipulate certain stimuli, treatments, or environmental conditions and observe how the condition or behaviour of the subject is affected or changed.

How are they Active agents…..!!!

+

After the experimenter defines a problem he has to propose a tentative answer to the problem or hypothesis.

Further, he has to test the hypothesis and confirm or disconfirm it.

Hypothesis Testing….. Experimental

Research

+Characteristics of True ExperimentRandomization: the experimenter

assigns subjects to a control or experimental group on a random basis.

Manipulation: the experimenter does something to atleast some of the subjects in the study.

Control: the experimenter introduces controls over the experimental situation, including the use of a control group, that does not receive the treatment.

Randomization

• The solution to getting two groups as similar as possible is to allocate subjects using some random system, by tossing a coin maybe: heads they go to the treatment group, tails to the control group.

+How to Randomize ?

Toss a coin

Use random number tables

100 Random Numbers*

10438 49959 31396 13643 40346 60236 27124 26460 01893 02152 54491 70917 95080 49150 86939 47014 93348 27787 51027 28192 23256 11506 46755 41010 84803 53423 74122 85467 77990 00420 33128 35005 76258 79058 97216 42741 06166 62777 92944 55559 67713 04693 20310 00016 60900 51691 52759 49555 59168 30328 16847 30733 89076 40605 12574 43550 50623 55963 39941 88007 43809 02961 72390 64104 42078 81599 63036 45282 71985 91875 58763 58100 33533 07234 59572 02557 26719 54232 18579 52095 24988 32869 82667 59831 54895 43146 78394 72649 16442 58504 05761 17106 09629 32060 38210 66645 07897 10697 28856 47823 * This table of 100 random numbers was produced according to the following specifications: Numbers were randomly selected from within the range of 0 to 99999. Duplicate numbers were allowed.

EXPERIMENTAL DESIGNS

+

Two groups, randomized subjects, post-test only design (after only design):

Group Independent Variable Post-test

E X Ya

C - Ya

R

Basic Experimental Design:

+

The main advantage of this design is random assignment of subjects to groups, which assures the equivalence of the groups prior to experiment. Further, this design, in the absence of pretest, controls the effects of history, maturation and pre-testing etc.

This design is useful in the experimental studies at the preprimary or primary stage and the situations in which a pre-test is not appropriate or not available.

Basic Experimental Design:

+Two groups randomized subjects,

pre-test post-test design (Before-after design)

Group Pre Test Independent Variable Post-test

E Yb X Ya

C Yb - Ya

R

+

The main advantages of this design include:

Through initial randomization and pre-testing equivalence between the two groups can be ensured.

Randomization seems to control most of the extraneous variables.

But the design does not guarantee external validity of the experiment as the pretest may increase the subjects’ sensitivity to the manipulation of X.

Pre-test post-test design (Before-after design)

+Solomon Four Group Design

Group Pre Test Independent Variable Post-test

E 1 Yb X Ya

C 1 Yb - Ya

E 2 _ X Ya

C 2 _ - Ya

R

+This design is considered to be strong one as it actually involves conducting the experiment twice, once with pre-test and once without pre-test. Therefore, if the results of these two experiments are in agreement, the experimenter can have much greater confidence in his findings. The design seems to have two sources of weakness. One is practicability as it is difficult to conduct two simultaneous experiments, and the researcher encounters the difficulty of locating more subjects of the same kind.

+Factorial designs

Experiments may be designed to study simultaneously the effects of two or more variables. Such an experiment is called factorial experiment. Experiments in which the treatments are combinations of levels of two or more factors are said to be factorial.

When two independent factors have two levels each, we call it as 2x2 (spoken "two-bytwo”) factorial design. When three independent factors have two levels each, we call it 2x2x2 factorial design. Similarly, we may have 2x3, 3x3, 3x4, 3x3x3, etc.

+

Auditory A1

TactileA2

15 minsB1

A1 B1 A2 B1

30 minsB2

A1 B2 A2 B2

45 minsB3

A1 B3 A2 B3

Type of stimulation

Daily Exposure

Factorial designs

+CROSSOVER DESIGN

It involves the exposure of the same subjects to more than one experimental treatment.So it’s a within-subject design

Advantage

Highest equivalence among groups exposed to different conditions

Disadvantage

Carry-over effect

+Quasi-experimental design

Quasi experimental designs lack randomization, the signature of a true experiment

Provide less satisfactory degree of control and are used only when randomization is not feasible.

+Nonequivalent Control

group Design (Before-after Design)

It is probably the most frequently used design in social research. It is structured like a pretest-posttest randomized experiment, but it lacks the key feature of the randomized designs -- random assignment

+

Group Pre Test Independent Variable Post-test

E Yb X Ya

C Yb - Ya

+This design is especially susceptible to the internal validity threat of selection. Any prior differences between the groups may affect the outcome of the study.

Under the worst circumstances, this can lead us to conclude that our program didn't make a difference when in fact it did, or that it did make a difference when in fact it didn't.

(counterbalance can be used but carry over effect)

+Pre-experimental designsThey are least effective and provide little or no control of extraneous variables.

The one group pre-test post-test design :

This is a simple experimental research design without involvement of a control group.

Group Pre Test Independent Variable Post-test

E Yb X Ya

The design has the inherent limitation of using one group only. The design also lacks scope of controlling extraneous variables

+Nonequivalent control group posttest only design

This design provides some improvement over the previous by adding a control group which is not exposed to the experimental treatment.

Group Independent Variable Post-test

E X Ya

C - Ya

+

The major limitation of the design is that there is no provision for establishing the equivalence of the experimental (E) and control (C) groups. However, since no pretest is used, this design controls for the effects of extraneous variables pre-testing.

+Time series design

In this information is collected over an extended period and an intervention is introduced during that period.

Time Series (one group Pre-Test Post-test) design

O1 O2 O3 O4 X O5 O6 O7 O8

Time Series Non equivalent control group design

E O1 O2 O3 O4 X O5 O6 O7 O8

C O1 O2 O3 O4 O5 O6 O7 O8

+

Time series with multiple institution of treatments

O1 O2 X O3 O4 X O5 O6 X O7 O8

Time series with withdrawn and reinstituted treatment

O1 O2 X O3 O4 ( -X) O5 O6 X O7 O8

Time series design

+Time series design

+Validity of experimentation

Internal validity

Internal validity refers to the extent to which the manipulated or independent variables actually have a genuine effect on the observed results or dependent variable and the observed results were not affected by the extraneous variables. This validity is affected by the lack of control of extraneous variables.

+

External validity :

External validity is the extent to which the relationships among the variables can be generalized outside the experimental setting like other population, other variables. This validity is concerned with the generalizability or representativeness of the findings of experiment, i.e. to what population, setting and variables can the results of the experiment be generalized.

Validity of experimentation

+Factors affecting validity of experimentation

Threats to internal validity

History: The variables, other than the independent variables, that may occur between the first and the second measurement of the subjects (Pre-test and post test).

Selection: it’s the biasness resuling from the preexisting difference between the groups. When individuals are not assigned randomly to groups-, we must always be alert to the possibility that groups are non-equivalent.

+

Maturation: The changes that occur in the subjects over a period of time and confused with the effects of the independent variables.

Testing : Pre-testing, at the beginning of an experiment, may be sensitive to subjects, which may produce a change among them and may affect their post-test performance.

Factors affecting validity of experimentation

+

Measuring Instruments : Different measuring instruments, scorers, interviewers or the observers used at the pre and post testing stages; and unreliable measuring instruments or techniques are threats to the validity of an experiment

Experimental mortality : It refers to the differential loss of subjects from the comparison groups. Such loss of subjects may affect the findings of the study. For example, if some subjects in the experimental group who received the low scores on the pre-test drop out after taking the test, this group may show higher mean on the post-test than the control group.

Factors affecting validity of experimentation

+THREATS TO EXTERNAL VALIDITY

Hawthorn effect

Novelty effects

Interaction of history and treatment effect

Experimenter effect

+CONTROLLING EXTRANEOUS ANDINTERVENING VARIABLESMethods of controlling extraneous variables include :

• randomization

• homogeneous sampling techniques

• matching

• building the variables into the design

• statistical control

+Steps in a randomized

controlled trial

1. Select study subjects2. Measure baseline variables3. Randomize4. Blinding the intervention5. Follow subjects6. Measure outcome

If at all possible, you don't want the patients to know whether they are in the treatment group or the control group. You can achieve this by giving them identical looking tablets. (blinding)

What is Blinding?What is Blinding?

Types of Blinding?Types of Blinding?

• Single blind - participants are not aware of treatment group

• Double blind - both participants and investigators unaware, we refer to the design as a double blind randomised controlled trial (the gold standard among experimental designs).

• Single blind - participants are not aware of treatment group

• Double blind - both participants and investigators unaware, we refer to the design as a double blind randomised controlled trial (the gold standard among experimental designs).

• Triple blinded: The most common meaning is that the subject, researcher and person administering the treatment (often a pharmacist) are blinded to what is being given. Alternately, it may mean that the patient, researcher and statistician are blinded. The team monitoring the response may be unaware of the intervention being given in the control and study groups.

Types of Blinding?Types of Blinding?

Basic Trial Design

PopulationPopulation

SampleSample

Treatment outcome

ControlControl OutcomePlacebo

RandomizationRandomization

+Other difficulties encountered during RCT

Although randomised controlled trials are powerful tools, their use is limited by ethical and practical concerns.

For example, a non-random study suggested that multivitamin supplementation during pregnancy could prevent neural tube defects in children. Although the

study was seriously flawed, ethics committees were unwilling to deprive patients of this potentially useful treatment, making it difficult to carry out the trial which later showed that folic acid was the effective part of the multivitamin cocktail.

+

On the other hand, failure to perform trials may result in harmful treatments being used. For example, neonates were widely treated with high concentrations of oxygen until randomised trials identified oxygen as a risk factor for retinopathy of prematurity.

In other circumstances a randomised controlled trial may be ethical but infeasible—for example, because of difficulties with randomization or recruitment.

Other difficulties encountered during RCT

+Other difficulties encountered during RCT

+Clinical Trials Registry of

India (CTRI)

All interventional trials should be prospectively registered

(before recruitment of first subject)

+Registration Data Set of the CTRI

Public Title of Study Scientific Title of Study, Acronym, if

any Secondary IDs, (UTN, Protocol No etc) Principal Investigator’s Name and

Address* Contact Person (Scientific Query) Contact Person (Public Query) Source/s of Material or Monetary

Support Primary Sponsor Secondary Sponsor Countries of Recruitment Site/s of study Name of Ethics Committee and

approval status Regulatory Clearance obtained from

DCGI Health Condition/Problem studied

Study Type Intervention and Comparator agent Key inclusion/Exclusion Criteria Method of generating randomization

sequence* Method of allocation concealment* Blinding and masking* Primary Outcome/s Secondary Outcome/s Target sample size Phase of Trial Date of first enrollment Estimated duration of trial Status of Trial Brief Summary

Items are marked with an * are optional fields

+ 50

Questions or Comments

?