Draft

Lesson 1: How to read and analyze RCT  

The purpose of this lesson is to enable you to understand and appreciate the basic principles of randomized controlled trials. Specifically, in this lesson, we shall discuss briefly the principles of randomized controlled trials, the basic principles of experimental study designs in health care (with special reference to intervention studies),  the process of randomization, the process of analyzing randomized controlled trials and how to critique a randomized trial and what to study when analyzing randomized controlled trials. Finally, we shall critically appraise a randomized controlled trial study as part of completion of this lesson.

What are randomized controlled trials  

Randomized controlled trials are study designs where participants in a trial are first randomly (non systematically) assigned to either an intervention or a control condition and then the participants in each arm of the trial are periodically followed up for relevant health outcomes. The rates of development of health outcomes in the intervention and control arms are measured, and compared to yield relative risk estimates. In drug trials where efficacy of a drug is tested against another usual treatment or another comparable drug, or a placebo condition (or a dummy treatment), the measure of efficacy is typically stated in terms of relative risk estimate that is less than one (RR < 1) to indicate that compared to individuals who were allotted to the control arm, individuals who were assigned to the intervention arm had less likelihood of adverse events. Correspondingly, if health outcomes were measured in terms of “success” or cure from a specific illness, then a relative risk greater than one (RR > 1) would denote that the treatment was successful.  A randomized controlled trial is also known as an experimental study design as a randomized controlled trial comes closest to experiments conducted with animals when it comes to human studies.

The two key terms in an RCT are randomization, and control. The term randomization denotes that the participants were comparable with each other prior to their random allocation to either the treatment (or intervention) arm and the control arms. The term “control” denote that a valid comparison group must be present to evaluate that the treatment or intervention really worked. In theory, a randomized controlled trial tests the thesis that in individuals who are comparable to each other in all aspects, after adjustment of all other conditions, the difference in the outcomes observed between those individuals allotted to the intervention arm and individuals allotted to the control can only be explained in terms of the intervention.  Because the studies are done under controlled conditions that the researcher or the investigator alone can allocate intervention and determine control conditions, therefore the study is essentially is a “controlled” study. Therefore the extent of success of an intervention (such as a drug) on the basis of an RCT is termed as “efficacy” of a drug or intervention. On the other hand, the beneficial effect of an intervention or a drug for a specific outcomes  is termed as “effectiveness” when the effect of the intervention is studied in the context of community settings (ie not under controlled conditions). Finally, as an aside, “efficiency” of an intervention is termed as the beneficial effect or desired effect relative to the amount of resources expended in order to organize the intervention and deliver them.

For any intervention to be truly efficacious for a given set of health outcomes (or even a particular health outcome), the study must account for the role of the play of chance,  bias, and confounding factors. In randomized controlled trials, the role of chance is addressed by selecting the number of participants prior to the study based on formula derived on the basis of the frequency of occurrence of the health outcomes. Bias, in randomized controlled trials is

minimized by using the principle of blinding and intention to treat (ITT). Blinding is a procedure which ensures that either the participant, or the investigator, or both the participant and the investigator are made “unaware” of the allocation status of the intervention and additionally from the process of data collection to observe the health outcomes. The health outcomes measurements happen in such a way that the investigator while collecting data on the health outcomes is unaware as to the allocation status of the intervention from among the intervention or the control arm participants. Further, it is possible that the participants in a trial (even if it is randomized controlled trial with blinding), be aware of the status of the intervention (ie, they are receiving an actual intervention as opposed to a placebo or an alternative or usual medication or treatment). This might result in alteration of the behaviour of the participant in the intervention arm for example (although it can also happen to a participant in the control) so that the participant actually switches to the intervention arm (or the control arm). While this may distort the results, the results are analyzed for individuals in the intervention and the control arm keeping the participants of the research in the same order in which they were initially allotted to their original sequence of interventions. This practice is known as “intention to treat” and ensures that there Is no “contamination” of the results or bias that may arise as a result of self selection of individuals to move to one or the other arm. Finally, the role of confounding is believed to be controlled for by the process of randomization. Randomization (see the following section) is a process where a random numbers table is used to non selectively allocate an individual to either the control arm or the intervention arm and then individuals are followed through. Because this is done without regard to specific characteristics of an individual, therefore, it is believed that the simple process of randomization will “distribute” all other factors randomly between the intervention and the control arms.

Thus, in brief, randomized controlled trials are essentially experimental studies applied in the context of human health and related intervention studies (randomized controlled trials are applicable to other types of studies as well, for example educational interventions, policy level interventions in other disciplines, behavioural sciences, law and others). The theory being, all things considered , the only difference in outcomes  observed (or no diference) can be attributed to the intervention.

Types of RCT  

Parallel group RCT. In most common applications, RCTs involve steps of identification of a population group, then applying the process of randomization dividing up the participants into two groups with one group receiving the intervention and the other receiving control condition. After this initial step and measurements made on each participant group with respect of outcomes, the groups are then followed up for observation of outcomes.  This common application of RCT is known as “parallel group” RCT. There are other situations. 

Factorial RCT. As discussed in the previous section, in cluster RCTs, clusters of individuals are taken up for randomization. In other kinds of studies, several combinations of treatments are studied even within these clusters. Let’s say effectiveness of two different treatment conditions are being studied; further, in addition, interaction of these two treatment conditions are being considered. One of them is Treatment X (the comparison for this Treatment X is Placebo for Treatment X, which we refer to as Placebo X). Let’s say the other treatment is treatment Y (corresponding control condition for treatment Y is placebo Y). Thus, there are four different combinations of treatment possible, Treatment X compared with treatment Y, treatment X being compared with placebo Y, treatment Y being compared with placebo X, and placebo X being compared with placebo Y (see the following table). Each individual is then randomized within one of the four study groups.

Crossover RCT. In the above condition, combinations of different treatments were studied with respect to their interactions. In other situations, because of restriction in the number of individuals, a crossing over of the comparison conditions are often done in RCTs. Here, initially the participants are randomized to receive either the experimental intervention (or experimental condition) or the control condition. After a certain period of time, no one receives any intervention (neither intervention nor placebo). This is followed by reversal of administration of placebo and interventions so that individuals who earlier received placebo now receive the intervention and those who received the intervention earlier now receive the control condition.

Cluster RCT. In parallel group, factorial, or crossover RCTs,  randomization is at an individual level.  However, there are situations when interventions  (and control conditions) are applicable to entire subgroups or individuals within a cluster or group. Because interventions (and placebo or allotment of alternative conditions) are at the level of clusters of individuals, these trials are termed as cluster RCTs.

Superiority, non-inferiority, and equivalence trials. As discussed in the previous sections,  in theory, RCTs test the assumption that the only factor that can explain the difference in the outcomes between the intervention and the control group is the intervention. However, depending on the content and nature of the “comparison”, RCTs are variously defined as superiority trials, non-inferiority trials, and equivalence trials. Superiority trials are those trials where a treatment X is believed to be “superior” (ie more beneficial and less harmful) than an alternative treatment condition (or placebo). In non-inferiority trials, the aim is to test that the treatment X under investigation is not inferior (ie as much beneficial or not more harmful) than the alternative treatment condition. In “equivalence” trials, which are typically applied to drug trials, two or more drugs are compared head to head to test whether the level of efficacy are equivalent (they are comparable with each other, or a new drug or treatment is as beneficial (or as harmful no more no less) when compared to an established drug or treatment.

Pragmatic versus traditional RCTs. As discussed above, randomized controlled trials are experimental study designs. As such, these studies take place under the controlled circumstances where the experimenter or the investigator decides the intervention, the participants, conducts randomization, and then allocates the participants into the intervention or the control arm. As a result, even if the results of the study shows effect in favour of the intervention over the control arm and the results are statistically significant, it is uncertain the extent to which the results can be applicable to a larger audience in the community. Therefore, beneficial or desired effects of interventions shown in RCTs are termed as "efficacy studies". On the other hand, desired or beneficial effects of interventions that are conducted within society at large are known as effectiveness studies. However, traditional RCTs have a few limitations (more detailed discussions below). Since participant selection is very narrow in RCTs, this tight and highly specified inclusion and exclusion criteria limit external validity (or generalizability of the results of an RCT beyond the study population or likes of study population). On the other hand, effectiveness of an intervention may also be obtained based on an observational study (will be discussed subsequently; however, in an observational study, it is not always possible to control for potential confounding factors, and observational studies are open to different types of bias. Therefore, some authors, such as Hotopf (2007) have proposed a specific type of RCT termed as “pragmatic RCT”. The main difference between pragmatic and traditional RCT is in the wide, relatively less strict selection of participants for study in pragmatic RCT to increase the external validity of the study. In addition, pragmatic RCTs, that are commonly used in studying impact of behavioural interventions (psychology or psychiatry), concealment of allocation of participants to intervention and control arm is a major issue to allow for minimizing respondent and observation bias.  

Randomization and use of random number tables The guiding principle of RCTs is equipoise, tha the investigator is agnostic of the outcome of the intervention and control group for the participants being compared. However, to maintain that participants in both arms of the intervention and the control are completely comparable (such that participants in the control group would be no different from the participants in the intervention arm had he or she received the intervention), a process termed as randomization is done. A random numbers table is used to generate a list of numbers in random sequence. The identity codes of individual participant in the trial are matched with each number generated as a result of the random number generation process or through the random numbers table. The numbers (and therefore participants) are then assigned to either treatment or the control arm. The allocation of individual participant to either treatment or control arm is conducted in a manner so that it'd be impossible for anyone to predict the allocation of any specific member to one of the comparison groups based on any specific characteristic.

The most important purpose of randomization is to control for the effects of confounding. Since randomization equally allocates individuals to either intervention or control group without regard to any other factor, it is believed that all potential confounding variables, including those that were not even initially considered are also accounted for.

Follow up of the events in intervention and control arms Following randomization, an initial set of evaluation on the stated outcomes are measured on each participant belonging to each group. This is an initial measurement of the outcomes to be compared after the trial period is over to estimate the effect of the intervention versus alternative. The intervention and the alternative conditions are then administered to each individual in each of the arms of the trial. Here, the concepts of blinding is an important.

Blinding refers to the phenomenon where either the investigator, or the observer, or the participant in the trial does not know the status of the arm he or she is allocated. Thus, during the process of randomization, if the investigator alone does not know the arm to which each participant is allocated, then such a trial is termed as a single blind trial. If in addition to the investigator, the participant too, does not know to which group he or she belongs, this is known as “double blind” trial.

Blinding of a trial is necessary to avoid bias. In the absence of blinding, it is possible that the investigator knows in advance about the allocation status of a participant, there may be errors in observation or recording of the outcome driven by what the investigator or the research team believes in. Similarly, if a patient or participant is aware of the arm to which he or she belongs, then the responses that are dependent of specific responses from the patient (ie those variables that cannot be measured objectively) are open to bias.

Comparison of health outcomes in the intervention and control arm participants

The participants in both the intervention and the control arms are followed up in time by investigators. After completion of the trial, the numbers of events of the outcome (number of participants in the intervention and the comparison arm with outcomes of interest) are counted. Based on thes counts, the following values are calculated:

1. Experiemntal event Rate. This is defined as the number of participants with the outcome over those who were allocated the intervention (EER)

2. Control Event Rate. This is defined as the number of participants with the outcome over those who were allocated the comparison intervention (or placebo; termed CER)

Based on EER and CER, the following values are further calculated:

1. Absolute risk reduction = EER – CER

2. Relative Risk = EER / CER

3. Numbers needed to treat (NNT) = 1 / ARR. This indicates how many participants need to be treated in order to prevent one adverse outcome

These concepts are illustrated using the following table.

Table 1. Calculations relevant to the design of an RCT

Intervention Outcome status

Outcome present Outcome absent Total

Allocated to intervention arm

A B A+B

Allocated to control arm

C D C+D

Total A+C B+D A+B+C+D

Measurements

EER = (A / (A+B)) * 100

CER = (C / (C+D)) * 100

ARR = EER - CER

RR = EER/CER

Example

Advantages of randomized controlled trials Randomized controlled trials are at the apex of the classification system on internal validity. This is because all three components – chance, bias, and confounding variables are well controlled in an RCT unlike observational study designs. The role of chance is addressed during the sample size selection process with adequate number of individuals in each of the study and control arms The role of bias is addressed by using blinding, concealment of allocation, and intention to treat analysis. Finally, the role of confounding variables in the association between an intervention and outcome is addressed by the process of randomization itself.

Limitations of randomized controlled trials There are two major problems with RCTs – their lack of generalizability and their expense. However, as discussed above, although RCTs are not without limitations. While RCTs are excellent study designs for efficacy of interventions, the selection and follow up of individuals

for the study are so rigorous that they are limited when it comes to generalizability of the study findings beyond the sample studied to a wider population. Therefore, although RCT has very high internal validity, the external validity of RCT is limited because of its lack of generalizability. As a result, parallel group RCTs are not well suited for interventions such as behavioural interventions or studying interventions for other diseases that are less well defined. This problem is partly addressed using other study designs such as observational study designs but observational study designs are beset with problems of internal validity, in particular bias. Pragmatic RCTs provide a limited but useful alternative to traditional RCTs for addressing complex interventions.

The second issue relates to cost of RCT. Conducting RCTs are expensive and for rare diseases, it takes time for enough results in RCT to accrue to enable conclusion.

Worksheet One:

Critique and comment on the study on

 

 

Lesson two: How to read and analyze cohort studies

What is the basic principle of a cohort study?

A cohort study is an observational epidemiological study to test an association between an exposure or an intervention with a health outcome. In this study design, individuals are initially selected based on whether they are exposed to a specific exposure variable of interest or not. The exposed and non-exposed individuals in the study are then followed up over time to identify what proportion of “exposed” and “non-exposed” individuals develop the health outcome of interest. Based on the number of individuals who develop the disease outcomes of interest, a relative risk estimate is calculated for these individuals. The relative risk indicates the strength of association between the exposure of interest and the outcome under study.

What is a cohort?

A cohort is defined as a group of similar individuals. The “similarity” is based on some specific exposure variable. For example, all workers in a factory that are exposed to a given chemical in a given floor or working at a specific shift belong to a cohort. Another example may be nurses who work in operating rooms and are followed up for a study to assess specific operating room related exposure variables with specific health outcomes.

An important consideration of a cohort study is the principle of “follow up”. By “follow up” is indicated that in a cohort study, initially the exposed and non-exposed cohorts are defined. After defining the exposed and non-exposed (or unexposed) cohorts, these individuals are followed periodically over time to observe if specific health outcomes that are under the study are altered or developed. Based on such observations, the results of the cohort study are analyzed.

Types of cohort study

Depending on how and when the cohorts are assembled and analyzed, a cohort study can be described as a prospective cohort study or a retrospective cohort study. In both studies, as has been described earlier, the cohorts are selected on the basis of exposure and followed through in time to ascertain development of specific health outcomes.

Prospective cohort study. In prospective cohort study, as the name suggests, the cohorts on the basis of their exposure are assembled at a point in time when no disease outcome status are known at all in either exposed or non-exposed cohorts. The cohorts are then followed through in time to ascertain development of the disease outcomes.

Example

BMJ. 2010 Mar 11;340:c927. doi: 10.1136/bmj.c927.

Mortality among contraceptive pill users: cohort evidence from Royal College of General Practitioners' Oral Contraception Study.Hannaford PC, Iversen L, Macfarlane TV, Elliott AM, Angus V, Lee AJ.

Abstract

OBJECTIVE: To see if the mortality risk among women who have used oral contraceptives differs from that of never

users.

DESIGN: Prospective cohort study started in 1968 with mortality data supplied by participating general practitioners,

National Health Service central registries, or both.

SETTING: 1400 general practices throughout the United Kingdom.

PARTICIPANTS: 46 112 women observed for up to 39 years, resulting in 378 006 woman years of observation

among never users of oral contraception and 819 175 among ever users.

MAIN OUTCOME MEASURES: Directly standardised adjusted relative risks between never and ever users for all

cause and cause specific mortality.

RESULTS: 1747 deaths occurred in never users of oral contraception and 2864 in ever users. Compared with never

users, ever users of oral contraception had a significantly lower rate of death from any cause (adjusted relative risk

0.88, 95% confidence interval 0.82 to 0.93). They also had significantly lower rates of death from all cancers; large

bowel/rectum, uterine body, and ovarian cancer; main gynaecological cancers combined; all circulatory disease;

ischaemic heart disease; and all other diseases.

They had higher rates of violent deaths.

No association between overall mortality and duration of oral contraceptive use was observed, although some

disease specific relations were apparent. An increased relative risk of death from any cause between ever users and

never users was observed in women aged under 45 years who had stopped using oral contraceptives 5-9 years

previously but not in those with more distant use. The estimated absolute reduction in all cause mortality among ever

users of oral contraception was 52 per 100 000 woman years.

CONCLUSION: Oral contraception was not associated with an increased long term risk of death in this large UK

cohort; indeed, a net benefit was apparent. The balance of risks and benefits, however, may vary globally, depending

on patterns of oral contraception usage and background risk of disease.

Explanation

Note, from the abstract of the above study, it is clear that:

1. At the time of commencement of the study there were two sets of women, those who used oral contraceptive pills and those who never used oral contraceptive pills

2. The main health outcome of interest in this case was rates of death in both groups for any cause

3. The women were followed up in time. Hence, the researchers mentioned women-years as denominator. The conception of women-years is this. If 100 women are followed up for 1 year (or 50 women followed up for 2 years, or 25 women followed up for 4 years), the denominator is equivalent to 100 women-years.

4. At the end of the study health outcomes were compared between those who were exposed (ie in this case oral contraceptive pills versus those who were not) and the values were reported in terms of relative risk

Retrospective cohort study

In contrast to prospective cohort study, a retrospective cohort study is one where the exposure and outcomes have occurred all in the past. In other words, at the time of commencement of the study and assembly of the cohort, the status of exposure and outcome were all known. However, the participants in the study were still assembled with respect to their exposure and the outcomes were subsequently analyzed.

Example

BMJ. 2009 Dec 8;339:b4606. doi: 10.1136/bmj.b4606.

Cardiac outcomes in a cohort of adult survivors of childhood and adolescent cancer: retrospective analysis of the

Childhood Cancer Survivor Study cohort.

Mulrooney DA, Yeazel MW, Kawashima T, Mertens AC, Mitby P, Stovall M, Donaldson SS, Green DM, Sklar CA, Robison LL, Leisenring WM.

AbstractOBJECTIVES: To assess the incidence of and risks for congestive heart failure, myocardial infarction, pericardial

disease, and valvular abnormalities among adult survivors of childhood and adolescent cancers.

DESIGN: Retrospective cohort study.

SETTING: 26 institutions that participated in the Childhood Cancer Survivor Study.

PARTICIPANTS: 14,358 five year survivors of cancer diagnosed under the age of 21 with leukaemia, brain cancer,

Hodgkin's lymphoma, non-Hodgkin's lymphoma, kidney cancer, neuroblastoma, soft tissue sarcoma, or bone cancer

between 1970 and 1986. Comparison group included 3899 siblings of cancer survivors.

MAIN OUTCOME MEASURES: Participants or their parents (in participants aged less than 18 years) completed a

questionnaire collecting information on demographic characteristics, height, weight, health habits, medical conditions,

and surgical procedures occurring since diagnosis. The main outcome measures were the incidence of and risk

factors for congestive heart failure, myocardial infarction, pericardial disease, and valvular abnormalities in survivors

of cancer compared with siblings.

RESULTS: Survivors of cancer were significantly more likely than siblings to report congestive heart failure (hazard

ratio (HR) 5.9, 95% confidence interval 3.4 to 9.6; P<0.001), myocardial infarction (HR 5.0, 95% CI 2.3 to 10.4;

P<0.001), pericardial disease (HR 6.3, 95% CI 3.3 to 11.9; P<0.001), or valvular abnormalities (HR 4.8, 95% CI 3.0 to

7.6; P<0.001). Exposure to 250 mg/m(2) or more of anthracyclines increased the relative hazard of congestive heart

failure, pericardial disease, and valvular abnormalities by two to five times compared with survivors who had not been

exposed to anthracyclines. Cardiac radiation exposure of 1500 centigray or more increased the relative hazard of

congestive heart failure, myocardial infarction, pericardial disease, and valvular abnormalities by twofold to sixfold

compared to non-irradiated survivors. The cumulative incidence of adverse cardiac outcomes in cancer survivors

continued to increase up to 30 years after diagnosis.

CONCLUSION: Survivors of childhood and adolescent cancer are at substantial risk for cardiovascular disease.

Healthcare professionals must be aware of these risks when caring for this growing population.

Explanation

The following points are notable from the description of the above study.

1. The purpose of this study was to test the association between exposure to being diagnosed with cancer in childhood and morbidity from different cardiovascular disorders

2. The exposure variables were being diagnosed with cancer. The comparison group was selected on the basis of being diagnosed with no cancer from the siblings (thus genetic components were being addressed).

3. The outcome variables were a number of diagnosis of cardiovascular diseases. Note that when this study was conducted the results of cardiovascular abnormalities were already known.

Rates of development of health outcome in a cohort study

As with randomized controlled trial, with both prospective and retrospective cohort studies, the rates of development of the health outcome of interest are compared between those who are exposed and those who are unexposed are compared. The ratio of such comparison is expressed in the form of Relative Risk (Table 2)

Exposure level Outcome Level Total

Outcome present Outcome absent

Exposed A B A+B

Non exposed C D C+D

Total A+C B+D A+B+C+D

Rate of the disease among exposed = A/(A+B)

Rate of disease or outcome among non-exposed = C / (C+D)

Relative risk = Rate among exposed / Rate among non-exposed

Advantages of a cohort study

Next to randomized controlled trials, prospective and retrospective cohort studies are the two study designs that are believed to be the better study designs to study association between an exposure or an intervention with respect to an outcome. Both prospective and retrospective cohort study designs can control for the role of chance by selecting an adequate number of individuals to follow or extending the follow up period. For rare diseases for example, the consideration of person-year in the denominator allow for studying large number of individuals for a short period of time (relevant for commonly occuring diseases), or alternatively, studying fewer individuals over a longer period of time (relevant for relatively rare diseases). The role of bias is a major issue in cohort studies because exposure status is known. However, use of objective measures such as overall disease data (or diagnosis based on individuals who are unrelated to the study) are often used to ascertain outcomes (spot the possible biases in the example provided). Finally, the role of confounding variables cannot be well controlled in cohort studies because the comparison groups may not be comparable to each other in all respects. In the current study example, the confounding variables related to genetic and shared environmental variables might be controlled for but there may be yet other confounding factors which could not be adequately controlled for (for example level of physicial activity or effects of medications). A possible way to adjust for confounding variables is by way of multivariate analysis of data. Finally, cohort studies are advantageous in establishing a cause and effect association since it is relatively easier to control for the effect of time. Thus, in a cohort study, the sequence of exposure and occurrence of outcome is often known or can be assumed. This sequence of time between exposure and outcome occurrence is one of the strengths of cohort studies.

Critique of a cohort study

Critique of a cohort study using the STROBE statement will be left as an assignment for this week for your reflection paper.

References