Randomized Trials: Print Module


Introduction

The randomized trial is often described as the "gold standard" of epidemiological research. In contrast to observational study designs such as the cohort study and the case-control study, randomized trials are experimental in nature: the researchers randomly assign exposure to individuals, rather than simply observing the patterns of exposure occurring in the study population. This exercise will highlight the key features of the randomized trials design, along with investigating its strengths and weaknesses.

Faculty Highlight: Dr. Alan Berkman (1945-2009)

Dr. Alan Berkman was Associate Professor of Clinical Epidemiology and Sociomedical Sciences at the Mailman School of Public Health. Dr. Berkman was actively interested in Public Health and social change since (at least) his days as a medical student at Columbia University. Dr. Berkman's work included the development of comprehensive programs in resource-constrained settings that integrated prevention, treatment and mitigation of HIV/AIDS at a population level. He was an early advocate for the global expansion of HIV care and treatment programs and explored ways that such programs could be used to strengthen prevention efforts.

His work in sub-Saharan Africa and the Caribbean, while initially focused on adults, brought his attention to the issue of the family as a unit and the particular impact of the epidemic on children. His research interests included the development of behavioral interventions that promote adherence to care and prevention for HIV+ individuals; structural interventions to strengthen family and community-based efforts to stabilize the lives of orphans and vulnerable children, and policy issues including HIV testing algorithms and innovative approaches for prevention of maternal to child transmission.

Read more about Dr. Berkman's work

  1. Berkman A, Cerwonka E, Sohler N, Susser E. A randomized trial of a brief HIV risk reduction intervention for men with severe mental illness. Psychiatr Serv. 2006 Mar;57(3):407-9.
  2. Berkman A, Garcia J, Munoz-Laboy M, Paiva V, Parker R. A critical analysis of the Brazilian response to HIV/AIDS: lessons learned for controlling and mitigating the epidemic in developing countries. Am J Public Health. 2005 Jul;95(7):1162-72.
  3. Berkman A. Confronting global AIDS: prevention and treatment. Am J Public Health. 2001 Sep;91(9):1348-9.

Learning Objectives

  1. Learn to apply the main features of designing a Randomized Trial.
    1. Formulate research hypotheses.
    2. Define eligibility criteria for study participants.
    3. Define exposure and outcomes.
  2. Learn the basic design features of a randomized trial.
    1. Describe the purpose of randomization.
    2. Describe the purpose of placebo control.
    3. Describe the purpose of blinding.
  3. Learn the main principles of analysis of a randomized trial.
    1. Describe the importance of "Intent-to-Treat" analysis.
    2. Calculate and interpret the incidence proportion (risk), risk ratio (relative risk), and risk difference.
  4. Learn the advantages and disadvantages of a randomized trial compared to other study designs.
    1. Describe some ethical considerations with respect to randomized trials.
    2. Describe the differences between observational and experimental studies.
    3. Describe the benefits and detriments of randomized trials with respect to internal validity, external validity (generalizability), cost-effectiveness and feasibility.

Student Role

The Epiville Department of Health has been inundated with calls concerning the sudden increase in cases of Susser Syndrome throughout the community. Along with attempts at understanding the cause(s) of this condition, the EDOH is invested in finding treatments for those with the disease.

Researchers at the Epiville General Hospital have teamed up with the Epiville Department of Health and DoseEmAll Pharmaceuticals to conduct a Phase III Randomized Clinical Trial on the effectiveness of a new drug (KimFaderol) designed to treat the most severe symptoms of Susser Syndrome. Researchers are hoping that this drug will be the first to demonstrate efficacy in treating the symptoms of Susser Syndrome. Your supervisor has asked you to oversee the design and analysis of the randomized trial.

You can learn more about Susser Syndrome on the Epiville Department of Health webpage.

Intellectually curious?

Learn more about the phases of clinical trials.


Study Design

Since there is currently no treatment for Susser Syndrome available, and since KimFaderol is not yet on the market, the only way patients will potentially have access to this treatment is by enrolling in this study.The researchers are using a randomized, placebo-controlled, double-blind trial to test the effectiveness of the drug in preventing the loss of proprioception, the most severe consequence of Susser Syndrome, in patients with stage I or II of the disease over a 12-month period of follow-up. This follow-up period was chosen to allow for sufficient time for individuals to progress to Stage III while minimizing the costs and complications associated with a longer-duration study.

Before you begin enrollment in the study, your boss asks you to prepare all of the information you will need to give to prospective subjects so they can give their informed consent (or refusal) to participate in the study. As you might expect, there is a great deal of information that needs to be disseminated to potential study participants to allow them to make an informed decision about participation in the study. There is also quite a bit of administrative work that needs to be done before the study begins.

1. Based on the facts presented, which do you think is the best hypothesis to study in this randomized trial?

  1. Individuals randomized to receive KimFaderol will be more likely to be cured of Susser Syndrome than will individuals randomized to placebo
  2. Individuals randomized to receive KimFaderol will have a lower risk of being diagnosed with Susser Syndrome than will individuals randomized to receive placebo
  3. Individuals with Stage I or II Susser Syndrome randomized to receive KimFaderol will be less likely to develop impaired proprioception than will individuals with Susser Syndrome randomized to receive placebo
Answer (a) — incorrect: Since KimFaderol is designed to prevent the onset of Stage III of Susser Syndrome (characterized by severely impaired proprioception), but does not claim to cure the disease, we would not expect KimFaderol treatment to lead to cure.
Answer (b) — incorrect: Everyone in this study has already been diagnosed with Stage I or Stage II Susser Syndrome.
Answer (c) — correct: This hypothesis specifies (1) the exposure groups (KimFaderol and placebo), (2) the outcome under study (impaired proprioception, (3) the population of interest (individuals diagnosed with Susser Syndrome but not yet experiencing impaired proprioception, and (4) the expected direction of effect.

[ Interactive Exercise ] This exercise describes the purpose of randomization.

2. What is the primary purpose of randomization in the study design as outlined above?

  1. The biggest strength of the randomized trial design is the experimental manipulation (randomization) of the treatment of interest such that all other causes of disease are equally distributed between those who receive the treatment and those that do not. Thus, treatment will not be associated with any other risk factors for disease (see interactive exercise above for more!)
  2. To ensure that individuals in both the treatment and placebo groups actually take the drug that they are administered
  3. To ensure that the clinicians and study investigators are unaware of the exposure status of the individuals in the study
Answer (a) — correct: One of the biggest strengths of a randomized trial design is its experimental manipulation of the treatment of interest, such that treatment assignment is, on average, not associated with other risk factors for the outcome under investigation. (See interactive exercise above for more!)
Answer (b) — incorrect: Unfortunately, randomization cannot guarantee that individuals actually take the treatment that they are randomized to receive.
Answer (c) — incorrect: This is the purpose of blinding in a randomized trial, not the purpose of randomization. (Blinding prevents clinicians from assigning treatment differentially based on knowledge of treatment assignment.)

3. What is the purpose of placebo control in the study design as outlined above?

  1. To ensure that the clinicians and study investigators are unaware of the exposure status of the individuals in the study
  2. To account for the "placebo effect" and to minimize the possibility that individuals in the study know what treatment they have been assigned to
  3. To ensure that individuals in both the treatment and placebo groups actually take the drug that they are administered
Answer (a) — incorrect: While placebo control can aid in masking the exposure status of individuals to the researchers, it is not sufficient to do so. This requires blinding.
Answer (b) — correct: Using a placebo control serves the dual role of accounting for the possibility that individuals who believe they are receiving the study drug will show a beneficial effect of treatment (the "placebo effect") and to prevent individuals from actually knowing whether they are taking an active treatment or not.
Answer (c) — incorrect: Although having a placebo which looks very much like the "active" drug may aid in compliance, it cannot guarantee that individuals actually take the treatment that they are randomized to receive.

Intellectually curious?

Learn more about the use of placebo in randomized trials, and the controversy surrounding when it is ethical to do so.


Data Collection

Now that you have your hypothesis and have learned the basic principles of the randomized trial study design, it's time to collect the data. First, you need to devise eligibility criteria, and also make sure you have developed an appropriate measure for the outcome of interest.

You decide on the following eligibility requirements for your study:

  • Individuals must have been diagnosed with Stage I or II Susser Syndrome at Epiville General Hospital, but have no evidence of Stage III Susser Syndrome at the time of the study initiation.
  • Individuals with medical conditions contraindicating treatment with KimFaderol are excluded
  • Individuals with other severe illnesses giving them a low probability of surviving for at least 12 months are excluded

Of the 194 cases of Susser Syndrome diagnosed at the Epiville General Hospital, 120 meet the eligibility criteria. Luckily for you, all give informed consent and agree to participate in the study! You randomly assign 60 of these individuals to receive KimFaderol and 60 individuals to receive an inactive placebo.

To assess the outcome of interest, you assign a clinician to examine each study participant 12 months after the beginning of the study for evidence of severely impaired proprioception. The appearance of this impairment is very pronounced, and individuals are classified as having developed impaired proprioception if they can not move their hands or feet with their eyes closed. Individuals failing the proprioception assessment are classified as "Stage III" Susser Syndrome, while individuals not showing impaired proprioception are classified as "Stage I or II" Susser Syndrome.


Data Analysis

After 12 months of follow-up you finally get the results of the study. They are presented in the table below.

 Outcome Classification (after 12 months of treatment and follow-up) 
Treatment GroupProgression to Stage III Susser SyndromeNo Progression to Stage III Susser SyndromeTotal
KimFaderol Group154560
Placebo Group52860

4. Based on the table above, which of the following represents an appropriate method to analyze whether KimFaderol treatment is associated with preventing the progression to Stage III of Susser Syndrome in this population?

  1. Compare the incidence proportion of Stage III Susser Syndrome in the KimFaderol group to the incidence proportion of Stage III Susser Syndrome in the placebo group.
  2. Conduct a functional MRI scan of all individuals developing severe symptoms and compare the results of the scan between those assigned to KimFaderol and those assigned to placebo.
  3. Compare the rate of symptom appearance in the KimFaderol group to the rate of symptom appearance in the placebo group.
Answer (a) — correct: Since we are assessing whether the experimental treatment prevents the occurrence of impaired proprioception (which defines the occurrence of Stage III Susser Syndrome) we can compare the incidence proportion of new severe symptoms between the two groups.
Answer (b) — incorrect: We do not need to perform any more diagnostic tests in order to assess our hypothesis. We are interested in whether KimFaderol prevents impaired proprioception (as assessed by the neurologist), and do not need a fMRI to tell us this.
Answer (c) — incorrect: We have no information on person-time of observation, and no information on the time of occurrence of symptoms, since we assessed outcome only once, 12 month after the start of the treatment. Therefore we cannot calculate rates.

5. Perform the following calculations:

a. Calculate the incidence proportion of Stage III Susser Syndrome in the KimFaderol group to 2 decimal places. Answer — 0.25
b. Calculate the incidence proportion of Stage III Susser Syndrome in the placebo group. Answer — 0.87

6. Are one or both of the following answers correct concerning what you can conclude by comparing the two incidence proportions calculated above?

  1. Only the following statement: The incidence of Stage III Susser Syndrome is 3.5 times higher in the placebo group compared with the KimFaderol group.
  2. Only the following statement: The incidence of Stage III Susser Syndrome is 62% less in the KimFaderol group compared with the placebo group.
  3. Both A and B are correct.
Answer (a) — incorrect: While the above is true, comparing the ratio of the incidence proportion in the placebo group to the KimFaderol group, other answers are also correct.
Answer (b) — incorrect: While the above is true, comparing the difference between the incidence proportion in the placebo group and the KimFaderol group, other answers are also correct.
Answer (c) — correct: Choice A compares the ratio of incidence proportions (the Risk Ratio) in the two treatment groups, while choice B compares the difference between incidence proportions (the Risk Difference) in the two groups. The Risk Ratio is often used to evaluate the strength of the association, while the Risk Difference is often used to measure the public health importance of a given association. Thus, they are both important in the evaluation of the results of the study.

You give the results of this analysis to your supervisor. However, she is concerned that you have underestimated the "true effect" of the drug on preventing the development of Stage III Susser Syndrome. She argues that you have included all individuals who were assigned to take the drug, but not all of these individuals actually took it. Since data were collected on the actual intake of the prescribed treatments by each participating individual in both arms of the study, your boss suggests doing an analysis by excluding those individuals who did not take their assigned treatment.

As a student of Principles of Epidemiology, you are hesitant to analyze the data in this way. The data on KimFaderol provided by the manufacturer show that individuals who consume large amounts of alcohol may have severe side effects if given the drug. There is concern that these side effects may prevent some individuals from complying with their treatment assignment. In addition, alcohol consumption is known to complicate Susser Syndrome, such that individuals who drink regularly are at higher risk for developing Stage III Susser Syndrome than are individuals who do not drink regularly.

You look into the data and see that, while all individuals in the placebo arm were compliant with their treatment, 10 of the 60 individuals in the KimFaderol arm were non-compliant. Eight of these individuals were classified as having Stage III Susser Syndrome, while the other 2 were not.

7. If you were to exclude the noncompliant individuals from your analysis, what would the new ratio be comparing the incidence of Stage III Susser Syndrome among those who actually took placebo to the incidence of Stage III Susser Syndrome among those who actually took KimFaderol?

Answer: Answer — 6.19

[ Interactive Exercise ] Click here for an interactive explanation of why the results of "Intent-to-Treat" analysis often differ from "compliance" analyses.

8. What is the purpose of Intent-to-Treat analysis?

  1. To ensure a valid comparison between those who actually took the drug and those who did not.
  2. To maintain the comparability between the treatment and exposure groups that was obtained through randomization.
  3. To ensure comparability between the treatment groups at baseline.
Answer (a) — incorrect: Intent-to-Treat compares individuals based on treatment assignment, regardless of whether or not they actually took the drug.
Answer (b) — correct: Not using Intent-to-Treat analysis means that the investigator is no longer assigning the exposure, and the "experimental" nature of the design is thus compromised. As a result, comparison groups may no longer be comparable with regard to different characteristics.
Answer (c) — incorrect: This is the purpose of randomization.


Discussion Questions

Carefully consider the following questions. Write down your answers (1 - 2 paragraphs) for question # 1 within a word document and submit your answers to your seminar leader. Be prepared to discuss all questions during the seminar section.

A recent study was conducted "to evaluate the efficacy of milk fortified with specific multiple micronutrients on morbidity in children compared with the same milk without fortification." Children aged 1-3 were randomly assigned to received fortified milk (N=316) or control milk (N=317) for one year. The study was double-blinded. One of the study outcome measures was incidence of diarrhea. Investigators reported a relative risk of 0.84 comparing the fortification group to the standard milk group. (Sazawal et al. Effects of fortified milk on morbidity in young children in north India: community based, randomized, double masked placebo controlled trial. BMJ, 2008)

Here is a link to the article.

  1. Discuss how the results might have been affected if the comparison group was: (1) no milk instead of non-fortified milk; (2) milk fortified with other nutrients besides those being investigated in the study.
  2. Do you see any ethical issues in this study in terms of equipoise?
  3. Discuss the meaning of informed consent in vulnerable or underage populations. Can truly informed consent ever be obtained in these populations? How would you as a study investigator make sure that patients enrolled in the study were fully informed? What elements should go into an informed consent in this type of study?