Across the United States, a patchwork of communities have begun to lift the stay-at-home advisories brought on by COVID-19. But according to experts, identifying “therapeutic, prophylactic, and preventative treatments” is critical to reopening the country. Globally, we’ve seen a tremendous push by biopharma, regulators, and public health agencies to develop a vaccine; at the time of writing, there are 133 vaccine candidates and 14 clinical trials in progress to demonstrate efficacy and safety.
The traditional vaccine clinical development process involves three trial phases (Figure 1), and an approval for any vaccine can take years. But with the devastating global impact of COVID-19, the need for widespread inoculation has accelerated the process. Regulators such as the U.S. Food and Drug Administration (FDA) are acting with a “sense of urgency” and turning to real-world data (RWD) and real-world evidence (RWE) to quickly assess and approve potential treatments.
After a vaccine is approved and widely distributed, regulators will continue to monitor vaccine safety in the post-market setting, including via voluntary and spontaneous reporting through the Vaccine Adverse Event Reporting System (VAERS) and post-licensure phase IV safety studies using RWD.
Below, we summarize two common self-controlled study designs that regulators employ to make vaccine safety decisions.
Studying vaccine safety using RWD
Drug safety studies commonly use a cohort study design, which identifies a group of drug-exposed patients and an unexposed control group, then follows patients over time to compare the incidence of adverse events of interest. As always, when conducting studies using RWD, standards of principled database epidemiology, such as transparency and emulating a target trial, must be judiciously applied.
However, questions may arise with a cohort design, especially in situations of widespread vaccination or rare adverse events: Are there enough unvaccinated subjects to include in the control group? Are vaccinated and unvaccinated individuals similar enough to yield a valid comparison? To mitigate these concerns, self-controlled study designs leverage exposed and unexposed periods within the same individual. This helps to avoid many sources of bias by automatically adjusting for measured and unmeasured confounders—factors that could muddle the estimated effect of a vaccine on an adverse event because they are associated with both the vaccine and adverse event, but are not on the causal pathway—that are stable over time.
Self-controlled study designs
Two commonly used self-controlled study designs in vaccine safety are the self-controlled case series (SCCS) and the self-controlled risk interval (SCRI) study designs.
Self-controlled case series study design:
Why choose this particular design?
Introduced in 1995 by Farrington et al., the SCCS study design has been implemented in a variety of vaccine settings, including mumps-measles-rubella. Advantages of the SCCS study design versus a traditional cohort study design include the ability to control for time-invariant confounders (including unknown confounders). This design is also helpful in situations where widespread vaccination would make it challenging to identify unvaccinated comparators.
How do we implement this design?
Only cases—individuals experiencing an adverse event—identified over a pre-specified observation period are included in SCCS study design (Figure 2). SCCS studies evaluate and compare the incidence rates during the risk intervals and control periods. As with any method, the research team must ensure key study design assumptions are met.
What are the strengths and limitations?
SCCS studies account for measured and unmeasured confounders that do not vary over time by comparing event rates during exposed and unexposed time within the same individual; time-varying confounders must be adjusted for in the analysis. SCCS designs are appropriate for transient exposures (such as vaccines) and acute-onset outcomes.
Self-control risk interval study design
Why choose this particular design?
The SCRI is a simplified version of the SCCS study design that has been widely used in vaccine safety. Between 2002 and 2017, 22 out of 47 vaccine safety studies conducted using data from electronic health records (EHR) employed the SCRI study design.
How do we implement this design?
Only vaccinated patients are included in the SCRI study design, with the vaccination date assigned as the index date—the date the patient meets the inclusion criteria and enters the study (Figure 3). Intervals on a patient’s longitudinal timeline are pre-specified as either the “risk interval” (that is, a period promptly following vaccination) or the “control interval” (that is, a time period or periods occurring pre- or post-vaccination). Like the SCCS, the analysis compares the rate of adverse events during the risk interval to the rate during the control interval within each patient, with rates calculated based on the number of events occuring during the interval and the amount of time the patient contributes to the interval.
What are the strengths and limitations?
The SCRI is similar to the SCCS in its general strengths and limitations. Compared to the SCCS design, SCRI tends to be less susceptible to time-varying confounders due to the shorter analysis period, and the data processing is simpler. However, the statistical power is reduced compared to the SCCS design due to the inclusion of less unexposed time.
Case example: 2009 swine flu (H1N1) vaccine and Guillain-Barré syndrome (GBS) risk
We have seen cohort and self-controlled study designs employed in past pandemics to help make informed decisions on the risk/benefit trade offs of vaccines. In 2009, an outbreak of H1N1 prompted the rapid development of new vaccines and subsequent post-market safety monitoring, which identified an association between the H1N1 vaccine and GBS, a rare but potentially fatal condition. Researchers employed self-controlled study designs to inform the risk/benefit profile, using data from the Post-Licensure Rapid Immunization Safety Monitoring (PRISM) program, a network of five data sources, to conduct safety surveillance. Using the SCRI design, the incidence rate ratio of 2.50 (95% CI 0.42-15.0) for the H1N1 vaccine and GBS was suggestive of an increased risk. While the signal was later confirmed in an additional study showing an estimated 1.6 excess cases of GBS per one million individuals vaccinated, the authors concluded that the “benefits of inactivated pandemic vaccines greatly outweigh the risks.”
The response to COVID-19
Several initiatives are underway to better understand and respond to COVID-19: the FDA established the Coronavirus Treatment Acceleration Program (CTAP) to quickly act upon incoming requests for developing treatments, for example, and cross-industry working groups are collaborating on COVID-19 research initiatives to elucidate the impact on patients. Aetion and HealthVerity recently launched a tool to support biopharma and regulators in their evaluations of COVID-19 treatment approaches.
Given the rapid development of COVID-19 vaccines and treatments, RWE will play a critical role in understanding their safety and effectiveness. And such high-quality RWE generation will command structured, critical thinking, a well-articulated research question, justification of design element choices, and use of appropriate study designs, including the self-controlled designs described here, to ensure transparency and validity.
As the global response to COVID-19 evolves, health care and community leaders are working together to advance treatments and flatten the curve. Eventually, with effective interventions available to patients, the infectious disease epidemiologist will pass the baton to the pharmacoepidemiologist to continue the collective COVID-19 story.