Ulka Campbell, Ph.D., is Aetion’s Head of Scientific Strategy. In this role, she uses her 20+ years of industry experience to provide methodological and regulatory advice on real-world safety and effectiveness studies across therapeutic areas. In addition, she has co-authored several publications on best practices for regulatory-grade real world studies, causal inference, and epidemiologic methods, is an Adjunct Assistant Professor of Epidemiology at Columbia University, and has taught courses in pharmacoepidemiology, causal inference, and confounding control methods.
We sat down with Ulka to learn more about her new publication, which outlines a screening tool that biopharma teams can use to identify development programs that have the best chances of using real-world evidence (RWE) to support US Food and Drug Administration (FDA) approval.
Responses have been edited for clarity and length.
Q: What was the goal of this publication?
A: RWE for regulatory purposes, beyond the traditional use for post-marketing safety, is having somewhat of a moment right now – we are seeing biopharma companies really think about how RWE can complement traditional evidence-generation activities. We’ve also seen FDA seize this opportunity as well. The agency released six guidances over the past two years, with more expected soon. As we parse these alongside other FDA guidances and compare them to our experience at Aetion reviewing FDA approvals, we have identified a number of key learnings on when RWE is most likely to successfully support an approval, and equally important, when it is unlikely to be successful. We wanted to create a tool that will help a biopharma company assess its portfolio and gain an early understanding of where RWE of effectiveness may be acceptable for FDA decision-making.
Q: How did you go about creating the particular parameters for the SURF screening tool?
A: We began with a critical assessment of FDA’s substantial evidence guidance, which outlines several different examples of how a sponsor can provide evidence of effectiveness that rises to the level of support needed for a product approval. We then considered our internal experience with regulatory-facing studies and our interactions with FDA in forums (like Type C meetings) to provide additional context to how these scenarios might play out in practice. Finally, we incorporated learnings from our library of FDA approvals since 2019, which catalogs how sponsors use RWE to support product applications and how FDA incorporates it into its decision-making process. Through this process, we identified a set of themes that arose repeatedly, which are reflected in the success stories you see in the paper.
Q: Can you describe, at a high level, the success factors you identified as giving a sponsor the best chance for using RWE as primary evidence of effectiveness in an FDA application?
A: The screening tool requires the user to consider a series of questions, which reflect six success factors:
Question 1 asks if the indication is a life-threatening or severely debilitating disease with unmet need. This screening step reflects FDA’s legal authority to tolerate evidence generated from study designs with greater uncertainty in order to get drugs approved for patients with serious illnesses that are not addressed by current therapies.
Question 2 asks if a randomized, concurrently controlled trial is infeasible. FDA typically wants to see a randomized controlled trial as the evidence source for a product application. RWE is best suited to situations where this gold standard design cannot be implemented, either due to a limited patient population, a comparator assignment is unethical, or other reasons.
Question 3 asks if the primary endpoint can be objectively measured in real-world data (RWD), a key concern of FDA’s that increases the interpretability of results.
Question 4 asks whether the natural history of the indication is established. If the natural course of a disease is well-understood, it is easier to interpret and rely on the RWE supporting effectiveness claims in a product application.
Question 5 prompts the sponsor to assess its total evidence package and understand what supporting evidence exists to provide additional comfort in the results seen from an RWE study. It is difficult for RWE to stand on its own, given the inevitable uncertainty from measurement error in the outcome, potential residual confounding, etc. Robust corroborating evidence can help address latent concerns around an RWE study.
Question 6 asks if the sponsor has existing evidence that there is a large treatment benefit associated with the therapy. This step is somewhat of a “nice to have,” as this evidence may not be available at the time of pivotal study planning. What ultimately matters is the observation of a large treatment benefit, which increases the interpretability of RWE. Early evidence of a large treatment effect increases confidence that it may be observed in the pivotal study.
Q: One of the steps involves whether the natural history of the indication is sufficiently established. Can you explain what natural history is and how it can be established?
A: FDA’s rare disease natural history guidance defines natural history as “the course a disease takes in the absence of intervention in individuals with the disease, from the disease’s onset until either the disease’s resolution or the individual’s death.” A strong understanding of a disease’s natural history is critical to identifying confounding factors and thus helps to facilitate interpretable results from an RWE study.
If there are gaps in natural history understanding, a sponsor can conduct a study to help establish natural history. To do this, its RWE team (or external experts) should be consulted to design a study that identifies demographic, genetic, environmental, and other factors that predict the disease’s development and outcomes.
Q: Once a particular development program has passed the SURF screening tool, what are the next steps? Is a biopharma company ready to conduct a study and submit to FDA?
A: The SURF screen is definitely not the end of the road. If a development program passes the screen, it merely signals that the program is one in which RWE may be acceptable from a regulatory feasibility perspective. Sponsors still need to conduct a systematic scientific feasibility assessment to generate a robust study design and identify fit-for-purpose data sources. There are many resources available to assist with these subsequent steps, including the SPACE and HARPER templates for study design and SPIFD for data feasibility (see our webinar on these tools here). It is also important to solicit FDA feedback on the study design before initiation.
Q: Is the SURF screening tool applicable to the European Medicines Agency’s (EMA) drug authorisation process?
A: The SURF screen is built on guidance and case studies from the US FDA, but we plan to conduct a similar analysis of European Union legislation and EMA guidance to augment the tool. At this stage, we believe that the principles of SURF are generally applicable to EMA, given consistency in guidance and approach with FDA.
If you would like to speak with one of our experts on this topic or any other issues related to RWE generation, please contact us.