Ask the Accelerators is a series highlighting the perspectives and work of organizations and individuals participating in the COVID-19 Evidence Accelerator. Organized by the Reagan-Udall Foundation for the U.S. Food and Drug Administration (FDA) and Friends of Cancer Research, the COVID-19 Evidence Accelerator joins leaders in the health care data and analytics space to explore how real-world data (RWD) can inform key COVID-19 research questions.
In working together with leaders across the health policy community in Washington, D.C., and Duke University, the Duke-Margolis Center for Health Policy (the Center) aims to address pressing issues in health care, including the advancement and use of real-world evidence (RWE).
Nirosha Mahendraratnam Lederer, Ph.D., M.S.P.H., health outcomes researcher by training and self-proclaimed “policy wonk,” oversees the Center’s RWE portfolio and leads its Real-World Evidence Collaborative, which joins biopharma, academia, data companies, payers, and patient groups (with regulatory stakeholder observers) to assess how RWD and RWE can support decision-making and improve patient treatment options and outcomes. She also represents the Center in the COVID-19 Evidence Accelerator, sharing progress as the team collaborates with other data and analytics experts to evaluate remdesivir use in patients hospitalized with COVID-19.
Here, Dr. Lederer shares more on the Center’s RWE initiatives, including ways in which RWE can complement randomized controlled trial (RCT) data in regulatory submissions and how COVID-19 has emphasized the importance of RWE as a decision-making tool. Read more to learn what she’s most excited for in the work to come, and why she thinks “the time for RWD and RWE is now.”
Responses have been edited for clarity and length.
Q: Can you please summarize some of the work that the Center is doing around RWE?
A: At the Center, we have two main work streams related to real-world data and real-world evidence. The first is through a cooperative agreement with the FDA’s Center for Drug Evaluation and Research, where we support convenings on strategic priorities including real-world evidence—a clear priority as mandated by the 21st Century Cures Act and the sixth Prescription Drug User Fee Act—and others, such as considerations for including pregnant women in trials and treating opioid use disorders.
The other major work stream is through the Real-World Evidence Collaborative. Over the past few years, the Collaborative has been focused on FDA decision-making, specifically labeling changes for marketed products and effectiveness, but I always say that data and evidence aren’t collected or generated in a silo. We should be thinking about evidence generation for drug development throughout the product lifecycle and the health care ecosystem. This year, we have taken active steps to grow that portfolio, and to start thinking about how other decision-makers, including regulators, payers, providers, health care systems, and patients, are using RWD and RWE.
In addition, in partnership with the National Health Council and with funding through PCORI, we are developing a training program for co-developing RWD and RWE with patients.
Q: Have any of those projects or goals changed because of COVID-19?
A: COVID-19 has actually accelerated and enhanced the work we’re doing. In recent years, we’ve worked on defining how to evaluate and describe data fitness for use—one of the three prongs of the FDA’s RWE Framework. We also did some work characterizing the role of real-world evidence as part of an evidence package when submitted to the FDA, and when and where RWE can complement clinical trial data.
We built on that work this year, expanding it to think about how to develop a real-world endpoint, and the role of real-world data for external control or comparator arms. All of this work has advanced significantly because of COVID-19, which we will explore at our October 1 Annual RWE Conference, and each piece is essential to understanding and using COVID-19 RWD and RWE.
Unfortunately, because we are in a pandemic fighting a novel disease, every piece of data we can get is precious. One positive thing we’ve seen is that stakeholders, some who are competitors, have set aside competing interests and worked together to build a better RWD and RWE infrastructure for COVID-19 research. I’m optimistic this cooperation will lay the foundation for future partnerships and collaborations, which is a major component of getting real-world data and evidence into use.
Q: How can RWE complement data from RCTs as part of an evidence package, and how can biopharma incorporate both into their submissions?
A: The FDA uses a totality of evidence approach and doesn’t make decisions based off of a single study; there are several pieces of information that go into an evidence package. We have to think about RCTs and RWE complementing each other rather than RWE replacing an RCT, and to do so, we have to determine why a randomized controlled trial isn’t feasible in a certain context, and how RWE can support the evidence package.
In the Center’s 2019 paper, “Understanding the Need for Non-interventional Studies Using Secondary Data to Generate RWE for Regulatory Decision-making, and Demonstrating their Credibility,” we break these considerations into ethical, operational, and resource barriers to randomizing, blinding, and controlling in the real world. Using this as a guide can help researchers decide whether they’ll need real-world data for the study, then make sure that the data can actually answer the research question. Of course, this is all contingent on fit-for-use data and robust methods to make valid causal inference.
It’s critical to start with a well-defined research question. You should not ask, “What questions can I answer with my data set?” Your process for developing that question should be hypothesis-driven and scientifically rigorous. It’s also important to pre-specify all data curation and analytic methods in the study protocol to ensure transparency and credibility.
Q: What have you and your team learned from working with the COVID-19 Evidence Accelerator?
A: I am so impressed with how our community has come together in the COVID-19 Evidence Accelerator; it’s a testament to everyone involved, as well as the leadership provided by the Reagan-Udall Foundation for the FDA, Friends of Cancer Research, and the FDA. Because COVID-19 is so new, and we’re working so quickly to capitalize on every piece of information, it is essential that we collaborate and learn from each other.
I’ve also been impressed with the eagerness that the participants are bringing to the work, and the quality of the work they’re producing. There’s a lot of great science being generated, and the speed at which things are moving is unprecedented.
Some of the most valuable learnings coming out of the Evidence Accelerator are the enhanced relationships and the opportunity for different stakeholders to learn about data and methods they normally wouldn’t be exposed to. It enhances the field for everyone, so I look forward to seeing how we continue to collaborate post-COVID-19.
Q: What challenges have you encountered in using RWD and RWE to fill evidence gaps for COVID-19 research?
A: Because this is a new disease, and we’re studying new products, we’ve come across some logistical and operational challenges. For example, it took a while before we had an ICD code for identifying COVID-19, or CMS codes for remdesivir and convalescent plasma. We had to come up with novel or different methods to identify these products and patients, which could change depending on the data source—occasionally raising questions as to how comparable the different data sets were.
We’re also working toward gaining alignment in data capture. That doesn’t necessarily mean that everyone is using the same common data model or curation methods, rather, it’s making sure that we’re transparent in how we’re identifying the variables and methods used for analyses.
Through the Evidence Accelerator and otherwise, we are consistently encountering challenges to studying inpatient therapeutics due to factors like immortal time bias and confounding by severity. The more we can do to identify these limitations up front, the closer we come to developing a robust, standardized study design where we can align the study design and analysis. In the future, we could have a master RWE protocol, complete with data harmonization, study design harmonization, and alignment in methods. It’s exciting to think about what’s going to come next.
Q: Have approaches to evidence generation changed due to COVID-19?
A: As I mentioned, we should always work from the research question and the data, but I think we may have better data infrastructure post-COVID-19. There’s been an incredible acceleration of data capture and curation with regard to electronic health records, so I think the types of questions that we’ll be able to answer with RWD may be different.
We’ve learned a lot from COVID-19, and we’re still learning about how to work with COVID-specific RWD and RWE. Those lessons will help all decision makers, including the FDA, as they determine when and how they’ll use RWD and RWE. Ultimately, COVID-19 has highlighted the importance of RWE and its role in the evidence paradigm, and if anything, it will advance all the things that the agency has been working on to date.
Q: What do you see as the most exciting opportunity for RWE?
A: A lot of times, people think of RWD as a tool for optimizing clinical trials. That’s an important use case, but we’ve seen exciting next steps coming out of the RECOVERY Trial in the U.K., where the team leveraged a large sample size and objective outcomes to conduct a successful, randomized real-world study. It’s incredible, and it’s important to think about how we could replicate something like that in the U.S.—we’re exploring EHR-embedded, practical trials at the Center.
There is so much inherent value in secondary data; it includes broader populations and outcomes that are often under-studied in clinical trials, and it can reflect changes to standard of care better than trial data. I hope that, as we become more comfortable in understanding how to interpret and use RWD and RWE, we have the opportunity to capitalize on these valuable insights and improve our health care system. RWE can play a major role as we think about how to optimize value.
I’m optimistic for the future. When it comes to RWD and RWE, beyond COVID-19 and the opportunities born from the unprecedented collaboration, we have the technology to do incredible data curation. Pharmacoepidemiology is bursting, and we have great, novel methods. We’re learning so much, and I think the time for RWD and RWE is now.