The digital age has transformed the amount and quality of data available and given rise to innovative methods for storing, sharing, and analyzing this data. However, there are many places in the digital transformation pathway and real-world evidence (RWE) generation process that lack trust both in the data and among stakeholders — and without trust, progress stalls.
How do regulators, health technology assessment (HTA) agencies, payers, and other stakeholders know the manufacturer is not “cherry picking” results to confirm their expectations? Are manufacturers using valid methods, and how can we know?
At the International Society for Pharmacoeconomics and Outcomes Research (ISPOR)’s European Conference in Copenhagen, regulators, industry, academia, and other speakers addressed these questions. Over five days of short courses and sessions, attendees explored the “digital transformation of health care” and key questions related to trust and transparency of RWE, including how to establish process standards and methods of generating RWE. The conversations represent and industry-wide effort to build trust among all stakeholders.
Transparency
On the heels of the ISPOR Summit: “Building trust in real-world evidence: The role of study registration,” in October, transparency was also an energetic discussion at ISPOR Europe.
At the Summit and ISPOR, the joint ISPOR-ISPE Special Task Force focused on ways of elevating the prominence and usability of RWE in healthcare decision-making, while addressing a main critique of RWE studies — that protocols may be obscured; the black box problem. By promoting registration of all hypothesis testing observational study protocols, ISPOR-ICPE seek to improve reliability and reproducibility of studies, limit concern for “cherry picking” results, and limit peer review bias. This work is still in the early stages with, as discussed at ISPOR, the near, medium, and long-term goals of:
- Near term: Identify location for registration of study protocols
- Medium term: Determine what characteristics should be captured in registration and what the process of study registration looks like
- Long term: Incentivize researchers to register studies
In this way, ISPOR-ISPE are building a culture of transparency in RWE generation. While transparency does not equal study quality, it is a great first step to trusting the evidence generation process and creates greater clarity to evaluate the validity of the data.
Reproducibility
Shirley Wang, Ph.D., M.Sc., and Sebastian Schneeweiss, M.D., Sc.D., both from Brigham and Women’s Hospital and Harvard Medical School, also discussed both transparency and the importance of reproducibility for building trust in RWE protocols. Reproducibility — using the same study protocol and data to generate the same results — is a tenet of validated science, and without transparency in exactly how the results were generated, reproducibility is not possible. Increased transparency in study protocols through a registration process will enable opportunities for replicating studies and build trust in the results.
Standardization in methods
In 2019, there has been substantial activity among regulatory and HTA agencies around RWE standard setting and this, too, was discussed at ISPOR. Setting methodological standards and guidance for use of RWE will facilitate trust in and inform decision-making.
The FDA, NICE, ICER, and CADTH are just a few agencies working on developing RWE frameworks and guidance for manufacturers. These frameworks will promote manufacturer confidence in investing in RWE and increase understanding of how RWE will impact their regulatory and market access approvals. Manufacturers following these methodological standards and frameworks will be more likely to trust that their investment in principled database RWE throughout the product life cycle is worth the investment.
With innovations in data, technology, and methodology, RWE is positioned to have substantial impact on health care decision-making, with known limitations to its use.