As pharma manufacturers and regulatory agencies increase the role of real-world data (RWD) in decision-making, researchers continue to scrutinize the quality and usefulness of the sources of those data. While health insurance claims databases are the most commonly used source for RWD, at least in the US, the wider use of electronic health records (EHRs) can raise questions about the accuracy and completeness of data collected in the clinical setting—and therefore its validity in real-world studies.
We ask Andrew Bate, Senior Director and Analytics Team Lead for Epidemiology at Pfizer and formerly on the board of directors of both the International Society for Pharmacoepidemiology and the International Society of Pharmacovigilance, about the limitations and opportunities in working with EHR data.
Q: Researchers tend to rely on claims data to generate real-world evidence (RWE). Why should studies be done with data from electronic health records as well?
A: There are so many important health care questions that are challenging to answer. We have a responsibility to make the most of all of the available data that can shed light on those questions, including data from electronic health records. Real-world data from claims and EHRs both offer the benefit of generalizability: We have long-term follow up and power from huge populations.
Health care databases are built for purposes other than pharmacoepidemiologic research, and thus are not absolutely perfect for all questions of interest in understanding routine use of medicines and health care more generally. They can, however, provide powerful insights as they reflect what was actually recorded during routine health care provision. If we understand the nature of the data, what it’s good at, and what it’s weaker on for a given question, then it becomes very useful—as long as we tailor our question in a way that is suitable to evaluation in real-world data.
Claims systems and EHRs obviously are built for very different purposes, and so naturally some questions are more suited or can only be addressed by one or the other data sources. Sometimes a linked data set of claims and EHR data might be needed. In all cases, the need to follow good pharmacoepidemiology practice when conducting studies applies to all forms of real-world data, including both claims and EHRs.
Q: Do you turn to real-world data only when a randomized controlled trial cannot be done?
A: I think health care tends to oversimplify discussions about hierarchies of evidence when, in fact, different data sources can address different questions. To answer some questions, a randomized clinical trial is absolutely the appropriate approach to take. But for others, real-world evidence generated from claims data and EHR data, or even other data sources such as patient-generated data, is critical because a clinical trial is not going to be able to answer the question of interest.
EHR and claims data enable researchers to look at issues that may not have been prioritized when a clinical trial was conducted, or were simply not possible to answer in a given clinical trial: What additional things happen to patients when they take medications, what is the impact of other medications outside the trial? For a given question, you might go to a claims database or to EHR data or even a patient registry, whichever is fit for purpose.
We always need to be cognizant in working with such data that there is always a chance that specific data of relevance may not have been recorded or recorded inaccurately or even erroneously; and that such errors may be non-random in their occurrence.
Q: Why would you select EHR data over claims data?
A: I have heard claims data described as scaffolding in building work. It’s reliable time-stamped data. We know we don’t have all the detail we would like, but for what data we do have, we have a strong understanding of its reliability and how to best use it in credible real-world evidence studies. In the US, claims databases have been used for pharmacoepidemiologic studies for many decades, and the databases that are most commonly used have been and continue to be extensively validated. For example, on common endpoints of importance in pharmacoepidemiologic studies, the field conducts studies to compare recorded codes to the information that can be gleaned directly from chart reviews.
Sometimes, however, claims data sets can be insufficient for wholly addressing a study question of interest, and this is when EHR data tends to come in—when there are details critical to the study question that are not in the claims data. If claims data is the scaffolding of a building, EHR data can be thought of as providing more of the interior details, the furnishings in the rooms. EHRs tend to offer clinical detail, such as data on smoking, disease severity, exercise, diet, alcohol use, and family history and biomarkers.
We recognize that EHRs are often confined to particular treatment settings, and there may be gaps in the records as patients move to other health care providers or are lost to follow up. We need to be mindful of these aspects of EHRs and design our studies carefully, navigating the strengths and limitations of the data. We must consider them in our result interpretation as we assess the potential for generalizability of the study results.
In the US, use of EHRs is relatively recent, so while there is robust evidence emerging for their appropriate use, it is less extensive than exists for claims. I would expect greater use of EHRs in the US as electronic data capture and linking across systems continues to improve, along with the body of evidence for their use in pharmacoepidemiology.
Usage of claims and EHRs is very different in other countries. For example, high quality EHR data in the UK, particularly covering primary care, has been used for credible RWE generation for decades, facilitated by the nature of UK health care delivery.
We always need to select the right data source(s) for the question at hand, and be careful not to oversimplify the differences across claims databases (e.g., governmental and commercial claims databases) as well as across EHRs.
Q: How can unstructured data in clinical notes of EHRs be utilized, and what is its value?
A: We did some work a couple years ago in an EHR database using natural language processing to search within the narratives for indications of acute liver disease. We could look at the timing and combination of different lab tests with varying thresholds, text used in clinical notes as well as considering combinations of ICD codes. Finding terms used in the clinical narrative allowed us to identify acute liver disease earlier than if we relied on diagnosis and procedure codes for acute liver disease in claims data.
Q: It’s not an overstatement to say that clinicians in the US are at odds with their EHRs. They input data and clinical notes under duress and often late at night after a long day of seeing patients; they make mistakes. How do you address human behavior when working with these data? How do you respond to skepticism about the quality of EHR data due to these realities?
A: First, all health care databases require lots of validation and quality assurance work around data collection from the data aggregator all the way through the data management lifecycle to the end of researchers conducting a study. There are many ways we can and should assess data quality and accuracy, and the more we are able to be clear about where and why there may be poorer data quality, and to quantify those issues, then the more confidence we can have in any form of RWD. There are many possible and differing approaches needed that together can provide this confidence. For example, we can look at concordance across health care databases or concordance between EHRs and claims data. We can compare rates to national statistics. We also have knowledge around obviously contradictory factors; we can look for contradictions in the captured data. Having a death code recorded and then additional data collection after this code is one obvious example.
There needs to be an understanding, however, that this is real-world data; it is not what happened in the health care encounter. Real-world data is a manifestation of what someone has chosen to record about health care encounters.
We talk about claims as if these are perfect data. But those data don’t represent health care treatment. Claims data perfectly represent a billing system—for example, why is a given ICD diagnosis code used in a certain situation and not another? Which combinations of ICD codes are most predictive of what we see on chart review? That’s validation work. The same principles apply to working with EHR data as they do for claims data. When we understand the circumstances around when and how data are recorded, we can build those potential considerations into how we conduct and interpret studies.
Pharmacoepidemiology studies and the researchers that have conducted studies need to demonstrate that they’ve appropriately accessed, analyzed, interpreted, and acted on those data. We must have audit trails, quality processes, and be very explicit that we’ve conducted the study appropriately.
It’s fundamentally about access to data. All stakeholders—industry, academic, regulators, and other groups—need sustained, reliable access to these data. There needs to be—from a public health perspective—very clear, transparent audit trails in terms of how we all touch data, what we do with it, why we do what we do with it. Sometimes RWD is still too hard to access. The more we clearly demonstrate the value of real-world data, the more there will be recognition of the need to facilitate fast, sustained access to data. And measuring the impact and value of the studies we do is a critical part of that.
Q: How do you address the limitations of EHR data in terms of accuracy and completeness?
A: It’s in some ways detective work, really. We need to use a variety of approaches and leverage learnings from other studies. Being transparent about the extent of our knowledge around accuracy and completeness of data in a given study is important. Linkage between claims and EHRs is clearly useful sometimes, but this is often not possible for studies where we need a very large sample size for the study question. Another way to do this is through the use of validated proxy measures which we have assessed on a small test set.
I’ll give you an example. In the UK, EHRs are very rich in data because people tend to go to the same primary care physician for a long period of time; they’re brilliant for monitoring the occurrence of disease over time, treatment patterns, and so on, but not so good for measuring disease severity. So, when researchers wanted to examine the association between chronic kidney disease and psoriasis patients, they anticipated that any risk would vary by disease severity. UK primary care physicians in general do not always directly measure and record psoriasis severity in EHRs. So, the researchers embedded a prospective cohort in which they asked a small subgroup of doctors to measure psoriasis severity by body surface area coverage and record that data in their EHR. Then, they were able to use those data to assess and show that routinely collected EHR data gave an accurate assessment of psoriasis severity in these patients. The researchers then examined psoriasis disease severity across the wider database and were able to study the relationship between psoriasis severity and chronic kidney disease, using the power of the wider cohort.
So, there’s a lot we can do with EHR data. The point is not whether the data are perfect or not. Instead, we ask: Where is it likely to be strongest for a given question? Where is it likely to be more limited? What additional measures, approaches or perhaps even additional data collection can we put in place to support conducting a robust, credible RWE study? And how can we access the data needed to appropriately test important hypotheses about the effects and use of medicines and other health care resources?
Q: What will be the payoff to clinicians from all this data entry work they’re doing?
A: I feel strongly that we as health care researchers have a responsibility in working with data collected by clinicians to make the most out of it to support their work. Delivering population-level statistics and insights derived from EHR and claims data is one way to do t
at.
In any health care encounter, it seems to me that patients are rarely given any kind of population-level statistics, e.g., “There are 10,000 people like you who just had this diagnosis, this is what happened to them, and this why knowing that may help you manage your diagnosis.” I believe that as physicians realize their health care delivery can be enhanced through a better understanding on a population level of what’s happening to patients, they will perceive the value of the data they collect—particularly if there are systems in place to help the physicians with data collection. They will see how it can start driving down the costs of health care encounters by improving outcomes.
I suspect we will see better personalized health care as decision-support systems get better, and rich, accurate data is a key factor in that development. Personally, I strongly believe clinicians will see personal benefits of accurate and timely data capture in terms of more effective health care encounters and that compelling use cases will become more prevalent in the near future.
Q: What difference will true interoperability make?
A: We’re seeing benefits of the different data sources across the lifecycle of drug development. Now, the question is how do all stakeholders ensure globally that primary and specialist care are linked, including oncology treatment? Interoperability is needed to link outpatient and inpatient EHRs, primary care and specialty care, and billing and EHR systems.
We also need to ensure that data linkage and access complies with international privacy and confidentiality laws and that regulations protect patient data. In pharmacoepidemiology, multiple database studies are sometimes needed, so ensuring access to data from different geographies is important. As these systems get bigger, as they become more interlinked, actual data collection will keep improving, and opportunities to use those data will get better and better.