Skip to content
AdminOct 1, 20206 min read

ICPE research spotlight: Part II: Exploring new methods for RWE research

This year at ICPE All Access, Aetion’s scientific research is represented across 15 posters and presentations spanning novel applications of real-world evidence (RWE), innovative methods for epidemiologic research, and the standards needed to facilitate decision-grade RWE generation. In our “ICPE research spotlight” series, we share what you need to know about each, including important takeaways, a breakdown of the study design, and how the work shines new light on the ways RWE can inform decision-making.

At the same time that stakeholders across health care are realizing the expanding applications for RWE, epidemiologists, data scientists, and observational researchers are developing innovative methods to facilitate and streamline their work. 

In the following six posters, all prepared by Aetion scientists using the Aetion Evidence Platform® for ICPE All Access, we share explorations into new methods to advance RWE research.

Read on to learn more about these methodologies, including a machine learning technique that can build on propensity scoring to more completely adjust for confounding, an assessment of optimal caliper sizes, a comparison of methods for estimating annualized health care costs, a comparison of international medical coding practices, and a look at how to include uninsured individuals in real-world data (RWD) research. 

A neural-network driven longitudinal propensity score for evaluation of drug treatment effects
Jeremy Rassen, Sc.D.

Traditionally, propensity scores summarize covariates measured over a set period of time, but we could achieve more nuanced results by considering a patient’s specific course of treatment, and how particular treatment patterns can impact results. 

This study explores whether a recurrent neural network—a machine learning technique using temporal sequences to predict an endpoint—can more completely adjust for confounding than a traditional propensity score in a comparative drug effectiveness question.

Dr. Rassen found that a longitudinal propensity score estimated with a recurrent neural network can indeed account for patients’ full course of care, and may help capture nuanced patterns from insurance claims, electronic health records (EHRs), or other longitudinal data to further minimize residual confounding. 

View the poster here

Caliper considerations for propensity score matching
Liz Garry, Ph.D., M.P.H.; Wes Eddings, Ph.D.; Aditya Rajan, M.P.H.; Mandy Patrick, M.S.; Nicolle Gatto, Ph.D.

Propensity score matching aims to improve covariate balance between the control and intervention groups in an RWD analysis. Researchers may differ the “strictness” in matching by varying the caliper; a smaller caliper yields a closer, more conservative match, but typically reduces the sample size and power to detect differences between groups.  

This study evaluates a set of calipers to determine how far one can adjust a caliper without creating an imbalance, or changing the interpretation of the estimate. The research team emulated the existing ROCKET-AF trial, which compares stroke or systemic stroke among two treatment options.

Covariate balance was achieved using a high caliper, which allowed for most exposed patients to be matched and yielded an estimate that would lead to the same regulatory decision as the clinical trial. The study also demonstrates that more research is needed to determine the optimal caliper for analyses, and shows that good covariate balance in RWE studies can be achieved with different caliper levels in propensity score matching. 

View the poster here. 

Comparing methods for estimating annualized costs using administrative claims
Mandy Kelly, M.S.; Kammy Kuang, M.P.H.; Liza Gibbs; Kaiyi Luo, M.S.; and Natalia Petruski-Ivleva, Ph.D.

Studies on health care costs often rely on administrative claims data, which researchers can analyze using a range of methods. However, the results of these studies may vary depending on the methods used to estimate annualized costs, and the ways they vary can be inconsistent across disease areas. These limitations are rarely discussed. 

This study compares three different methods used in health care resource utilization studies to estimate annual health care costs in two different chronic disease populations: women with endometriosis, and patients with type 2 diabetes and chronic kidney disease.

The research team found that different methods used to estimate health care costs will produce different estimates for the same patient population depending on the assumptions made about the data. This work demonstrates that, when reporting annualized costs, transparency of methodology is key, and sensitivity analyses of multiple approaches should be considered to understand the range of potential health care costs—and to avoid over- or underestimates.

View the poster here

Comparison of ICD-10 coding patterns between the United States and Japan
Trisha Prince, M.P.H.; Kammy Kuang, M.P.H.; and Emily Rubinstein, M.P.H.

This research set to investigate whether the differences in coding systems between the United States, which uses a modification on the World Health Organization’s ICD-10 diagnostic system, and Japan, which uses standard disease codes that can be mapped to ICD-10 codes within administrative claims, drive different distributions in the ICD-10 codes employed when characterizing patients with prevalent cardiovascular conditions. 

Using MarketScan and Japanese Medical Data Center (JMDC) data, the research team found that diagnoses are more commonly coded using subclassification codes than unspecified diagnoses in MarketScan compared to JMDC, and that MarketScan offers more granular information than JMDC for ICD-10 diagnoses. 

Ultimately, this work concluded that researchers should understand and account for the unique attributes of a region’s data when looking at RWD across locations, and the implications that these differences in coding can have on the RWE used for decision-making. 

View the poster here

Studying uninsured Americans through the lens of survey data and electronic health records
Dawn Albright, M.S.; Elizabeth Dabrowski, M.S.; Liza Gibbs; Dana Teltsch, Ph.D.

U.S. based administrative claims databases, by definition, do not contain information about people without health insurance. Therefore, these individuals are excluded from most RWE studies. EHRs represent one way to mitigate this issue of inclusivity in clinical research; a physician won’t submit a claim to an insurance company for a patient without insurance, but they will still document the patient’s condition in their EHR. 

This study assesses how results from Optum’s EHR compare to the Medical Expenditure Panel Survey, a data source commonly used to study those without insurance, in order to understand the most fit-for-purpose database for research questions related to the American uninsured population. 

View the poster here

Insurance journeys: Quantifying transitions through insurance types
Dawn Albright, M.S.; Elizabeth Dabrowski, M.S.; Liza Gibbs; Dana Teltsch, Ph.D.

In addition to providing data on uninsured individuals, EHRs also contain information on insurance types that can illustrate how people move among insurance coverage statuses over time. 

For this study, researchers generated Sankey plots to visualize these insurance journeys, with a focus on populations with major depressive disorder and rheumatoid arthritis. The research team found that many partially insured or uninsured people continued to be partially or uninsured throughout the study period. This is important given that certain conditions—particularly those that impact one’s quality of life and productivity, and therefore may affect their employment status—may be correlated with a lack of health insurance. Understanding how to include these individuals in clinical research could offer a more complete picture of the course of certain diseases, such as those included in this study. 

View the poster here