Feasibility of capturing real-world data from health information technology systems at multiple centers to assess cardiac ablation device outcomes: A fit-for-purpose informatics analysis report

Validation and clinical discovery demonstration of breast cancer data from a real-world data extraction platform

Objective

To validate and demonstrate the clinical discovery utility of a novel patient-mediated, medical record collection and data extraction platform developed to improve access and utilization of real-world clinical data.

Materials and Methods

Clinical variables were extracted from the medical records of 1011 consented patients with breast cancer. To validate the extracted data, case report forms completed using the structured data output of the platform were compared to manual chart review for 50 randomly-selected patients with metastatic breast cancer. To demonstrate the platform's clinical discovery utility, we identified 194 patients with early-stage clinical data who went on to develop distant metastases and utilized the platform-extracted data to assess associations between time to distant metastasis (TDM) and early-stage tumor histology, molecular type, and germline BRCA status.

Results

The platform-extracted data for the validation cohort had 97.6% precision (91.98%-100% by variable type) and 81.48% recall (58.15%-95.00% by variable type) compared to manual chart review. In our discovery cohort, the shortest TDM was significantly associated with metaplastic (739.0 days) and inflammatory histologies (1005.8 days), HR-/HER2- molecular types (1187.4 days), and positive BRCA status (1042.5 days) as compared to other histologies, molecular types, and negative BRCA status, respectively. Multivariable analyses did not produce statistically significant results.

Discussion

The precision and recall of platform-extracted clinical data are reported, although specificity could not be assessed. The data can generate clinically-relevant insights.

Conclusion

The structured real-world data produced by a novel patient-mediated, medical record-extraction platform are reliable and can power clinical discovery.

Predictive utility of the impedance drop on AF recurrence using digital intraprocedural data linked to electronic health record data

Background

Local impedance drop in cardiac tissue during catheter ablation may be a valuable measure to guide atrial fibrillation (AF) ablation procedures for greater effectiveness.

Objective

The study sought to assess whether local impedance drop during catheter ablation to treat AF predicts 1-year AF recurrence and what threshold of impedance drop is most predictive.

Methods

We identified patients with AF undergoing catheter ablation in the Mercy healthcare system. We downloaded AF ablation procedural data recorded by the CARTO system from a cloud-based analytical tool (CARTONET) and linked them to individual patient electronic health records. Average impedance drops in anatomical region of right and left pulmonary veins were calculated. Effectiveness was measured by a composite outcome of repeat ablation, AF rehospitalization, direct current cardioversion, or initialization of a new antiarrhythmic drug post-blanking period. The association between impedance drop and 1-year AF recurrence was assessed by logistic regression adjusting for demographics, clinical, and ablation characteristics. Bootstrapping was used to determine the most predictive threshold for impedance drop based on the Youden index.

Results

Among 242 patients, 23.6% (n = 57) experienced 1-year AF recurrence. Patients in the lower third vs upper third of average impedance drop had a 5.9-fold (95% confidence interval [CI] 1.81-21.8) higher risk of recurrence (37.0% vs 12.5%). The threshold of 7.2 Ω (95% CI 5.75-7.7 Ω) impedance drop best predicted AF recurrence, with sensitivity of 0.73 and positive predictive value of 0.33. Patients with impedance drop ≤7.2 Ω had 3.5-fold (95% CI 1.39-9.50) higher risk of recurrence than patients with impedance drop >7.2 Ω, and there was no statistical difference in adverse events between the 2 groups of patients. Sensitivity analysis on right and left wide antral circumferential ablation impedance drop was consistent.

Conclusion

Average impedance drop is a strong predictor of clinical success in reducing AF recurrence but as a single criterion for predicting recurrence only reached 73% sensitivity and 33% positive predictive value.

Using real-world data from health systems to evaluate the safety and effectiveness of a catheter to treat ischemic ventricular tachycardia

BACKGROUND

The ThermoCool STSF catheter is used for ablation of ischemic ventricular tachycardia (VT) in routine clinical practice, although outcomes have not been studied and the catheter does not have Food and Drug Administration (FDA) approval for this indication. We used real-world health system data to evaluate its safety and effectiveness for this indication.

METHODS

Among patients undergoing ischemic VT ablation with the ThermoCool STSF catheter pooled across two health systems (Mercy Health and Mayo Clinic), the primary safety composite outcome of death, thromboembolic events, and procedural complications within 7 days was compared to a performance goal of 15%, which is twice the expected proportion of the primary composite safety outcome based on prior studies. The exploratory effectiveness outcome of rehospitalization for VT or heart failure or repeat VT ablation at up to 1 year was averaged across health systems among patients treated with the ThermoCool STSF vs. ST catheters.

RESULTS

Seventy total patients received ablation for ischemic VT using the ThermoCool STSF catheter. The primary safety composite outcome occurred in 3/70 (4.3%; 90% CI, 1.2-10.7%) patients, meeting the pre-specified performance goal, p = 0.0045. At 1 year, the effectiveness outcome risk difference (STSF-ST) at Mercy was - 0.4% (90% CI: - 25.2%, 24.3%) and at Mayo Clinic was 12.6% (90% CI: - 13.0%, 38.4%); the average risk difference across both institutions was 5.8% (90% CI: - 12.0, 23.7).

CONCLUSIONS

The ThermoCool STSF catheter was safe and appeared effective for ischemic VT ablation, supporting continued use of the catheter and informing possible FDA label expansion. Health system data hold promise for real-world safety and effectiveness evaluation of cardiovascular devices.

Endovascular Aneurysm Repair Devices as a Use Case for Postmarketing Surveillance of Medical Devices

Importance

The US Food and Drug Administration (FDA) is building a national postmarketing surveillance system for medical devices, moving to a "total product life cycle" approach whereby more limited premarketing data are balanced with postmarketing surveillance to capture rare adverse events and long-term safety issues.

Objective

To assess the methodological requirements and feasibility of postmarketing device surveillance using endovascular aneurysm repair devices (EVARs), which have been the subject of safety concerns, using clinical data from a large health care system.

Design, Setting, and Participants

This retrospective cohort study included patients with electronic health record (EHR) data in the Veterans Affairs Corporate Data Warehouse.

Exposure

Implantation of an AFX Endovascular AAA System (AFX) device (any of 3 iterations) or a non-AFX comparator EVAR device from January 1, 2011, to December 21, 2021.

Main Outcomes and Measures

The primary outcomes were rates of type III endoleaks and all-cause mortality; and rates of these outcomes associated with AFX devices compared with non-AFX devices, assessed using Cox proportional hazards regression models and doubly robust causal modeling. Information on type III endoleaks was available only as free-text mentions in clinical notes, while all-cause mortality data could be extracted using structured data. Device-specific information required by the FDA is ascertained using unique device identifiers (UDIs), which include factors such as model numbers, catalog numbers, and manufacturer-specific product codes. The availability of UDIs in EHRs was assessed.

Results

In total, 13 941 patients (mean [SD] age, 71.8 [7.4] years) received 1 of the devices of interest (AFX with Strata [AFX-S]: 718 patients [5.2%]; AFX with Duraply [AFX-D]: 404 patients [2.9%]; or AFX2: 682 patients [4.9%]), and 12 137 (87.1%) received non-AFX devices. The UDIs were not recorded in the EHR for any patient with an AFX device, and partial UDIs were available for 19 patients (0.1%) with a non-AFX device. This necessitated the development of advanced natural language processing tools to define the cohort of patients for analysis. The study identified a significantly higher risk of type III endoleaks at 5 years among patients receiving any of the AFX device iterations, including the most recent version, AFX2 (11.6%; 95% CI, 8.1%-15.1%) compared with that among patients with non-AFX devices (5.7%; 95% CI, 2.2%-9.2%; absolute risk difference, 5.9%; 95% CI, 2.3%-9.4%). However, there was no significantly higher all-cause mortality for any of the AFX device iterations, including for AFX2 (19.0%; 95% CI, 16.0%-22.0%) compared with non-AFX devices (18.0%; 95% CI, 15.0%-21.0%; absolute risk difference, 1.0%; 95% CI, -2.1% to 4.1%).

Conclusions and Relevance

The findings of this cohort study suggest that clinical data can be used for the postmarketing device surveillance required by the FDA. The study also highlights ongoing challenges to performing larger-scale surveillance, including lack of consistent use of UDIs and insufficient relevant structured data to efficiently capture certain outcomes of interest.

Integrating real-world data to assess cardiac ablation device outcomes in a multicenter study using the OMOP common data model for regulatory decisions: implementation and evaluation

The objective of this study is to describe application of the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) to support medical device real-world evaluation in a National Evaluation System for health Technology Coordinating Center (NESTcc) Test-Case involving 2 healthcare systems, Mercy Health and Mayo Clinic. CDM implementation was coordinated across 2 healthcare systems with multiple hospitals to aggregate both medical device data from supply chain databases and patient outcomes and covariates from electronic health record data. Several data quality assurance (QA) analyses were implemented on the OMOP CDM to validate the data extraction, transformation, and load (ETL) process. OMOP CDM-based data of relevant patient encounters were successfully established to support studies for FDA regulatory submissions. QA analyses verified that the data transformation was robust between data sources and OMOP CDM. Our efforts provided useful insights in real-world data integration using OMOP CDM for medical device evaluation coordinated across multiple healthcare systems.

Exploring unique device identifier implementation and use for real-world evidence: a mixed-methods study with NESTcc health system network collaborators

Objectives

To examine the current state of unique device identifier (UDI) implementation, including barriers and facilitators, among eight health systems participating in a research network committed to real-world evidence (RWE) generation for medical devices.

Design

Mixed methods, including a structured survey and semistructured interviews.

Setting

Eight health systems participating in the National Evaluation System for health Technology research network within the USA.

Participants

Individuals identified as being involved in or knowledgeable about UDI implementation or medical device identification from supply chain, information technology and high-volume procedural area(s) in their health system.

Main outcomes measures

Interview topics were related to UDI implementation, including barriers and facilitators; UDI use; benefits of UDI adoption; and vision for UDI implementation. Data were analysed using directed content analysis, drawing on prior conceptual models of UDI implementation and the Exploration, Preparation, Implementation, Sustainment framework. A brief survey of health system characteristics and scope of UDI implementation was also conducted.

Results

Thirty-five individuals completed interviews. Three of eight health systems reported having implemented UDI. Themes identified about barriers and facilitators to UDI implementation included knowledge of the UDI and its benefits among decision-makers; organisational systems, culture and networks that support technology and workflow changes; and external factors such as policy mandates and technology. A final theme focused on the availability of UDIs for RWE; lack of availability significantly hindered RWE studies on medical devices.

Conclusions

UDI adoption within health systems requires knowledge of and impetus to achieve operational and clinical benefits. These are necessary to support UDI availability for medical device safety and effectiveness studies and RWE generation.

Safety and Effectiveness of a Catheter With Contact Force and 6-Hole Irrigation for Ablation of Persistent Atrial Fibrillation in Routine Clinical Practice

IMPORTANCE

The ThermoCool SmartTouch catheter (ablation catheter with contact force and 6-hole irrigation [CF-I6]) is approved by the US Food and Drug Administration (FDA) for paroxysmal atrial fibrillation (AF) ablation and used in routine clinical practice for persistent AF ablation, although clinical outcomes for this indication are unknown. There is a need to understand whether data from routine clinical practice can be used to conduct regulatory-grade evaluations and support label expansions.

OBJECTIVE

To use health system data to compare the safety and effectiveness of the CF-I6 catheter for persistent AF ablation with the ThermoCool SmartTouch SurroundFlow catheter (ablation catheter with contact force and 56-hole irrigation [CF-I56]), which is approved by the FDA for this indication.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective, comparative-effectiveness cohort study included patients undergoing catheter ablation for persistent AF at Mercy Health or Mayo Clinic from January 1, 2014, to April 30, 2021, with up to a 1-year follow-up using electronic health record data.

EXPOSURES

Use of the CF-I6 or CF-I56 catheter.

MAIN OUTCOMES AND MEASURES

The primary safety outcome was a composite of death, thromboembolic events, and procedural complications within 7 to 90 days. The exploratory effectiveness outcome was a composite of AF-related hospitalization events after a 90-day blanking period. Propensity score weighting was used to balance baseline covariates. Risk differences were estimated between catheter groups and averaged across the 2 health care systems, testing for noninferiority of the CF-I6 vs the CF-I56 catheter with respect to the safety outcome using 2-sided 90% CIs.

RESULTS

Overall, 1450 patients (1034 [71.3%] male; 1397 [96.3%] White) underwent catheter ablation for persistent AF, including 949 at Mercy Health (186 CF-I6 and 763 CF-I56; mean [SD] age, 64.9 [9.2] years) and 501 at Mayo Clinic (337 CF-I6 and 164 CF-I56; mean [SD] age, 63.7 [9.5] years). A total of 798 (55.0%) had been treated with class I or III antiarrhythmic drugs before ablation. The safety outcome (CF-I6 - CF-I56) was similar at both Mercy Health (1.3%; 90% CI, -2.1% to 4.6%) and Mayo Clinic (-3.8%; 90% CI, -11.4% to 3.7%); the mean difference was noninferior, with a mean of 0.5% (90% CI, -2.6% to 3.5%; P < .001). The effectiveness was similar at 12 months between the 2 catheter groups (mean risk difference, -1.8%; 90% CI, -7.3% to 3.7%).

CONCLUSIONS AND RELEVANCE

In this cohort study, the CF-I6 catheter met the prespecified noninferiority safety criterion for persistent AF ablation compared with the CF-I56 catheter, and effectiveness was similar. This study demonstrates the ability of electronic health care system data to enable safety and effectiveness evaluations of medical devices.

Advancing Patient Safety Surrounding Medical Devices: Barriers, Strategies, and Next Steps in Health System Implementation of Unique Device Identifiers

Background

The requirement for medical device manufacturers to label their devices with a unique device identifier (UDI) was formalized by the 2013 US Food and Drug Administration Unique Device Identification System Rule. However, parallel regulatory requirement for US health systems to use UDIs, particularly the electronic documentation of UDIs during patient care is lacking. Despite the lack of regulation, some health systems have implemented and are using UDIs. To assess the current state, we studied representative health system UDI implementation experiences, including barriers and the strategies to overcome them, and identified next steps to advance UDI adoption.

Methods

Semi-structured interviews were performed with health system personnel involved in UDI implementation in their cardiac catheterization labs or operating rooms. Interviews were transcribed and analyzed using the framework methodology of Ritchie and Spencer. An expert panel evaluated findings and informed barriers, strategies, and next steps.

Results

Twenty-four interviews at ten health systems were performed. Identified barriers were internal (lack of organizational support, information technology gaps, clinical resistance) and external (information technology vendor resistance, limitations in manufacturer support, gaps in reference data, lack of an overall UDI system). Identified strategies included relationship building, education, engagement, and communication. Next steps to advance UDI adoption focus on education, research, support, and policy.

Conclusions and Implications

Delineation of UDI implementation barriers and strategies provides guidance and support for health systems to adopt the UDI standard and electronically document UDIs during clinical care. Next steps illuminate critical areas for attention to advance UDI adoption and achieve a comprehensive UDI system in health care to strengthen patient care and safety.

Synergies between centralized and federated approaches to data quality: a report from the national COVID cohort collaborative

OBJECTIVE

In response to COVID-19, the informatics community united to aggregate as much clinical data as possible to characterize this new disease and reduce its impact through collaborative analytics. The National COVID Cohort Collaborative (N3C) is now the largest publicly available HIPAA limited dataset in US history with over 6.4 million patients and is a testament to a partnership of over 100 organizations.

MATERIALS AND METHODS

We developed a pipeline for ingesting, harmonizing, and centralizing data from 56 contributing data partners using 4 federated Common Data Models. N3C data quality (DQ) review involves both automated and manual procedures. In the process, several DQ heuristics were discovered in our centralized context, both within the pipeline and during downstream project-based analysis. Feedback to the sites led to many local and centralized DQ improvements.

RESULTS

Beyond well-recognized DQ findings, we discovered 15 heuristics relating to source Common Data Model conformance, demographics, COVID tests, conditions, encounters, measurements, observations, coding completeness, and fitness for use. Of 56 sites, 37 sites (66%) demonstrated issues through these heuristics. These 37 sites demonstrated improvement after receiving feedback.

DISCUSSION

We encountered site-to-site differences in DQ which would have been challenging to discover using federated checks alone. We have demonstrated that centralized DQ benchmarking reveals unique opportunities for DQ improvement that will support improved research analytics locally and in aggregate.

CONCLUSION

By combining rapid, continual assessment of DQ with a large volume of multisite data, it is possible to support more nuanced scientific questions with the scale and rigor that they require.

JP. Feasibility of using real-world data in the evaluation of cardiac ablation catheters: a test-case of the National Evaluation System for Health Technology Coordinating Center

OBJECTIVES

To determine the feasibility of using real-world data to assess the safety and effectiveness of two cardiac ablation catheters for the treatment of persistent atrial fibrillation and ischaemic ventricular tachycardia.

DESIGN

Retrospective cohort.

SETTING

Three health systems in the USA.

PARTICIPANTS

Patients receiving ablation with the two ablation catheters of interest at any of the three health systems.

MAIN OUTCOME MEASURES

Feasibility of identifying the medical devices and participant populations of interest as well as the duration of follow-up and positive predictive values (PPVs) for serious safety (ischaemic stroke, acute heart failure and cardiac tamponade) and effectiveness (arrhythmia-related hospitalisation) clinical outcomes of interest compared with manual chart validation by clinicians.

RESULTS

Overall, the catheter of interest for treatment of persistent atrial fibrillation was used for 4280 ablations and the catheter of interest for ischaemic ventricular tachycardia was used 1516 times across the data available within the three health systems. The duration of patient follow-up in the three health systems ranged from 91% to 97% at ≥7 days, 89% to 96% at ≥30 days, 77% to 90% at ≥6 months and 66% to 84% at ≥1 year. PPVs were 63.4% for ischaemic stroke, 96.4% for acute heart failure, 100% at one health system for cardiac tamponade and 55.7% for arrhythmia-related hospitalisation.

CONCLUSIONS

It is feasible to use real-world health system data to evaluate the safety and effectiveness of cardiac ablation catheters, though evaluations must consider the implications of variation in follow-up and endpoint ascertainment among health systems.