A simple signaling rule for variable life-adjusted display derived from an equivalent risk-adjusted CUSUM chart

Risk-adjusted observed minus expected cumulative sum (RA O-E CUSUM) chart for visualisation and monitoring of surgical outcomes

To improve patient safety, surgeons can continually monitor the surgical outcomes of their patients. To this end, they can use statistical process control tools, which primarily originated in the manufacturing industry and are now widely used in healthcare. These tools belong to a broad family, making it challenging to identify the most suitable methodology to monitor surgical outcomes. The selected tools must balance statistical rigour with surgeon usability, enabling both statistical interpretation of trends over time and comprehensibility for the surgeons, their primary users. On one hand, the observed minus expected (O-E) chart is a simple and intuitive tool that allows surgeons without statistical expertise to view and interpret their activity; however, it may not possess the sophisticated algorithms required to accurately identify important changes in surgical performance. On the other hand, a statistically robust tool like the cumulative sum (CUSUM) method can be helpful but may be too complex for surgeons to interpret and apply in practice without proper statistical training. To address this issue, we developed a new risk-adjusted (RA) O-E CUSUM chart that aims to provide a balanced solution, integrating the visualisation strengths of a user-friendly O-E chart with the statistical interpretation capabilities of a CUSUM chart. With the RA O-E CUSUM chart, surgeons can effectively monitor patients' outcomes and identify sequences of statistically abnormal changes, indicating either deterioration or improvement in surgical outcomes. They can also quantify potentially preventable or avoidable adverse events during these sequences. Subsequently, surgical teams can try implementing changes to potentially improve their performance and enhance patient safety over time. This paper outlines the methodology for building the tool and provides a concrete example using real surgical data to demonstrate its application.

A monitoring framework for health care processes using Generalized Additive Models and Auto-Encoders

In recent years, there has been a considerable focus on developing effective methods for monitoring health care processes. Utilizing Statistical Process Monitoring (SPM) approaches, particularly risk-adjusted control charts, has emerged as a highly promising approach for achieving robust frameworks for this aim. Considering risk-adjusted control charts, longitudinal health care process data is typically monitored by establishing a regression relationship between various risk factors (explanatory variables) and patient outcomes (response variables). While the majority of prior research has primarily employed logistic models in risk-adjusted control charts, there are more intricate health care processes that necessitate the incorporation of both parametric and nonparametric risk factors. In such scenarios, the Generalized Additive Model (GAM) proves to be a suitable choice, albeit it often introduces higher computational complexity and associated challenges. Surprisingly, there are limited instances where researchers have proposed advancements in this direction. The primary objective of this paper is to introduce an SPM framework for monitoring health care processes using a GAM over time, coupled with a novel risk-adjusted control chart driven by machine learning techniques. This control chart is implemented on a data set encompassing two stroke types: ischemic and hemorrhagic. The key focus of this study is to monitor the stability of the relationship between stroke types and predefined explanatory variables over time within this data set. Extensive simulation results, based on real data from patients with acute stroke, demonstrate the remarkable flexibility of the proposed method in terms of its detection capabilities compared to conventional approaches.

Modeling the patient mix for risk-adjusted CUmulative SUM charts

The improvement of surgical quality and the corresponding early detection of its changes is of increasing importance. To this end, sequential monitoring procedures such as the risk-adjusted CUmulative SUM chart are frequently applied. The patient risk score population (patient mix), which considers the patients’ perioperative risk, is a core component for this type of quality control chart. Consequently, it is important to be able to adapt different shapes of patient mixes and determine their impact on the monitoring scheme. This article proposes a framework for modeling the patient mix by a discrete beta-binomial and a continuous beta distribution for risk-adjusted CUSUM charts. Since the model-based approach is not limited by data availability, any patient mix can be analyzed. We examine the effects on the control chart’s false alarm behavior for more than 100,000 different scenarios for a cardiac surgery data set. Our study finds a negative relationship between the average risk score and the number of false alarms. The results indicate that a changing patient mix has a considerable impact and, in some cases, almost doubles the number of expected false alarms.

Monitoring of high-yield and periodical processes in health care

Statistical control charts have found valuable applications in health care, having been largely adopted from operations research in manufacturing. However, the most common types are not best-suited to monitor high-yield processes (outcomes comprising true/false fractions, 'near-zero') and periodical processes (characterized by sequences of single populations of finite sizes), but rather to monitor variable vital signs levels and, to a lesser degree, service performance indicators. We discuss control charts that are most suitable for fraction non-conforming measurements. We focus particularly on high-yield and periodical processes, i.e. range in which out-of-control conditions are expected and should be identified. For these conditions, we discuss control charts based on the family of hypergeometric distributions, explaining and comparing their application to more traditional alternatives with two health care case studies. We demonstrate that hypergeometric-type control charts provide higher sensitivity in timely identification of changing rare event fractions and are well-suited for monitoring of periodical processes, while remaining more resistant to false alarms, versus their alternatives.

Quicker detection risk-adjusted cumulative sum charting procedures

When a patient is operated on, the surgical outcome depends on two major factors: (i) the patient's health condition and (ii) the surgical process comprising the surgeon, the supporting staff, operating environment, and equipment. An outcome is usually represented by one if a patient dies within 30 days of an operation and zero otherwise. Another method of measuring the outcome is to use survival time with truncation on the 30th day for monitoring purposes. In order to monitor a surgical process effectively, the health condition of a patient must be taken into consideration. This is usually done using a log-likelihood ratio statistic based on an outcome, that is, risk adjusted according to the health condition of the patient. The 30-day wait results in delay in signaling when a deterioration occurs. The consequence of having to wait even though a death has occurred is the potential loss of lives because of delay in signaling. Regular updating of patients' information can improve the sensitivity of a charting procedure. The main objective of this article is to develop and study the class of risk-adjusted cumulative sum procedures that are updated on a regular basis based on patients' current conditions, without having to wait 30 days. Our study shows that these charts do in fact signal earlier and there are differences among the various updating techniques and monitoring statistics.

Medical device active surveillance of spontaneous reports: A literature review of signal detection methods

PURPOSE

The collection and analysis of real-world data for the active monitoring of medical device performance and safety has become increasingly important. Spontaneous reports, such as those in the Food & Drug Administration's (FDA's) Manufacturer and User Facility Device Experience (MAUDE), provide early warning of potential issues with marketed devices. This review synthesizes the current literature on medical device surveillance signal detection and provides a framework for application of methods to active surveillance of spontaneous reports.

METHODS

Ovid MEDLINE, Ovid Embase, Scopus, and PubMed databases were systematically searched up to January 2019. Additionally, five methods articles from pharmacovigilance were added that had potential applications to medical devices.

RESULTS

Among 105 articles included, the most common source of data (84%) was registries; median time between data collection and publication was 8 years. Surgical procedure outcome signal detection articles comprised 83% while 14% were on device outcome signal detection. The most common family of methods cited (70%) was Sequential Probability Ratio.

CONCLUSION

Application of any signal detection algorithm requires careful consideration of influential factors, data limitations, and algorithmic assumptions. We recommend approaches using disproportionality, statistical process control, and sequential probability tests and provide R packages to further development efforts. The small number of published examples suggest that further development of statistical methods and technological solutions to analyze large amounts of data for device safety and performance is needed. Fundamental differences in products, data infrastructure, and the regulatory landscape suggest that medical device vigilance requires its own body of research distinct from pharmacovigilance.