Accuracy assessment methods for physiological model selection toward evaluation of closed-loop controlled medical devices

A GENERATIVE APPROACH TO TESTING THE PERFORMANCE OF PHYSIOLOGICAL CONTROL ALGORITHMS

Background

Physiological closed-loop control algorithms play an important role in the development of autonomous medical care systems, a promising area of research that has the potential to deliver healthcare therapies meeting each patient's specific needs. Computational approaches can support the evaluation of physiological closed-loop control algorithms considering various sources of patient variability that they may be presented with.

Method of Approach

In this paper, we present a generative approach to testing the performance of physiological closed-loop control algorithms. This approach exploits a generative physiological model (which consists of stochastic and dynamic components that represent diverse physiological behaviors across a patient population) to generate a select group of virtual subjects. By testing a physiological closed-loop control algorithm against this select group, the approach estimates the distribution of relevant performance metrics in the represented population. We illustrate the promise of this approach by applying it to a practical case study on testing a closed-loop fluid resuscitation control algorithm designed for hemodynamic management.

Results

In this context, we show that the proposed approach can test the algorithm against virtual subjects equipped with a wide range of plausible physiological characteristics and behavior, and that the test results can be used to estimate the distribution of relevant performance metrics in the represented population.

Conclusions

In sum, the generative testing approach may offer a practical, efficient solution for conducting pre-clinical tests on physiological closed-loop control algorithms.

A Lumped-Parameter Model of the Cardiovascular System Response for Evaluating Automated Fluid Resuscitation Systems

Physiological closed-loop controlled (PCLC) medical devices, such as those designed for blood pressure regulation, can be tested for safety and efficacy in real-world clinical settings. However, relying solely on limited animal and clinical studies may not capture the diverse range of physiological conditions. Credible mathematical models can complement these studies by allowing the testing of the device against simulated patient scenarios. This research involves the development and validation of a low-order lumped-parameter mathematical model of the cardiovascular system's response to fluid perturbation. The model takes rates of hemorrhage and fluid infusion as inputs and provides hematocrit and blood volume, heart rate, stroke volume, cardiac output and mean arterial blood pressure as outputs. The model was calibrated using data from 27 sheep subjects, and its predictive capability was evaluated through a leave-one-out cross-validation procedure, followed by independent validation using 12 swine subjects. Our findings showed small model calibration error against the training dataset, with the normalized root-mean-square error (NRMSE) less than 10% across all variables. The mathematical model and virtual patient cohort generation tool demonstrated a high level of predictive capability and successfully generated a sufficient number of subjects that closely resembled the test dataset. The average NRMSE for the best virtual subject, across two distinct samples of virtual subjects, was below 12.7% and 11.9% for the leave-one-out cross-validation and independent validation dataset. These findings suggest that the model and virtual cohort generator are suitable for simulating patient populations under fluid perturbation, indicating their potential value in PCLC medical device evaluation.

Development and validation of a mathematical model of heart rate response to fluid perturbation

Physiological closed-loop controlled (PCLC) medical devices monitor and automatically adjust the patient's condition by using physiological variables as feedback, ideally with minimal human intervention to achieve the target levels set by a clinician. PCLC devices present a challenge when it comes to evaluating their performance, where conducting large clinical trials can be expensive. Virtual physiological patients simulated by validated mathematical models can be utilized to obtain pre-clinical evidence of safety and assess the performance of the PCLC medical device during normal and worst-case conditions that are unlikely to happen in a limited clinical trial. A physiological variable that plays a major role during fluid resuscitation is heart rate (HR). For in silico assessment of PCLC medical devices regarding fluid perturbation, there is currently no mathematical model of HR validated in terms of its predictive capability performance. This paper develops and validates a mathematical model of HR response using data collected from sheep subjects undergoing hemorrhage and fluid infusion. The model proved to be accurate in estimating the HR response to fluid perturbation, where averaged between 21 calibration datasets, the fitting performance showed a normalized root mean square error (NRMSE) of [Formula: see text]. The model was also evaluated in terms of model predictive capability performance via a leave-one-out procedure (21 subjects) and an independent validation dataset (6 subjects). Two different virtual cohort generation tools were used in each validation analysis. The generated envelope of virtual subjects robustly met the defined acceptance criteria, in which [Formula: see text] of the testing datasets presented simulated HR patterns that were within a deviation of 50% from the observed data. In addition, out of 16000 and 18522 simulated subjects for the leave-one-out and independent datasets, the model was able to generate at least one virtual subject that was close to the real subject within an error margin of [Formula: see text] and [Formula: see text] NRMSE, respectively. In conclusion, the model can generate valid virtual HR physiological responses to fluid perturbation and be incorporated into future non-clinical simulated testing setups for assessing PCLC devices intended for fluid resuscitation.

Closed-Loop Controlled Fluid Administration Systems: A Comprehensive Scoping Review

Physiological Closed-Loop Controlled systems continue to take a growing part in clinical practice, offering possibilities of providing more accurate, goal-directed care while reducing clinicians’ cognitive and task load. These systems also provide a standardized approach for the clinical management of the patient, leading to a reduction in care variability across multiple dimensions. For fluid management and administration, the advantages of closed-loop technology are clear, especially in conditions that require precise care to improve outcomes, such as peri-operative care, trauma, and acute burn care. Controller design varies from simplistic to complex designs, based on detailed physiological models and adaptive properties that account for inter-patient and intra-patient variability; their maturity level ranges from theoretical models tested in silico to commercially available, FDA-approved products. This comprehensive scoping review was conducted in order to assess the current technological landscape of this field, describe the systems currently available or under development, and suggest further advancements that may unfold in the coming years. Ten distinct systems were identified and discussed.