Robust PID control of propofol anaesthesia: Uncertainty limits performance, not PID structure

Real-Time Personalised Pharmacokinetic-Pharmacodynamic Modelling in Propofol Anesthesia through Bayesian Inference

Pharmacological models describe a patient's response to the administration of a medicinal drug based on parameters derived from population studies. However, considerable inter-patient variability exists, such that population models may underperform when used to predict the actual response of a specific individual. In applications which demand predictive accuracy-such as target-controlled infusion of anesthetic agents-modeling uncertainty may reduce system dependability and introduce clinical risk. Our work investigates the use of Bayesian inference, implemented through a particle filter algorithm, to refine a prior model of propofol pharmacokinetics-pharmacodynamics and estimate patient-specific parameters in real-time. We report here on an observational clinical study conducted on 40 adults undergoing general anesthesia, where we evaluated the performance of Bayesian inference-personalized models in forecasting forward trends of depth of anesthesia (Bispectral Index) measurements and compared it with that of a traditional population-based pharmacological model. Our results show a significant reduction in prediction error metrics for the patient-specific models. Our study demonstrates the viability and practical implementability of Bayesian inference as a tool for real-time intra-operative estimation of personalized pharmacological models in anesthesia applications.

Improved Individualized Patient-Oriented Depth-of-Hypnosis Measurement Based on Bispectral Index

Total intravenous anesthesia is an anesthesiologic technique where all substances are injected intravenously. The main task of the anesthesiologist is to assess the depth of anesthesia, or, more specifically, the depth of hypnosis (DoH), and accordingly adjust the dose of intravenous anesthetic agents. However, it is not possible to directly measure the anesthetic agent concentrations or the DoH, so the anesthesiologist must rely on various vital signs and EEG-based measurements, such as the bispectral (BIS) index. The ability to better measure DoH is directly applicable in clinical practice-it improves the anesthesiologist's assessment of the patient state regarding anesthetic agent concentrations and, consequently, the effects, as well as provides the basis for closed-loop control algorithms. This article introduces a novel structure for modeling DoH, which employs a residual dynamic model. The improved model can take into account the patient's individual sensitivity to the anesthetic agent, which is not the case when using the available population-data-based models. The improved model was tested using real clinical data. The results show that the predictions of the BIS-index trajectory were improved considerably. The proposed model thus seems to provide a good basis for a more patient-oriented individualized assessment of DoH, which should lead to better administration methods that will relieve the anesthesiologist's workload and will benefit the patient by providing improved safety, individualized treatment, and, thus, alleviation of possible adverse effects during and after surgery.

A modified PID-based control scheme for depth-of-hypnosis control: Design and experimental results

BACKGROUND AND OBJECTIVE

Many methodologies have been proposed for the control of total intravenous anesthesia in general surgery, as this yields a reduced stress for the anesthesiologist and an increased safety for the patient. The objective of this work is to design a PID-based control system for the regulation of the depth of hypnosis by propofol and remifentanil coadministration that takes into account the clinical practice.

METHODS

With respect to a standard PID control system, additional functionalities have been implemented in order to consider specific requirements related to the clinical practice. In particular, suitable boluses are determined and used in the induction phase and a nonzero baseline infusion is used in the maintenance phase when the predicted effect-site concentration drops below a safety threshold.

RESULTS

The modified controller has been experimentally assessed on a group of 10 patients receiving general anesthesia for elective plastic surgery. The control system has been able to induce and maintain adequate anesthesia without any manual intervention from the anesthesiologist.

CONCLUSIONS

Results confirm the effectiveness of the overall design approach and, in particular, highlight that the new version of the control system, with respect to a standard PID controller, provides significant advantages from a clinical standpoint.

Continuous action deep reinforcement learning for propofol dosing during general anesthesia

PURPOSE

Anesthesiologists simultaneously manage several aspects of patient care during general anesthesia. Automating administration of hypnotic agents could enable more precise control of a patient's level of unconsciousness and enable anesthesiologists to focus on the most critical aspects of patient care. Reinforcement learning (RL) algorithms can be used to fit a mapping from patient state to a medication regimen. These algorithms can learn complex control policies that, when paired with modern techniques for promoting model interpretability, offer a promising approach for developing a clinically viable system for automated anesthestic drug delivery.

METHODS

We expand on our prior work applying deep RL to automated anesthetic dosing by now using a continuous-action model based on the actor-critic RL paradigm. The proposed RL agent is composed of a policy network that maps observed anesthetic states to a continuous probability density over propofol-infusion rates and a value network that estimates the favorability of observed states. We train and test three versions of the RL agent using varied reward functions. The agent is trained using simulated pharmacokinetic/pharmacodynamic models with randomized parameters to ensure robustness to patient variability. The model is tested on simulations and retrospectively on nine general anesthesia cases collected in the operating room. We utilize Shapley additive explanations to gain an understanding of the factors with the greatest influence over the agent's decision-making.

RESULTS

The deep RL agent significantly outperformed a proportional-integral-derivative controller (median episode median absolute performance error 1.9% ± 1.8 and 3.1% ± 1.1). The model that was rewarded for minimizing total doses performed the best across simulated patient demographics (median episode median performance error 1.1% ± 0.5). When run on real-world clinical datasets, the agent recommended doses that were consistent with those administered by the anesthesiologist.

CONCLUSIONS

The proposed approach marks the first fully continuous deep RL algorithm for automating anesthestic drug dosing. The reward function used by the RL training algorithm can be flexibly designed for desirable practices (e.g. use less anesthetic) and bolstered performances. Through careful analysis of the learned policies, techniques for interpreting dosing decisions, and testing on clinical data, we confirm that the agent's anesthetic dosing is consistent with our understanding of best-practices in anesthesia care.