Model-based control of plasma glycemia: Tests on populations of virtual patients

Subcutaneous insulin administration by deep reinforcement learning for blood glucose level control of type-2 diabetic patients

BACKGROUND

Type-2 diabetes mellitus is characterized by insulin resistance and impaired insulin secretion in the human body. Many endeavors have been made in terms of controlling and reducing blood glucose via the medium of automated controlling tools to increase precision and efficiency and reduce human error. Recently, reinforcement learning algorithms are proved to be powerful in the field of intelligent control, which was the motivation for the current study.

METHODS

For the first time, a reinforcement algorithm called normalized advantage function (NAF) algorithm has been applied as a model-free reinforcement learning method to regulate the blood glucose level of type-2 diabetic patients through subcutaneous injection. The algorithm has been designed and developed in a model-free approach to avoid additional inaccuracies and parameter uncertainty introduced by the mathematical models of the glucoregulatory system. Insulin doses constitute the control action that is designed to be stated directly in clinical language with the unit IU. In this regard, a new environment state is considered in addition to the glucose level to take into account the delayed effect of insulin elimination under the skin. Finally, a simple but practical reward function is developed to be used with the NAF algorithm to correct the glucose level and maintain it in the desired range.

RESULTS

The simulation environment was set up to imitate the basal-bolus process accurately. Results for 30 days of simulation of the designed controller on three different average virtual patients verify the feasibility and effectiveness of the method and reveal our proposed controller's learning ability. Moreover, as the insulin elimination dynamic was taken into account, a more complete and more realistic model than the previously studied models has emerged.

CONCLUSION

NAF has proved a promising control approach, able to successfully regulate and significantly reduce the fluctuation of the blood glucose without meal announcements, compared to standard optimized open-loop basal-bolus therapies. The method and its results, which are directly in the clinical language, are applicable in real-time clinical situations.

A mathematical model of food intake

The metabolic, hormonal and psychological determinants of the feeding behavior in humans are numerous and complex. A plausible model of the initiation, continuation and cessation of meals taking into account the most relevant such determinants would be very useful in simulating food intake over hours to days, thus providing input into existing models of nutrient absorption and metabolism. In the present work, a meal model is proposed, incorporating stomach distension, glycemic variations, ghrelin dynamics, cultural habits and influences on the initiation and continuation of meals, reflecting a combination of hedonic and appetite components. Given a set of parameter values (portraying a single subject), the timing and size of meals are stochastic. The model parameters are calibrated so as to reflect established medical knowledge on data of food intake from the National Health and Nutrition Examination Survey (NHANES) database during years 2015 and 2016.

A revised Sorensen model: Simulating glycemic and insulinemic response to oral and intra-venous glucose load

In 1978, Thomas J. Sorensen defended a thesis in chemical engineering at the University of California, Berkeley, where he proposed an extensive model of glucose-insulin control, model which was thereafter widely employed for virtual patient simulation. The original model, and even more so its subsequent implementations by other Authors, presented however a few imprecisions in reporting the correct model equations and parameter values. The goal of the present work is to revise the original Sorensen's model, to clearly summarize its defining equations, to supplement it with a missing gastrio-intestinal glucose absorption and to make an implementation of the revised model available on-line to the scientific community.

A novel fast-slow model of diabetes progression: Insights into mechanisms of response to the interventions in the Diabetes Prevention Program

Several models for the long-term development of T2DM already exist, focusing on the dynamics of the interaction between glycemia, insulinemia and β-cell mass. Current models consider representative (fasting or daily average) glycemia and insulinemia as characterizing the compensation state of the subject at some instant in slow time. This implies that only these representative levels can be followed through time and that the role of fast glycemic oscillations is neglected. An improved model (DPM15) for the long-term progression of T2DM is proposed, introducing separate peripheral and hepatic (liver and kidney) insulin actions. The DPM15 model no longer uses near-equilibrium approximation to separate fast and slow time scales, but rather describes, at each step in slow time, a complete day in the life of the virtual subject in fast time. The model can thus represent both fasting and postprandial glycemic levels and describe the effect of interventions acting on insulin-enhanced tissue glucose disposal or on insulin-inhibited hepatic glucose output, as well as on insulin secretion and β-cell replicating ability. The model can simulate long-term variations of commonly used clinical indices (HOMA-B, HOMA-IR, insulinogenic index) as well as of Oral Glucose Tolerance or Euglycemic Hyperinsulinemic Clamp test results. The model has been calibrated against observational data from the Diabetes Prevention Program study: it shows good adaptation to observations as a function of very plausible values of the parameters describing the effect of such interventions as Placebo, Intensive LifeStyle and Metformin administration.

Consistency of compact and extended models of glucose-insulin homeostasis: The role of variable pancreatic reserve

Published compact and extended models of the glucose-insulin physiologic control system are compared, in order to understand why a specific functional form of the compact model proved to be necessary for a satisfactory representation of acute perturbation experiments such as the Intra Venous Glucose Tolerance Test (IVGTT). A spectrum of IVGTT’s of virtual subjects ranging from normal to IFG to IGT to frank T2DM were simulated using an extended model incorporating the population-of-controllers paradigm originally hypothesized by Grodsky, and proven to be able to capture a wide array of experimental results from heterogeneous perturbation procedures. The simulated IVGTT’s were then fitted with the Single-Delay Model (SDM), a compact model with only six free parameters, previously shown to be very effective in delivering precise estimates of insulin sensitivity and secretion during an IVGTT. Comparison of the generating, extended-model parameter values with the obtained compact model estimates shows that the functional form of the nonlinear insulin-secretion term, empirically found to be necessary for the compact model to satisfactorily fit clinical observations, captures the pancreatic reserve level of the simulated virtual patients. This result supports the validity of the compact model as a meaningful analysis tool for the clinical assessment of insulin sensitivity.

The Virtual Anemia Trial: An Assessment of Model-Based In Silico Clinical Trials of Anemia Treatment Algorithms in Patients With Hemodialysis

In silico approaches have been proposed as a novel strategy to increase the repertoire of clinical trial designs. Realistic simulations of clinical trials can provide valuable information regarding safety and limitations of treatment protocols and have been shown to assist in the cost‐effective planning of clinical studies. In this report, we present a blueprint for the stepwise integration of internal, external, and ecological validity considerations in virtual clinical trials (VCTs). We exemplify this approach in the context of a model‐based in silico clinical trial aimed at anemia treatment in patients undergoing hemodialysis (HD). Hemoglobin levels and subsequent anemia treatment were simulated on a per patient level over the course of a year and compared to real‐life clinical data of 79,426 patients undergoing HD. The novel strategies presented here, aimed to improve external and ecological validity of a VCT, significantly increased the predictive power of the discussed in silico trial.