Use of machine learning to identify patients at risk of sub-optimal adherence: study based on real-world data from 10,929 children using a connected auto-injector device

BACKGROUND

Our aim was to develop a machine learning model, using real-world data captured from a connected auto-injector device and from early indicators from the first 3 months of treatment, to predict sub-optimal adherence to recombinant human growth hormone (r-hGH) in patients with growth disorders.

METHODS

Adherence to r-hGH treatment was assessed in children (aged < 18 years) who started using a connected auto-injector device (easypod™), and transmitted injection data for ≥ 12 months. Adherence in the following 3, 6, or 9 months after treatment start was categorized as optimal (≥ 85%) versus sub-optimal (< 85%). Logistic regression and tree-based models were applied.

RESULTS

Data from 10,929 children showed that a random forest model with mean and standard deviation of adherence over the first 3 months, infrequent transmission of data, not changing certain comfort settings, and starting treatment at an older age was important in predicting the risk of sub-optimal adherence in the following 3, 6, or 9 months. Sensitivities ranged between 0.72 and 0.77, and specificities between 0.80 and 0.81.

CONCLUSIONS

To the authors' knowledge, this is the first attempt to integrate a machine learning model into a digital health ecosystem to help healthcare providers to identify patients at risk of sub-optimal adherence to r-hGH in the following 3, 6, or 9 months. This information, together with patient-specific indicators of sub-optimal adherence, can be used to provide support to at-risk patients and their caregivers to achieve optimal adherence and, subsequently, improve clinical outcomes.

Persistence of Use in Children Receiving Growth Hormone Therapy

Background: Long-term persistence of use and starting at the earliest possible age are associated with attainment of near-normal adult or final height in children who are treated with growth hormone therapy. However, there are many factors associated with a lack of persistence. Our aim was to study persistence of use in children with growth disorders using the easypod™ connected autoinjector e-device, which automatically records adherence data and can transmit them via an online portal (easypod™ connect). We investigated persistence of use, defined indicators of a long persistence of use, and developed a model to predict and identify children at risk of discontinuing treatment in the following 6 months. Data and Methods: Anonymized data from children transmitting over 10 injections between January 2007 and April 2020 were analyzed. A child was considered to discontinue the use if they had no injection in the last 6 months (before April 2020) or had an injection pause of at least 6 consecutive months. Persistence was estimated by Kaplan-Meier analyses and Weibull accelerated failure time modeling. To predict the individual risk of discontinuing the use in the following 6 months, individual survival probabilities curves were estimated for each patient still using the system, and the survival probabilities were then recalculated such that they were conditional on the fact that a child had already used the device for a certain time. The Harrell’s c-index was used to assess the algorithm performance. Results: Data were available for 17,651 children of whom 11,056 discontinued the use and 6,595 were still persistent in April 2020. Median persistence of use for all patients using the device was 2.1 years. There was a highly significant difference in median persistence of use between the regions: 1.0, 1.5 and 2.8 years in the available countries in the Asia-Pacific, America and Europe regions, respectively. Other indicators that had a significant positive impact on persistence of use were: at least one dose change a year, having auxological measurements recorded in the system (and if ‘Yes’, height standard deviation &lt; -2 at start of use had an additional effect), starting treatment at an early age, adherence ≥85%, customized injection speed setting and being male. For the individual prediction, random survival forests showed the best performance (Harrell’s c-index=0.72). Conclusions: Data from the connected autoinjector e-device showed that the persistence of use was approximately 2 years in children with growth disorders. We were able to define 8 indicators that had a positive impact on persistence of use of which several indicators were related to patient management. Our prediction model can be used to identify children needing support to reach longer persistence of use and subsequently optimal clinical outcomes.

A Machine Learning Approach for Identifying Children at Risk of Suboptimal Adherence to Growth Hormone Therapy

Background: Suboptimal adherence to recombinant human growth hormone (r-hGH) treatment can lead to suboptimal clinical outcomes. Being able to identify children who are at risk of suboptimal adherence in the near future, and take adequate measures to support adherence, may maximize clinical outcomes. Our aim was to develop a model based on data from the first 3 months of treatment to identify potential indicators of suboptimal adherence and predict adherence over the following 9 months using a machine learning approach.

Methods: We assessed adherence to r-hGH treatment in children with growth disorders in their first 12 months of treatment using a connected autoinjector and e-device (easypod™), which automatically transmits adherence data via an online portal (easypod™ connect). We selected children who started the use of the device before 18 years of age and who transmitted their injection data for at least 12 months. Adherence (mg injected/mg prescribed) between 4-12 months (outcome) was categorized as optimal (≥85%) versus suboptimal (&lt;85%). In addition to adherence over the first 3 months, comfort settings (needle speed, injection depth, injection speed, injection time), number of transmissions, number of dose changes, age at start and sex were used as potential indicators of suboptimal adherence. Several machine learning models were optimized on a class-balanced training dataset using a 5-fold cross-validation scheme. On the best performing model, machine learning interpretation techniques and chi-squared statistical tests were applied to extract the statistically significant indicators of suboptimal and optimal adherence. Results: Anonymized data were available for 10,943 children. The optimal prediction performances were achieved with the random forest algorithm. The mean adherence and the adherence standard deviation over the first 3 months were the two most important features for predicting adherence in the following 9 months. Not using the system’s features (e.g. not transmitting data often and not changing some of the comfort settings, such as the needle speed setting), as well as starting treatment at an older age were significantly associated with an increased risk of suboptimal adherence (p&lt;0.001). When tested on first-time seen data following the same class distribution as the original data, the model achieved a sensitivity of 80% and a specificity of 81%.

Conclusions: We developed a model predicting whether a child’s adherence in the following 9 months will be below or above the optimal threshold (85%) based on early data from the first 3 months of treatment and we identified the indicators of suboptimal adherence. These results can be used to identify children needing additional medical or other support to reach optimal adherence and therefore optimal clinical outcomes.