A validation of an intelligent decision-making support system for the nutrition diagnosis of bariatric surgery patients

Analytical validation of Exandra: a clinical decision support system for promoting guideline-directed therapy of type-2 diabetes in primary care - a collaborative study with experts from Diabetes Canada

BACKGROUND

Individuals with type 2 diabetes (T2D) have a high prevalence of cardiovascular and renal comorbidities. Despite clinical practice guidelines recommending the use of cardiorenal protective medications, many people with T2D are not prescribed these medications. A clinical decision support system called Exandra was developed to provide treatment recommendations for individuals with T2D based on current clinical practice guidelines from Diabetes Canada. The current study aimed to medically validate Exandra via review by external medical experts in T2D.

METHODS

Validation of Exandra took place in two phases. Test cases using simulated clinical scenarios and recommendations were generated by Exandra. In Phase 1 of the validation, reviewers evaluated whether they agreed with Exandra's recommendations with a "yes," "no," or "not sure" response. In Phase 2, reviewers were interviewed about their "no" and "not sure" responses to determine possible reasons and potential fixes to the Exandra system. The primary outcome was the precision rate of Exandra following the interviews and final adjudication of the cases. The target precision rate was 90%.

RESULTS

Exandra displayed an overall precision rate of 95.5%. A large proportion of cases that were initially labeled "no" or "not sure" by reviewers were changed to "yes" following the interview phase. This was largely due to the validation using a simplified user interface compared with the complexity of the actual Exandra system, and reviewers needing clarification of how the outputs would be displayed on the Exandra platform.

CONCLUSION

Exandra displayed a high level of accuracy and precision in providing guideline-directed recommendations for managing T2D and its common comorbidities. The results of this study indicate that Exandra is a promising tool for improving the management of T2D and its comorbidities.

Artificial Intelligence in Surgery: A Systematic Review of Use and Validation

Background: Artificial Intelligence (AI) holds promise for transforming healthcare, with AI models gaining increasing clinical use in surgery. However, new AI models are developed without established standards for their validation and use. Before AI can be widely adopted, it is crucial to ensure these models are both accurate and safe for patients. Without proper validation, there is a risk of integrating AI models into practice without sufficient evidence of their safety and accuracy, potentially leading to suboptimal patient outcomes. In this work, we review the current use and validation methods of AI models in clinical surgical settings and propose a novel classification system. Methods: A systematic review was conducted in PubMed and Cochrane using the keywords "validation", "artificial intelligence", and "surgery", following PRISMA guidelines. Results: The search yielded a total of 7627 articles, of which 102 were included for data extraction, encompassing 2,837,211 patients. A validation classification system named Surgical Validation Score (SURVAS) was developed. The primary applications of models were risk assessment and decision-making in the preoperative setting. Validation methods were ranked as high evidence in only 45% of studies, and only 14% of the studies provided publicly available datasets. Conclusions: AI has significant applications in surgery, but validation quality remains suboptimal, and public data availability is limited. Current AI applications are mainly focused on preoperative risk assessment and are suggested to improve decision-making. Classification systems such as SURVAS can help clinicians confirm the degree of validity of AI models before their application in practice.

Development of an Artificial Intelligence-Guided Citizen-Centric Predictive Model for the Uptake of Maternal Health Services Among Pregnant Women Living in Urban Slum Settings in India: Protocol for a Cross-sectional Study With a Mixed Methods Design

BACKGROUND

Pregnant women are considered a "high-risk" group with limited access to health facilities in urban slums in India. Barriers to using health services appropriately may lead to maternal and child mortality, morbidity, low birth weight, and children with stunted growth. With the increase in the use of artificial intelligence (AI) and machine learning in the health sector, we plan to develop a predictive model that can enable substantial uptake of maternal health services and improvements in adverse pregnancy health care outcomes from early diagnostics to treatment in urban slum settings.

OBJECTIVE

The objective of our study is to develop and evaluate the AI-guided citizen-centric platform that will support the uptake of maternal health services among pregnant women seeking antenatal care living in urban slum settings.

METHODS

We will conduct a cross-sectional study using a mixed methods approach to enroll 225 pregnant women aged 18-44 years, living in the urban slums of Delhi for more than 6 months, seeking antenatal care, and who have smartphones. Quantitative and qualitative data will be collected using an Open Data Kit Android-based tool. Variables gathered will include sociodemographics, clinical history, pregnancy history, dietary history, COVID-19 history, health care facility data, socioeconomic status, and pregnancy outcomes. All data gathered will be aggregated into a common database. We will use AI to predict the early at-risk pregnancy outcomes (in terms of the type of delivery method, term, and related complications) depending on the needs of the beneficiaries translating into effective service-delivery improvements in enhancing the use of maternal health services among pregnant women seeking antenatal care. The proposed research will help policy makers to prioritize resource planning, resource allocation, and the development of programs and policies to enhance maternal health outcomes. The academic research study has received ethical approval from the University Research Ethics Committee of Dehradun Institute of Technology (DIT) University, Dehradun, India.

RESULTS

The study was approved by the University Research Ethics Committee of DIT University, Dehradun, on July 4, 2021. Enrollment of the eligible participants will begin by April 2022 followed by the development of the predictive model by October 2022 till January 2023. The proposed AI-guided citizen-centric tool will be designed, developed, implemented, and evaluated using principles of human-centered design that will help to predict early at-risk pregnancy outcomes.

CONCLUSIONS

The proposed internet-enabled AI-guided prediction model will help identify the potential risk associated with pregnancies and enhance the uptake of maternal health services among those seeking antenatal care for safer deliveries. We will explore the scalability of the proposed platform up to different geographic locations for adoption for similar and other health conditions.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/35452.

Current Applications of Artificial Intelligence in Bariatric Surgery

The application of artificial intelligence technologies is growing in several fields of healthcare settings. The aim of this article is to review the current applications of artificial intelligence in bariatric surgery. We performed a review of the literature on Scopus, PubMed and Cochrane databases, screening all relevant studies published until September 2021, and finally including 36 articles. The use of machine learning algorithms in bariatric surgery is explored in all steps of the clinical pathway, from presurgical risk-assessment and intraoperative management to complications and outcomes prediction. The models showed remarkable results helping physicians in the decision-making process, thus improving the quality of care, and contributing to precision medicine. Several legal and ethical hurdles should be overcome before these methods can be used in common practice.

A Scoping Review of Artificial Intelligence and Machine Learning in Bariatric and Metabolic Surgery: Current Status and Future Perspectives

Artificial intelligence (AI) is a revolution in data analysis with emerging roles in various specialties and with various applications. The objective of this scoping review was to retrieve current literature on the fields of AI that have been applied to metabolic bariatric surgery (MBS) and to investigate potential applications of AI as a decision-making tool of the bariatric surgeon. Initial search yielded 3260 studies published from January 2000 until March 2021. After screening, 49 unique articles were included in the final analysis. Studies were grouped into categories, and the frequency of appearing algorithms, dataset types, and metrics were documented. The heterogeneity of current studies showed that meticulous validation, strict reporting systems, and reliable benchmarking are mandatory for ensuring the clinical validity of future research.

The potential of artificial intelligence in enhancing adult weight loss: a scoping review

OBJECTIVE

To present an overview of how artificial intelligence (AI) could be used to regulate eating and dietary behaviours, exercise behaviours and weight loss.

DESIGN

A scoping review of global literature published from inception to 15 December 2020 was conducted according to Arksey and O'Malley's five-step framework. Eight databases (CINAHL, Cochrane-Central, Embase, IEEE Xplore, PsycINFO, PubMed, Scopus and Web of Science) were searched. Included studies were independently screened for eligibility by two reviewers with good interrater reliability (k = 0·96).

RESULTS

Sixty-six out of 5573 potential studies were included, representing more than 2031 participants. Three tenets of self-regulation were identified - self-monitoring (n 66, 100 %), optimisation of goal setting (n 10, 15·2 %) and self-control (n 10, 15·2 %). Articles were also categorised into three AI applications, namely machine perception (n 50), predictive analytics only (n 6) and real-time analytics with personalised micro-interventions (n 10). Machine perception focused on recognising food items, eating behaviours, physical activities and estimating energy balance. Predictive analytics focused on predicting weight loss, intervention adherence, dietary lapses and emotional eating. Studies on the last theme focused on evaluating AI-assisted weight management interventions that instantaneously collected behavioural data, optimised prediction models for behavioural lapse events and enhance behavioural self-control through adaptive and personalised nudges/prompts. Only six studies reported average weight losses (2·4-4·7 %) of which two were statistically significant.

CONCLUSION

The use of AI for weight loss is still undeveloped. Based on the current study findings, we proposed a framework on the applicability of AI for weight loss but cautioned its contingency upon engagement and contextualisation.

Using Bayesian Networks to Predict Long-Term Health-Related Quality of Life and Comorbidity after Bariatric Surgery: A Study Based on the Scandinavian Obesity Surgery Registry

Previously published literature has identified a few predictors of health-related quality of life (HRQoL) after bariatric surgery. However, performance of the predictive models was not evaluated rigorously using real world data. To find better methods for predicting prognosis in patients after bariatric surgery, we examined performance of the Bayesian networks (BN) method in predicting long-term postoperative HRQoL and compared it with the convolution neural network (CNN) and multivariable logistic regression (MLR). The patients registered in the Scandinavian Obesity Surgery Registry (SOReg) were used for the current study. In total, 6542 patients registered in the SOReg between 2008 and 2012 with complete demographic and preoperative comorbidity information, and preoperative and postoperative 5-year HROoL scores and comorbidities were included in the study. HRQoL was measured using the RAND-SF-36 and the obesity-related problems scale. Thirty-five variables were used for analyses, including 19 predictors and 16 outcome variables. The Gaussian BN (GBN), CNN, and a traditional linear regression model were used for predicting 5-year HRQoL scores, and multinomial discrete BN (DBN) and MLR were used for 5-year comorbidities. Eighty percent of the patients were randomly selected as a training dataset and 20% as a validation dataset. The GBN presented a better performance than the CNN and the linear regression model; it had smaller mean squared errors (MSEs) than those from the CNN and the linear regression model. The MSE of the summary physical scale was only 0.0196 for GBN compared to the 0.0333 seen in the CNN. The DBN showed excellent predictive ability for 5-year type 2 diabetes and dyslipidemia (area under curve (AUC) = 0.942 and 0.917, respectively), good ability for 5-year hypertension and sleep apnea syndrome (AUC = 0.891 and 0.834, respectively), and fair ability for 5-year depression (AUC = 0.750). Bayesian networks provide useful tools for predicting long-term HRQoL and comorbidities in patients after bariatric surgery. The hybrid network that may involve variables from different probability distribution families deserves investigation in the future.

Development of a Standardized Rating Tool for Drug Alerts to Reduce Information Overload

BACKGROUND

A well-known problem in current clinical decision support systems (CDSS) is the high number of alerts, which are often medically incorrect or irrelevant. This may lead to the so-called alert fatigue, an overriding of alerts, including those that are clinically relevant, and underuse of CDSS in general.

OBJECTIVES

The aim of our study was to develop and to apply a standardized tool that allows its users to evaluate the quality of system-generated drug alerts. The users' ratings can subsequently be used to derive recommendations for developing a filter function to reduce irrelevant alerts.

METHODS

We developed a rating tool for drug alerts and performed a web-based evaluation study that also included a user review of alerts. In this study the following categories were evaluated: "data linked correctly", "medically correct", "action required", "medication change", "critical alert", "information gained" and "show again". For this purpose, 20 anonymized clinical cases were randomly selected and displayed in our customized CDSS research prototype, which used the summary of product characteristics (SPC) for alert generation. All the alerts that were provided were evaluated by 13 physicians. The users' ratings were used to derive a filtering algorithm to reduce overalerting.

RESULTS

In total, our CDSS research prototype generated 399 alerts. In 98 % of all alerts, medication data were rated as linked correctly to drug information; in 93 %, the alerts were assessed as "medically correct"; 19.5 % of all alerts were rated as "show again". The interrater-agreement was, on average, 68.4 %. After the application of our filtering algorithm, the rate of alerts that should be shown again decreased to 14.8 %.

CONCLUSIONS

The new standardized rating tool supports a standardized feedback of user-perceived clinical relevance of CDSS alerts. Overall, the results indicated that physicians may consider the majority of alerts formally correct but clinically irrelevant and override them. Filtering may help to reduce overalerting and increase the specificity of a CDSS.