Clinical prediction rules

Using Artificial Intelligence to Label Free-Text Operative and Ultrasound Reports for Grading Pediatric Appendicitis

PURPOSE

Data science approaches personalizing pediatric appendicitis management are hampered by small datasets and unstructured electronic medical records (EMR). Artificial intelligence (AI) chatbots based on large language models can structure free-text EMR data. We compare data extraction quality between ChatGPT-4 and human data collectors.

METHODS

To train AI models to grade pediatric appendicitis preoperatively, several data collectors extracted detailed preoperative and operative data from 2100 children operated for acute appendicitis. Collectors were trained for the task based on satisfactory Kappa scores. ChatGPT-4 was prompted to structure free text from 103 random anonymized ultrasound and operative records in the dataset using the set variables and coding options, and to estimate appendicitis severity grade from the operative report. A pediatric surgeon then adjudicated all data, identifying errors in each method.

RESULTS

Within the 44 ultrasound (42.7%) and 32 operative reports (31.1%) discordant in at least one field, 98% of the errors were found in the manual data extraction. The appendicitis grade was erroneously assigned manually in 29 patients (28.2%), and by ChatGPT-4 in 3 (2.9%). Across datasets, the use of the AI chatbot was able to avoid misclassification in 59.2% of the records including both reports and extracted data approximately 40 times faster.

CONCLUSION

AI chatbot significantly outperformed manual data extraction in accuracy for ultrasound and operative reports, and correctly assigned the appendicitis grade. While wider validation is required and data safety concerns must be addressed, these AI tools show significant promise in improving the accuracy and efficiency of research data collection.

LEVELS OF EVIDENCE

Level III.

[Comparison of different prediction models for the occurrence of nausea and vomiting in the postoperative phase : A systematic qualitative comparison based on prospectively defined quality indicators]

BACKGROUND

Various prognostic prediction models exist for evaluating the risk of nausea and vomiting in the postoperative period (PONV). So far, no systematic comparison of these prognostic scores is available.

METHOD

A systematic literature search was carried out in seven medical databases to find publications on prognostic PONV models. Identified scores were assessed against prospectively defined quality criteria, including generalizability, validation and clinical relevance of the models.

RESULTS

The literature search revealed 62 relevant publications with a total of 81,834 patients which could be assigned to 8 prognostic models. The simplified Apfel score performed best, primarily because it was extensively validated. The Van den Bosch score and Sinclair score tied for second place. The simplified Koivuranta score was in third place.

CONCLUSION

The qualitative analysis highlights the strengths and weaknesses of each prediction system based on predetermined standardized quality criteria.

Deployment of a human-centred clinical decision support system for pulmonary embolism: evaluation of impact on quality of diagnostic decisions

Pulmonary embolism (PE) is a serious condition that presents a diagnostic challenge for which diagnostic errors often happen. The literature suggests that a gap remains between PE diagnostic guidelines and adherence in healthcare practice. While system-level decision support tools exist, the clinical impact of a human-centred design (HCD) approach of PE diagnostic tool design is unknown.

DESIGN

Before-after (with a preintervention period as non-concurrent control) design study.

SETTING

Inpatient units at two tertiary care hospitals.

PARTICIPANTS

General internal medicine physicians and their patients who underwent PE workups.

INTERVENTION

After a 6-month preintervention period, a clinical decision support system (CDSS) for diagnosis of PE was deployed and evaluated over 6 months. A CDSS technical testing phase separated the two time periods.

MEASUREMENTS

PE workups were identified in both the preintervention and CDSS intervention phases, and data were collected from medical charts. Physician reviewers assessed workup summaries (blinded to the study period) to determine adherence to evidence-based recommendations. Adherence to recommendations was quantified with a score ranging from 0 to 1.0 (the primary study outcome). Diagnostic tests ordered for PE workups were the secondary outcomes of interest.

RESULTS

Overall adherence to diagnostic pathways was 0.63 in the CDSS intervention phase versus 0.60 in the preintervention phase (p=0.18), with fewer workups in the CDSS intervention phase having very low adherence scores. Further, adherence was significantly higher when PE workups included the Wells prediction rule (median adherence score=0.76 vs 0.59, p=0.002). This difference was even more pronounced when the analysis was limited to the CDSS intervention phase only (median adherence score=0.80 when Wells was used vs 0.60 when Wells was not used, p=0.001). For secondary outcomes, using both the D-dimer blood test (42.9% vs 55.7%, p=0.014) and CT pulmonary angiogram imaging (61.9% vs 75.4%, p=0.005) was lower during the CDSS intervention phase.

CONCLUSION

A clinical decision support intervention with an HCD improves some aspects of the diagnostic decision, such as the selection of diagnostic tests and the use of the Wells probabilistic prediction rule for PE.

Prospective Validation of the Calgary Kids' Hand Rule: A Clinical Prediction Rule for Pediatric Hand Fracture Triage

Introduction: Pediatric hand fractures are common and routinely referred to surgeons, yet most heal well without surgical intervention. This trend inspired the development of the Calgary Kids' Hand Rule (CKHR), a clinical prediction rule designed to predict "complex" fractures that require surgical referral. The CKHR was adapted into a checklist whereby the presence of any 1 of 6 clinically or radiologically identifiable fracture characteristics predicts a complex fracture. The aim of this study was to assess the accuracy of the CKHR in a prospective sample of children with hand fractures. Methods: Physicians were asked to complete the CKHR checklist when referring pediatric patients (< 18 years) to hand surgeons at a Canadian pediatric hospital (April 2019-September 2020). Completed checklists represented predicted outcomes and were compared to observed outcomes (determined via chart review). Predictive accuracy (primary outcome) was evaluated based on sensitivity and specificity. Secondary outcomes were interrater reliability between referring physicians and surgeons, and survey assessment of CKHR user satisfaction. Results: In total 365 fractures were included, with only 16 requiring surgical intervention. Overall performance of the CKHR was good with 84% sensitivity and 71% specificity. Percent agreement between referring physicians and surgeons ranged from 84.1% to 96.3% on individual predictors, with 78.1% agreement on the presence of any predictors. Survey results showed general user satisfaction but also identified areas for improvement. Conclusion: This study posits the CKHR as an accurate and clinically useful prediction rule and highlights the importance of education for its effective use and eventual scale and spread.

How Real-World Data Can Facilitate the Development of Precision Medicine Treatment in Psychiatry

Precision medicine has the ambition to improve treatment response and clinical outcomes through patient stratification and holds great potential for the treatment of mental disorders. However, several important factors are needed to transform current practice into a precision psychiatry framework. Most important are 1) the generation of accessible large real-world training and test data including genomic data integrated from multiple sources, 2) the development and validation of advanced analytical tools for stratification and prediction, and 3) the development of clinically useful management platforms for patient monitoring that can be integrated into health care systems in real-life settings. This narrative review summarizes strategies for obtaining the key elements-well-powered samples from large biobanks integrated with electronic health records and health registry data using novel artificial intelligence algorithms-to predict outcomes in severe mental disorders and translate these models into clinical management and treatment approaches. Key elements are massive mental health data and novel artificial intelligence algorithms. For the clinical translation of these strategies, we discuss a precision medicine platform for improved management of mental disorders. We use cases to illustrate how precision medicine interventions could be brought into psychiatry to improve the clinical outcomes of mental disorders.

A systematic review of the frequency of features of the seven-point checklist in proven cutaneous melanoma: The importance of change

Background

Pigmented skin lesions in human adults can present with several different visible features that may indicate signs of malignancy, particularly melanoma. Patient and clinician awareness of these features can aid the early recognition and melanoma diagnosis improving patient outcomes. The seven-point checklist (7PCL) is a clinical prediction rule advocated by the National Institute for Health Care Excellence to aid the assessment of pigmented skin lesions in primary care to indicate referral for specialist opinion.

Objectives

Assess the current evidence to establish which features of the 7PC present more frequently, so public education and clinician assessment can be focused to maximise early diagnosis and minimise referrals of benign lesions.

Methods

A systematic review of published evidence identified studies that assessed the seven features of the 7PCL in histologically proven melanomas. Two independent reviewers screened eligible studies and independently extracted data and assessed quality.

Results

112 studies were screened, 20 were assessed in full, seven met the inclusion criteria. 1184 histologically diagnosed melanomas were assessed using the 7PCL. Four studies involved patients assessing 335 melanomas, and three involved clinicians who assessed 849 melanomas. The most common feature identified was a change in size of the lesion, and the least common was inflammation.

Conclusions

The most frequently occurring features of melanoma involve shape, size and colour, however focussing on changes in features, rather than irregularity, is more likely to identify early melanoma and increase the accuracy of referrals.

Evaluating the use of the ABCD2 score as a clinical decision aid in the emergency department: Retrospective observational study

OBJECTIVE

Clinical decision aids (CDAs) can help clinicians with patient risk assessment. However, there is little data on CDA calculation, interpretation and documentation in real-world ED settings. The ABCD2 score (range 0-7) is a CDA used for patients with transient ischaemic attack (TIA) and assesses risk of stroke, with a score of 0-3 being low risk. The aim of this study was to describe ABCD2 score documentation in patients with an ED diagnosis of TIA.

METHODS

Retrospective observational study of patients with a working diagnosis of a TIA in two Australian EDs. Data were gathered using routinely collected data from health informatics sources and medical records reviewed by a trained data abstractor. ABCD2 scores were calculated and compared with what was documented by the treating clinician. Data were presented using descriptive analysis and scatter plots.

RESULTS

Among the 367 patients with an ED diagnosis of TIA, clinicians documented an ABCD2 score in 45% (95% CI 40-50%, n = 165). Overall, there was very good agreement between calculated and documented scores (Cohen's kappa 0.90). The mean documented and calculated ABCD2 score were similar (3.8, SD = 1.5, n = 165 vs 3.7, SD = 1.8, n = 367). Documented scores on the threshold of low and high risk were more likely to be discordant with calculated scores.

CONCLUSIONS

The ABCD2 score was documented in less than half of eligible patients. When documented, clinicians were generally accurate with their calculation and application of the ABCD2. No independent predictors of ABCD2 documentation were identified.

Progress of the application clinical prediction model in polycystic ovary syndrome

Clinical prediction models play an important role in the field of medicine. These can help predict the probability of an individual suffering from disease, complications, and treatment outcomes by applying specific methodologies. Polycystic ovary syndrome (PCOS) is a common disease with a high incidence rate, huge heterogeneity, short- and long-term complications, and complex treatments. In this systematic review study, we reviewed the progress of clinical prediction models in PCOS patients, including diagnosis and prediction models for PCOS complications and treatment outcomes. We aimed to provide ideas for medical researchers and clues for the management of PCOS. In the future, models with poor accuracy can be greatly improved by adding well-known parameters and validations, which will further expand our understanding of PCOS in terms of precision medicine. By developing a series of predictive models, we can make the definition of PCOS more accurate, which can improve the diagnosis of PCOS and reduce the likelihood of false positives and false negatives. It will also help discover complications earlier and treatment outcomes being known earlier, which can result in better outcomes for women with PCOS.

Risk factors and a predictive nomogram for hemodynamic instability during adrenalectomy for large pheochromocytomas and paragangliomas: A retrospective cohort study

OBJECTIVE

This study aimed to investigate risk factors for intraoperative hemodynamic instability (HDI) and construct a clinical model for predicting intraoperative HDI for large pheochromocytomas and paragangliomas (PPGLs) patients.

METHODS

A single-center retrospective study of the clinicopathological data of patients undergoing surgery for PPGLs larger than 5 cm in diameter was conducted. A total of 215 eligible patients were enrolled in the study. Three advanced statistical methods were used to select independent risk factors in the training cohort for constructing a nomogram for predicting intraoperative HDI. The predictive performance of the model was assessed by area under the curve (AUC), positive predictive value (PPV), negative predictive value (NPV), and calibration. Decision curve analysis (DCA) and clinical impact curves (CIC) were used to assess predictive accuracy and clinical utility. The performance of the nomogram of was further internally validated.

RESULTS

Comorbid diabetes mellitus, anemia, hypoproteinemia, 24-h urine vanillylmandelic acid and intraoperative blood transfusion (P < 0.05) were identified as independent risk factors for constructing the nomogram. In the training cohort, the AUC, PPV and NPV of the nomogram were 0.846, 91.6% and 69.2%. In the validation cohort, the AUC, PPV and NPV were 0.842, 91.8% and 63.3%. These showed good predictive power of the model. The calibration curves demonstrated an optimal consistency between the nomogram-predicted and the actual observed survival probability. DCA and CIC examination showed superior clinical relevance.

CONCLUSIONS

The nomogram can objectively and accurately predict intraoperative HDI in patients with large PPGLs, which can help in individualized pre-treatment decision-making.

Development and validation of a clinical prediction model for detecting coronary heart disease in middle-aged and elderly people: a diagnostic study

OBJECTIVE

To develop and validate a multivariate prediction model to estimate the risk of coronary heart disease (CHD) in middle-aged and elderly people and to provide a feasible method for early screening and diagnosis in middle-aged and elderly CHD patients.

METHODS

This study was a single-center, retrospective, case-control study. Admission data of 932 consecutive patients with suspected CHD were retrospectively assessed from September 1, 2020 to December 31, 2021 in the Department of Integrative Cardiology at China-Japan Friendship Hospital. A total of 839 eligible patients were included in this study, and 588 patients were assigned to the derivation set and 251 as the validation set at a 7:3 ratio. Clinical characteristics of included patients were compared between derivation set and validation set by univariate analysis. The least absolute shrinkage and selection operator (Lasso) regression analysis method was performed to avoid collinearity and identify key potential predictors. Multivariate logistic regression analysis was used to construct a clinical prediction model with identified predictors for clinical practice. Bootstrap validation was used to test performance and eventually we obtained the actual model. And the Hosmer-Lemeshow test was carried out to evaluate the goodness-fit of the constructed model. The area under curve (AUC) of receiver operating characteristic (ROC), calibration curve, decision curve analysis (DCA), and clinical impact curve (CIC) were plotted and utilized with validation set to comprehensively evaluate the predictive accuracy and clinical value of the model.

RESULTS

A total of eight indicators were identified as risk factors for the development of CHD in middle-aged and elderly people by univariate analysis. Of these candidate predictors, four key parameters were defined to be significantly related to CHD by Lasso regression analysis, including age (OR 1.034, 95% CI 1.002 ~ 1.067, P = 0.040), hemoglobin A1c (OR 1.380, 95% CI 1.078 ~ 1.768, P = 0.011), ankle-brachial index (OR 0.078, 95% CI 0.012 ~ 0.522, P = 0.009), and brachial artery flow-mediated vasodilatation (OR 0.848, 95% CI 0.726 ~ 0.990, P = 0.037). The Hosmer-Lemeshow test showed a good calibration performance of the clinical prediction model (derivation set, χ2 = 7.865, P = 0.447; validation set, χ2 = 11.132, P = 0.194). The ROCs of the nomogram in the derivation set and validation set were 0.722 and 0.783, respectively, suggesting excellent predictive power and suitable performance. The clinical prediction model presented a greater net benefit and clinical impact based on DCA and CIC analysis.

CONCLUSION

Overall, the development and validation of the multivariate model combined the laboratory and clinical parameters of patients with CHD, which could be beneficial to the individualized prediction of middle-aged and elderly people, and helped to facilitate clinical assessments and decisions during treatment and management of CHD.

Influence of research evidence on the use of cardiovascular clinical prediction rules in primary care: an exploratory qualitative interview study

BACKGROUND

Cardiovascular clinical prediction rules (CPRs) are widely used in primary care. They accumulate research evidence through derivation, external validation, and impact studies. However, existing knowledge about the influence of research evidence on the use of CPRs is limited. Therefore, we explored how primary care clinicians' perceptions of and experiences with research influence their use of cardiovascular CPRs.

METHODS

We conducted an exploratory qualitative interview study with thematic analysis. Primary care clinicians were recruited from the WWAMI (Washington, Wyoming, Alaska, Montana and Idaho) region Practice and Research Network (WPRN). We used purposeful sampling to ensure maximum variation within the participant group. Data were collected by conducting semi-structured online interviews. We analyzed data using inductive thematic analysis to identify commonalities and differences within themes.

RESULTS

Of 29 primary care clinicians who completed the questionnaire, 15 participated in the interview. We identified two main themes relating to the influence of clinicians' perceptions of and experiences with cardiovascular CPR research on their decisions about using cardiovascular CPRs: "Seek and judge" and "be acquainted and assume." When clinicians are familiar with, trust, and feel confident in using research evidence, they might actively search and assess the evidence, which may then influence their decisions about using cardiovascular CPRs. However, clinicians, who are unfamiliar with, distrust, or find it challenging to use research evidence, might be passively acquainted with evidence but do not make their own judgment on the trustworthiness of such evidence. Therefore, these clinicians might not rely on research evidence when making decisions about using cardiovascular CPRs.

CONCLUSIONS

Clinicians' perceptions and experiences could influence how they use research evidence in decisions about using cardiovascular CPRs. This implies, when promoting evidence-based decisions, it might be useful to target clinicians' unfamiliarity, distrust, and challenges regarding the use of research evidence rather than focusing only on their knowledge and skills. Further, because clinicians often rely on evidence-unrelated factors, guideline developers and policymakers should recommend cardiovascular CPRs supported by high-quality evidence.

A Dynamic Prediction Model Supporting Individual Life Expectancy Prediction Based on Longitudinal Time-Dependent Covariates

In the field of clinical chronic diseases, common prediction results (such as survival rate) and effect size hazard ratio (HR) are relative indicators, resulting in more abstract information. However, clinicians and patients are more interested in simple and intuitive concepts of (survival) time, such as how long a patient may live or how much longer a patient in a treatment group will live. In addition, due to the long follow-up time, resulting in generation of longitudinal time-dependent covariate information, patients are interested in how long they will survive at each follow-up visit. In this study, based on a time scale indicator-restricted mean survival time (RMST)-we proposed a dynamic RMST prediction model by considering longitudinal time-dependent covariates and utilizing joint model techniques. The model can describe the change trajectory of longitudinal time-dependent covariates and predict the average survival times of patients at different time points (such as follow-up visits). Simulation studies through Monte Carlo cross-validation showed that the dynamic RMST prediction model was superior to the static RMST model. In addition, the dynamic RMST prediction model was applied to a primary biliary cirrhosis (PBC) population to dynamically predict the average survival times of the patients, and the average C-index of the internal validation of the model reached 0.81, which was better than that of the static RMST regression. Therefore, the proposed dynamic RMST prediction model has better performance in prediction and can provide a scientific basis for clinicians and patients to make clinical decisions.

Establishment and Validation of Predictive Model of Tophus in Gout Patients

(1) Background: A tophus is a clinical manifestation of advanced gout, and in some patients could lead to joint deformities, fractures, and even serious complications in unusual sites. Therefore, to explore the factors related to the occurrence of tophi and establish a prediction model is clinically significant. (2) Objective: to study the occurrence of tophi in patients with gout and to construct a predictive model to evaluate its predictive efficacy. (3) Methods: The clinical data of 702 gout patients were analyzed by using cross-sectional data of North Sichuan Medical College. The least absolute shrinkage and selection operator (LASSO) and multivariate logistic regression were used to analyze predictors. Multiple machine learning (ML) classification models are integrated to analyze and identify the optimal model, and Shapley Additive exPlanations (SHAP) interpretation was developed for personalized risk assessment. (4) Results: Compliance of urate-lowering therapy (ULT), Body Mass Index (BMI), course of disease, annual attack frequency, polyjoint involvement, history of drinking, family history of gout, estimated glomerular filtration rate (eGFR), and erythrocyte sedimentation rate (ESR) were the predictors of the occurrence of tophi. Logistic classification model was the optimal model, test set area under curve (AUC) (95% confidence interval, CI): 0.888 (0.839-0.937), accuracy: 0.763, sensitivity: 0.852, and specificity: 0.803. (5) Conclusions: We constructed a logistic regression model and explained it with the SHAP method, providing evidence for preventing tophus and guidance for individual treatment of different patients.

Use of machine learning in pediatric surgical clinical prediction tools: A systematic review

PURPOSE

Clinical prediction tools (CPTs) are decision-making instruments utilizing patient data to predict specific clinical outcomes, risk-stratify patients, or suggest personalized diagnostic or therapeutic options. Recent advancements in artificial intelligence have resulted in a proliferation of CPTs created using machine learning (ML)-yet the clinical applicability of ML-based CPTs and their validation in clinical settings remain unclear. This systematic review aims to compare the validity and clinical efficacy of ML-based to traditional CPTs in pediatric surgery.

METHODS

Nine databases were searched from 2000 until July 9, 2021 to retrieve articles reporting on CPTs and ML for pediatric surgical conditions. PRISMA standards were followed, and screening was performed by two independent reviewers in Rayyan, with a third reviewer resolving conflicts. Risk of bias was assessed using the PROBAST.

RESULTS

Out of 8300 studies, 48 met the inclusion criteria. The most represented surgical specialties were pediatric general (14), neurosurgery (13) and cardiac surgery (12). Prognostic (26) CPTs were the most represented type of surgical pediatric CPTs followed by diagnostic (10), interventional (9), and risk stratifying (2). One study included a CPT for diagnostic, interventional and prognostic purposes. 81% of studies compared their CPT to ML-based CPTs, statistical CPTs, or the unaided clinician, but lacked external validation and/or evidence of clinical implementation.

CONCLUSIONS

While most studies claim significant potential improvements by incorporating ML-based CPTs in pediatric surgical decision-making, both external validation and clinical application remains limited. Further studies must focus on validating existing instruments or developing validated tools, and incorporating them in the clinical workflow.

TYPE OF STUDY

Systematic Review LEVEL OF EVIDENCE: Level III.

Early Diagnosis of Late-Onset Neonatal Sepsis Using a Sepsis Prediction Score

Sepsis represents a common cause of morbidity in the Neonatal Intensive Care Unit (NICU). Our objective was to assess the value of clinical and laboratory parameters in predicting septicemia (positive blood culture) in NICU infants. In the first part of the present study (derivation cohort) we retrospectively reviewed the clinical files of 120 neonates with symptoms of suspected sepsis and identified clinical and laboratory parameters associated with proven sepsis on the day the blood culture was taken, as well as 24 h and 48 h earlier. These parameters were combined into a sepsis prediction score (SPS). Subsequently (validation study), we prospectively validated the performance of the SPS in a cohort of 145 neonates. The identified parameters were: temperature instability, platelet count < 150,000/mm³, feeding volume decrease > 20%, changes in blood glucose > 50%, CRP > 1 mg/dL, circulatory and respiratory deterioration. In the retrospective cohort, on the day the blood culture was obtained, a SPS ≥ 3 could predict sepsis with 82.54% sensitivity, 85.96% specificity, 5.88 PLR (Positive Likelihood Ratio), 0.20 NLR (Negative Likelihood Ratio), 86.67% PPV (Positive Predictive Value), 81.67% NPV (Negative Predictive Value) and 84.17% accuracy. In the prospective cohort, on the day the blood culture was obtained, a SPS ≥ 3 could predict sepsis with 76.60% sensitivity, 72.55% specificity, 2.79 PLR, 0.32 NLR, 83.72% PPV, 62.71% NPV and 75.17% accuracy. We concluded that this combination of clinical and laboratory parameters may assist in the prediction of septicemia in NICUs.

Development and validation of a clinical rule for the diagnosis of chikungunya fever in a dengue-endemic area

Rio de Janeiro is a dengue-endemic city that experienced Zika and chikungunya epidemics between 2015 and 2019. Differential diagnosis is crucial for indicating adequate treatment and assessing prognosis and risk of death. This study aims to derive and validate a clinical rule for diagnosing chikungunya based on 3,214 suspected cases consecutively treated at primary and secondary health units of the sentinel surveillance system (up to 7 days from onset of symptoms) in Rio de Janeiro, Brazil. Of the total sample, 624 were chikungunya, 88 Zika, 51 dengue, and 2,451 were negative for all these arboviruses according to real-time polymerase chain reaction (RT-qPCR). The derived rule included fever (1 point), exanthema (1 point), myalgia (2 points), arthralgia or arthritis (2 points), and joint edema (2 points), providing an AUC (area under the receiver operator curve) = 0.695 (95% CI: 0.662-0.725). Scores of 4 points or more (validation sample) showed 74.3% sensitivity (69.0% - 79.2%) and 51.5% specificity (48.8% - 54.3%). Adding more symptoms improved the specificity at the expense of a lower sensitivity compared to definitions proposed by government agencies based on fever alone (European Center for Disease Control) or in combination with arthralgia (World Health Organization) or arthritis (Pan American Health Organization, Brazilian Ministry of Health). The proposed clinical rule offers a rapid, low-cost, easy-to-apply strategy to differentiate chikungunya fever from other arbovirus infections during epidemics.

Evolution of Making Clinical Predictions in Neurosurgery

Prediction of clinical outcomes is an essential task for every physician. Physicians may base their clinical prediction of an individual patient on their intuition and on scientific material such as studies presenting population risks and studies reporting on risk factors (prognostic factors). A relatively new and more informative approach for making clinical predictions relies on the use of statistical models that simultaneously consider multiple predictors that provide an estimate of the patient's absolute risk of an outcome. There is a growing body of literature in the neurosurgical field reporting on clinical prediction models. These tools have high potential in supporting (not replacing) neurosurgeons with their prediction of a patient's outcome. If used sensibly, these tools pave the way for more informed decision-making with or for individual patients. Patients and their significant others want to know their risk of the anticipated outcome, how it is derived, and the uncertainty associated with it. Learning from these prediction models and communicating the output to others has become an increasingly important skill neurosurgeons have to master. This article describes the evolution of making clinical predictions in neurosurgery, synopsizes key phases for the generation of a useful clinical prediction model, and addresses some considerations when deploying and communicating the results of a prediction model. The paper is illustrated with multiple examples from the neurosurgical literature, including predicting arachnoid cyst rupture, predicting rebleeding in patients suffering from aneurysmal subarachnoid hemorrhage, and predicting survival in glioblastoma patients.

Development of a Nomogram for Predicting Asymptomatic Coronary Artery Disease in Patients with Ischemic Stroke

BACKGROUND

Coronary artery stenosis (CAS) ≥50% often coexists in patients with ischemic stroke, which leads to a significant increase in the occurrence of major vascular events after stroke. This study aimed to develop a nomogram for diagnosing the presence of ≥50% asymptomatic CAS in patients with ischemic stroke.

METHODS

A primary cohort was established that included 275 non-cardioembolic ischemic stroke patients who were admitted from January 2011 to April 2013 to a teaching hospital in southern China. The preoperative data were used to construct two models by the best subset regression and the forward stepwise regression methods, and a nomogram between these models was established. The assessment of the nomogram was carried out by discrimination and calibration in an internal cohort.

RESULTS

Out of the two models, model 1 contained eight clinical-related variables and exhibited the lowest Akaike Information Criterion value (322.26) and highest concordance index 0.716 (95% CI, 0.654-0.778). The nomogram showed good calibration and significant clinical benefit according to calibration curves and the decision curve analysis.

CONCLUSION

The nomogram, composed of age, sex, NIHSS score on admission, hypertension history, fast glucose level, HDL cholesterol level, LDL cholesterol level, and presence of ≥50% cervicocephalic artery stenosis, can be used for prediction of ≥50% asymptomatic coronary artery disease (CAD). Further studies are needed to validate the effectiveness of this nomogram in other populations.

Ethical considerations for precision psychiatry: A roadmap for research and clinical practice

Precision psychiatry is an emerging field with transformative opportunities for mental health. However, the use of clinical prediction models carries unprecedented ethical challenges, which must be addressed before accessing the potential benefits of precision psychiatry. This critical review covers multidisciplinary areas, including psychiatry, ethics, statistics and machine-learning, healthcare and academia, as well as input from people with lived experience of mental disorders, their family, and carers. We aimed to identify core ethical considerations for precision psychiatry and mitigate concerns by designing a roadmap for research and clinical practice. We identified priorities: learning from somatic medicine; identifying precision psychiatry use cases; enhancing transparency and generalizability; fostering implementation; promoting mental health literacy; communicating risk estimates; data protection and privacy; and fostering the equitable distribution of mental health care. We hope this blueprint will advance research and practice and enable people with mental health problems to benefit from precision psychiatry.

Prediction model for the risk of ESKD in patients with primary FSGS

The purpose of this study is to build a prediction model for accurate assessment of the risk of end-stage kidney disease (ESKD) in individuals with primary focal segmental glomerulosclerosis (FSGS) by integrating clinical and pathological features at biopsy. The prediction model was created based on a retrospective study of 99 patients with biopsy-proven primary FSGS diagnosed at our hospital between December 2012 and December 2019. We assessed discriminative ability and predictive accuracy of the model by C-index and calibration plot. Internal validation of the prediction model was performed with 1000-bootstrap procedure. Eight patients (8.1%) progressed to ESKD before 31 March 2021. Univariate analysis revealed that disease duration before biopsy, hematuria, hemoglobin, eGFR, and percentages of sclerosis and global sclerosis were associated with renal outcome. In multivariate analysis, three predictors were included in final prediction model: eGFR, hematuria, and percentage of sclerosis. The C-index of the model was 0.811 and 5-year calibration plot showed good agreement between predicted renal survival probability and actual observation. A nomogram and an online risk calculator were built on the basis of the prediction model. In conclusion, we constructed and internally validated the first prediction model for risk of ESKD in primary FSGS, which showed good discriminative ability and calibration performance. The prediction model provides an accurate and simple strategy to predict renal prognosis which may help to identify patients at high risk of ESKD and guide the management for patients with primary FSGS in clinical practice.

Using the Revised Cardiac Risk Index to Predict Major Postoperative Events for People With Kidney Failure: An External Validation and Update

Background

People with kidney failure have high risk of postoperative morbidity and mortality. Although the revised cardiac risk index (RCRI) is used to estimate the risk of major postoperative events, it has not been validated in this population. We aimed to externally validate the RCRI and determine whether updating the model improved predictions for people with kidney failure.

Methods

We derived a retrospective, population-based cohort of adults with kidney failure (maintenance dialysis or sustained estimated glomerular filtration rate < 15 mL/min per 1.73 m²) who had surgery in Alberta, Canada between 2005 and 2019. We categorized participants based on RCRI variables and assigned risk estimates of death or major cardiac events, and then estimated predictive performance. We re-estimated the coefficients for each RCRI variable and internally validated the updated model. Net benefit was estimated with decision curve analysis.

Results

After 38,541 surgeries, 1204 events (3.1%) occurred. The estimated C-statistic for the original RCRI was 0.64 (95% confidence interval: 0.62, 0.65). Examination of calibration revealed significant risk overestimation. In the re-estimated RCRI model, discrimination was marginally different (C-statistic 0.67 [95% confidence interval: 0.66, 0.69]), though calibration was improved. No net benefit was observed when the data were examined with decision curve analysis, whereas the original RCRI was associated with harm.

Conclusions

The RCRI performed poorly in a Canadian kidney failure cohort and significantly overestimated risk, suggesting that RCRI use in similar kidney failure populations should be limited. A re-estimated kidney failure-specific RCRI may be promising but needs external validation. Novel perioperative models for this population are urgently needed.

Improving ambulance dispatch triage to trauma: A scoping review using the framework of development and evaluation of clinical prediction rules

OBJECTIVE

Ambulance dispatch algorithms should function as clinical prediction rules, identifying high acuity patients for advanced life support, and low acuity patients for non-urgent transport. Systematic reviews of dispatch algorithms are rare and focus on study types specific to the final phases of rule development, such as impact studies, and may miss the complete value-added evidence chain. We sought to summarise the literature for studies seeking to improve dispatch in trauma by performing a scoping review according to standard frameworks for developing and evaluating clinical prediction rules.

METHODS

We performed a scoping review searching MEDLINE, EMBASE, CINAHL, the CENTRAL trials registry, and grey literature from January 2005 to October 2021. We included all study types investigating dispatch triage to injured patients in the English language. We reported the clinical prediction rule phase (derivation, validation, impact analysis, or user acceptance) and the performance and outcomes measured for high and low acuity trauma patients.

RESULTS

Of 2067 papers screened, we identified 12 low and 30 high acuity studies. Derivation studies were most common (52%) and rule-based computer-aided dispatch was the most frequently investigated (23 studies). Impact studies rarely reported a prior validation phase, and few validation studies had their impact investigated. Common outcome measures in each phase were infrequent (0 to 27%), making a comparison between protocols difficult. A series of papers for low acuity patients and another for pediatric trauma followed clinical prediction rule development. Some low acuity Medical Priority Dispatch System codes are associated with the infrequent requirement for advanced life support and clinician review of computer-aided dispatch may enhance dispatch triage accuracy in studies of helicopter emergency medical services.

CONCLUSIONS

Few derivation and validation studies were followed by an impact study, indicating important gaps in the value-added evidence chain. While impact studies suggest clinician oversight may enhance dispatch, the opportunity exists to standardize outcomes, identify trauma-specific low acuity codes, and develop intelligent dispatch systems.

Implementation of EHMRG Risk Model in an Italian Population of Elderly Patients with Acute Heart Failure

Acute heart failure (AHF) is a cardiac emergency with an increasing incidence, especially among elderly patients. The Emergency Heart failure Mortality Risk Grade (EHMRG) has been validated to assess the 7-days AHF mortality risk, suggesting the management of patients admitted to an emergency department (ED). EHMRG has never been implemented in Italian ED nor among elderly patients. We aimed to assess EHMRG score accuracy in predicting in-hospital death in a retrospective cohort of elderly subjects admitted for AHF from the ED to an Internal Medicine Department. We enrolled, in a 24-months timeframe, all the patients admitted to an Internal Medicine Department from ED for AHF. We calculated the EHMRG score, subdividing patients into six categories, and assessing in-hospital mortality and length of stay. We evaluated EHMRG accuracy with ROC curve analysis and survival with Kaplan−Meier and Cox models. We collected 439 subjects, with 45 in-hospital deaths (10.3%), observing a significant increase of in-hospital death along with EHMRG class, from 0% (class 1) to 7.7% (class 5b; p < 0.0001). EHMRG was fairly accurate in the whole cohort (AUC: 0.75; 95%CI: 0.68−0.83; p < 0.0001), with the best cutoff observed at >103 (Se: 71.1%; Sp: 72.8%; LR+: 2.62; LR-: 0.40; PPV: 23.0%; NPV: 95.7%), but performed better considering the events in the first seven days of admission (AUC: 0.83; 95%; CI: 0.75−0.91; p < 0.0001). In light of our observations, EHMRG can be useful also for the Italian emergency system to predict the risk of short-term mortality for AHF among elderly patients. EHMRG performance was better in the first seven days but remained acceptable when considering the whole period of hospitalization.

Quantitative prediction error analysis to investigate predictive performance under predictor measurement heterogeneity at model implementation

BACKGROUND

When a predictor variable is measured in similar ways at the derivation and validation setting of a prognostic prediction model, yet both differ from the intended use of the model in practice (i.e., "predictor measurement heterogeneity"), performance of the model at implementation needs to be inferred. This study proposed an analysis to quantify the impact of anticipated predictor measurement heterogeneity.

METHODS

A simulation study was conducted to assess the impact of predictor measurement heterogeneity across validation and implementation setting in time-to-event outcome data. The use of the quantitative prediction error analysis was illustrated using an example of predicting the 6-year risk of developing type 2 diabetes with heterogeneity in measurement of the predictor body mass index.

RESULTS

In the simulation study, calibration-in-the-large of prediction models was poor and overall accuracy was reduced in all scenarios of predictor measurement heterogeneity. Model discrimination decreased with increasing random predictor measurement heterogeneity.

CONCLUSIONS

Heterogeneity of predictor measurements across settings of validation and implementation reduced predictive performance at implementation of prognostic models with a time-to-event outcome. When validating a prognostic model, the targeted clinical setting needs to be considered and analyses can be conducted to quantify the impact of anticipated predictor measurement heterogeneity on model performance at implementation.

Capturing adolescents in need of psychiatric care with psychopathological symptoms: A population-based cohort study

BACKGROUND

The current study aims to overcome past methodological limitations and capture adolescents in need of psychiatric care with psychopathological symptoms in a cohort with unrestricted access to mental health professionals.

METHODS

The study source population consisted of a random sample of adolescents aged 16-17 years (N=1,369) assessed by the Israeli Draft Board. An adapted version of the Brief Symptom Inventory was used to identify clinically relevant psychopathological symptoms with scores categorized as severe if they were in the top 10th percentile of symptoms, otherwise not severe. An independent interview with a subsequent referral to a mental health professional was used to categorize adolescents in need of psychiatric care. To examine the association between severe psychopathological symptoms and the need for psychiatric care, logistic regression models were fitted unadjusted and adjusted for age, sex, and intellectual assessment scores. Adjusted classification measures were estimated to examine the utility of severe psychopathological symptoms for clinical prediction of need for psychiatric care.

RESULTS

Information on 1,283 adolescents was available in the final analytic sample. Logistic regression modeling showed a statistically significant (p<0.001) association between self-reported severe psychopathological symptoms and the need for psychiatric care (OR adjusted: 4.38; 95% CI: 3.55-5.40). Severe psychopathological symptoms had a classification accuracy of 83% (CI: 81%-85%).

CONCLUSIONS

Severe psychopathological symptoms, although accounting for a fair proportion of treatment seeking, would perhaps be better useful for classification purposes alongside other variables rather than in isolation.

Is the SORT score reliable in predicting postoperative 30-day mortality after a nonemergency surgery in Saudi population?

Context

The process of stratifying patient risk preoperatively helps in the decision about the best-possible postoperative care for patients. There have been many scoring systems that are used in anesthesia practice.

Aims

To find out whether there is any difference between the mortality predicted from SORT scoring and the observed mortality among Saudi patients.

Settings and Design

This was a prospective, observational study in which we included patients underoing nonemergency surgical procedures at the Security Forces Hospital, Riyadh.

Methods and Material

We calculated the SORT scores for all the included patients. We then collected the 30-day mortality data of all the patients having nonemergency surgical procedures.

Statistical Analysis Used

We calculated the expected mortality ratio. A P value of less than 0.05 was considered significant.

Results

The mean SORT mortality risk score (%) for the whole sample was 0.30. The expected number of deaths was 1.638 while the observed deaths were 2, which yields an O/E ratio of 0.819 (p-value: 0.006). The O/E mortality ratios for patients in each individual ASA class were found to be statistically insignificant which means that SORT score can reliably predict mortality for each ASA class.

Conclusions

SORT scores can be used to predict 30-day mortality after nonemergency surgeries in Saudi population.

Cancer diagnostic tools to aid decision-making in primary care: mixed-methods systematic reviews and cost-effectiveness analysis

BACKGROUND

Tools based on diagnostic prediction models are available to help general practitioners diagnose cancer. It is unclear whether or not tools expedite diagnosis or affect patient quality of life and/or survival.

OBJECTIVES

The objectives were to evaluate the evidence on the validation, clinical effectiveness, cost-effectiveness, and availability and use of cancer diagnostic tools in primary care.

METHODS

Two systematic reviews were conducted to examine the clinical effectiveness (review 1) and the development, validation and accuracy (review 2) of diagnostic prediction models for aiding general practitioners in cancer diagnosis. Bibliographic searches were conducted on MEDLINE, MEDLINE In-Process, EMBASE, Cochrane Library and Web of Science) in May 2017, with updated searches conducted in November 2018. A decision-analytic model explored the tools' clinical effectiveness and cost-effectiveness in colorectal cancer. The model compared patient outcomes and costs between strategies that included the use of the tools and those that did not, using the NHS perspective. We surveyed 4600 general practitioners in randomly selected UK practices to determine the proportions of general practices and general practitioners with access to, and using, cancer decision support tools. Association between access to these tools and practice-level cancer diagnostic indicators was explored.

RESULTS

Systematic review 1 - five studies, of different design and quality, reporting on three diagnostic tools, were included. We found no evidence that using the tools was associated with better outcomes. Systematic review 2 - 43 studies were included, reporting on prediction models, in various stages of development, for 14 cancer sites (including multiple cancers). Most studies relate to QCancer^® (ClinRisk Ltd, Leeds, UK) and risk assessment tools.

DECISION MODEL

In the absence of studies reporting their clinical outcomes, QCancer and risk assessment tools were evaluated against faecal immunochemical testing. A linked data approach was used, which translates diagnostic accuracy into time to diagnosis and treatment, and stage at diagnosis. Given the current lack of evidence, the model showed that the cost-effectiveness of diagnostic tools in colorectal cancer relies on demonstrating patient survival benefits. Sensitivity of faecal immunochemical testing and specificity of QCancer and risk assessment tools in a low-risk population were the key uncertain parameters.

SURVEY

Practitioner- and practice-level response rates were 10.3% (476/4600) and 23.3% (227/975), respectively. Cancer decision support tools were available in 83 out of 227 practices (36.6%, 95% confidence interval 30.3% to 43.1%), and were likely to be used in 38 out of 227 practices (16.7%, 95% confidence interval 12.1% to 22.2%). The mean 2-week-wait referral rate did not differ between practices that do and practices that do not have access to QCancer or risk assessment tools (mean difference of 1.8 referrals per 100,000 referrals, 95% confidence interval -6.7 to 10.3 referrals per 100,000 referrals).

LIMITATIONS

There is little good-quality evidence on the clinical effectiveness and cost-effectiveness of diagnostic tools. Many diagnostic prediction models are limited by a lack of external validation. There are limited data on current UK practice and clinical outcomes of diagnostic strategies, and there is no evidence on the quality-of-life outcomes of diagnostic results. The survey was limited by low response rates.

CONCLUSION

The evidence base on the tools is limited. Research on how general practitioners interact with the tools may help to identify barriers to implementation and uptake, and the potential for clinical effectiveness.

FUTURE WORK

Continued model validation is recommended, especially for risk assessment tools. Assessment of the tools' impact on time to diagnosis and treatment, stage at diagnosis, and health outcomes is also recommended, as is further work to understand how tools are used in general practitioner consultations.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42017068373 and CRD42017068375.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology programme and will be published in full in Health Technology Assessment; Vol. 24, No. 66. See the NIHR Journals Library website for further project information.

Validation of a Clinical Prediction Rule to Predict Asymptomatic Chlamydia and Gonorrhea Infections Among Internet-Based Testers

BACKGROUND

Clinical prediction rules (CPRs) can be used in sexually transmitted infection (STI) testing environments to prioritize individuals at the highest risk of infection and optimize resource allocation. We previously derived a CPR to predict asymptomatic chlamydia and/or gonorrhea (CT/NG) infection among women and heterosexual men at in-person STI clinics based on 5 predictors. Population differences between clinic-based and Internet-based testers may limit the tool's application across settings. The primary objective of this study was to assess the validity, sensitivity, and overall performance of this CPR within an Internet-based testing environment (GetCheckedOnline.com).

METHODS

We analyzed GetCheckedOnline online risk assessment and laboratory data from October 2015 to June 2019. We compared the STI clinic population used for CPR derivation (data previously published) and the GetCheckedOnline validation population using χ2 tests. Calibration and discrimination were assessed using the Hosmer-Lemeshow goodness-of-fit test and the area under the receiver operating curve, respectively. Sensitivity and the fraction of total screening tests offered were quantified for CPR-predicted risk scores.

RESULTS

Asymptomatic CT/NG infection prevalence in the GetCheckedOnline population (n = 5478) was higher than in the STI clinic population (n = 10,437; 2.4% vs. 1.8%, P = 0.007). When applied to GetCheckedOnline, the CPR had reasonable calibration (Hosmer-Lemeshow, P = 0.90) and discrimination (area under the receiver operating characteristic, 0.64). By screening only individuals with total risk scores ≥4, we would detect 97% of infections and reduce screening by 14%.

CONCLUSIONS

The application of an existing CPR to detect asymptomatic CT/NG infection is valid within an Internet-based STI testing environment. Clinical prediction rules applied online can reduce unnecessary STI testing and optimize resource allocation within publicly funded health systems.

Development and Validation of a Nomogram to Predict Frailty Progression in Nonfrail Chinese Community-Living Older Adults

OBJECTIVE

Frailty state progression is common among older adults, so it is necessary to identify predictors to implement individualized interventions. We aimed to develop and validate a nomogram to predict frailty progression in community-living older adults.

DESIGN

Prospective cohort study.

SETTING AND PARTICIPANTS

A total of 3170 Chinese community-living people aged ≥60 years were randomly assigned to a training set or validation set at a ratio of 6:4.

METHODS

Candidate predictors (demographic, lifestyle, and medical characteristics) were used to predict frailty state progression as measured with the Fried frailty phenotype at a 4-year follow-up, and multivariate logistic regression analysis was conducted to develop a nomogram, which was validated internally with 1000 bootstrap resamples and externally with the use of a validation set. The C index and calibration plot were used to assess discrimination and calibration of the nomogram, respectively.

RESULTS

After a follow-up period of 4 years, 64.1% (917/1430) of the participants in the robust group and 26.0% (453/1740) in the prefrail group experienced frailty progression, which included 9.1% and 21.0%, respectively, who progressed to frailty. Predictors in the final nomogram were age, marital status, physical exercise, baseline frailty state, and diabetes. Based on this nomogram, an online calculator was also developed for easy use. The discriminative ability was good in the training set (C index = 0.861) and was validated using both the internal bootstrap method (C index = 0.861) and an external validation set (C index = 0.853). The calibration plots showed good agreement in both the training and validation sets.

CONCLUSIONS AND IMPLICATIONS

An easy-to-use nomogram was developed with good apparent performance using 5 readily available variables to help physicians and public health practitioners to identify older adults at high risk for frailty progression and implement medical interventions.

Comparison of multiple statistical models for the development of clinical prediction scores to detect advanced colorectal neoplasms in asymptomatic Thai patients

A good clinical prediction score can help in the risk stratification of patients with colorectal cancer (CRC) undergoing colonoscopy screening. The aim of our study was to compare model performance of binary logistic regression (BLR), polytomous logistic regression (PLR), and classification and regression tree (CART) between the clinical prediction scores of advanced colorectal neoplasia (ACN) in asymptomatic Thai patients.We conducted a cross-sectional study of 1311 asymptomatic Thai patients to develop a clinical prediction model. The possible predictive variables included sex, age, body mass index, family history of CRC in first-degree relatives, smoking, diabetes mellitus, and the fecal immunochemical test in the univariate analysis. Variables with a P value of .1 were included in the multivariable analysis, using the BLR, CART, and PLR models. Model performance, including the area under the receiver operator characteristic curve (AUROC), was compared between the model types.ACN was diagnosed in 53 patients (4.04%). The AUROCs were not significantly different between the BLR and CART models for ACN prediction with an AUROC of 0.774 (95% confidence interval [95% CI]: 0.706-0.842) and 0.765 (95% CI: 0.698-0.832), respectively (P = .712). A significant difference was observed between the PLR and CART models in predicting average to moderate ACN risk with an AUROC of 0.767 (95% CI: 0.695-0.839 vs AUROC 0.675 [95% CI: 0.599-0.751], respectively; P = .009).The BLR and CART models yielded similar accuracies for the prediction of ACN in Thai patients. The PLR model provided higher accuracy for ACN prediction than the CART model.

Prediction of patient disposition: comparison of computer and human approaches and a proposed synthesis

OBJECTIVE

To compare the accuracy of computer versus physician predictions of hospitalization and to explore the potential synergies of hybrid physician-computer models.

MATERIALS AND METHODS

A single-center prospective observational study in a tertiary pediatric hospital in Boston, Massachusetts, United States. Nine emergency department (ED) attending physicians participated in the study. Physicians predicted the likelihood of admission for patients in the ED whose hospitalization disposition had not yet been decided. In parallel, a random-forest computer model was developed to predict hospitalizations from the ED, based on data available within the first hour of the ED encounter. The model was tested on the same cohort of patients evaluated by the participating physicians.

RESULTS

198 pediatric patients were considered for inclusion. Six patients were excluded due to incomplete or erroneous physician forms. Of the 192 included patients, 54 (28%) were admitted and 138 (72%) were discharged. The positive predictive value for the prediction of admission was 66% for the clinicians, 73% for the computer model, and 86% for a hybrid model combining the two. To predict admission, physicians relied more heavily on the clinical appearance of the patient, while the computer model relied more heavily on technical data-driven features, such as the rate of prior admissions or distance traveled to hospital.

DISCUSSION

Computer-generated predictions of patient disposition were more accurate than clinician-generated predictions. A hybrid prediction model improved accuracy over both individual predictions, highlighting the complementary and synergistic effects of both approaches.

CONCLUSION

The integration of computer and clinician predictions can yield improved predictive performance.

How hospitals can improve their public quality metrics: a decision-theoretic model

The public reporting of hospitals' quality of care is providing additional motivation for hospitals to deliver high-quality patient care. Hospital Compare, a consumer-oriented website by the Centers for Medicare and Medicaid Services (CMS), provides patients with detailed quality of care data on most US hospitals. Given that many quality metrics are the aggregate result of physicians' individual clinical decisions, the question arises if and how hospitals could influence their physicians so that their decisions positively contribute to hospitals' quality goals. In this paper, we develop a decision-theoretic model to explore how three different hospital interventions-incentivization, training, and nudging-may affect physicians' decisions. We focus our analysis on Outpatient Measure 14 (OP-14), which is an imaging quality metric that reports the percentage of outpatients with a brain computed tomography (CT) scan, who also received a same-day sinus CT scan. In most cases, same-day brain and sinus CT scans are considered unnecessary, and high utilizing hospitals aim to reduce their OP-14 metric. Our model captures the physicians' imaging decision process accounting for medical and behavioral factors, in particular the uncertainty in clinical assessment and a physician's diagnostic ability. Our analysis shows how hospital interventions of incentivization, training, and nudging affect physician decisions and consequently OP-14. This decision-theoretic model provides a foundation to develop insights for policy makers on the multi-level effects of their policy decisions.

A comprehensive scoping review of Bayesian networks in healthcare: Past, present and future

No comprehensive review of Bayesian networks (BNs) in healthcare has been published in the past, making it difficult to organize the research contributions in the present and identify challenges and neglected areas that need to be addressed in the future. This unique and novel scoping review of BNs in healthcare provides an analytical framework for comprehensively characterizing the domain and its current state. A literature search of health and health informatics literature databases using relevant keywords found 3810 articles that were reduced to 123. This was after screening out those presenting Bayesian statistics, meta-analysis or neural networks, as opposed to BNs and those describing the predictive performance of multiple machine learning algorithms, of which BNs were simply one type. Using the novel analytical framework, we show that: (1) BNs in healthcare are not used to their full potential; (2) a generic BN development process is lacking; (3) limitations exist in the way BNs in healthcare are presented in the literature, which impacts understanding, consensus towards systematic methodologies, practice and adoption; and (4) a gap exists between having an accurate BN and a useful BN that impacts clinical practice. This review highlights several neglected issues, such as restricted aims of BNs, ad hoc BN development methods, and the lack of BN adoption in practice and reveals to researchers and clinicians the need to address these problems. To map the way forward, the paper proposes future research directions and makes recommendations regarding BN development methods and adoption in practice.

Prediction of AESD and neurological sequelae in febrile status epilepticus

OBJECTIVE

The clinical prediction rule (CPR) for acute encephalopathy with biphasic seizures and late reduced diffusion (AESD) was developed with an area under the receiver operating characteristic curve (AUC) of 0.95 - 0.96. Our objective was to verify the AESD CPR in a new cohort and compare the utilities of three CPRs of acute encephalopathy: the Tada, Yokochi, and Nagase criteria.

METHODS

We reviewed the clinical data and medical charts of 580 consecutive patients (aged < 18 years) with febrile convulsive status epilepticus lasting for ≥ 30 min in 2002 - 2017 and measured the performance of the CPRs in predicting AESD and sequelae.

RESULTS

The CPRs predicted AESD with an AUC of 0.84 - 0.88. The Tada criteria predicted AESD with a positive predictive value (PPV) of 0.25 and a negative predictive value (NPV) of 0.99. The Yokochi criteria predicted AESD with a PPV and NPV of 0.20 and 0.95, respectively, after 12 h. The Nagase criteria predicted AESD with a PPV and NPV of 0.14 and 1.00, respectively, after 6 h. The PPVs of the Tada, Yokochi, and Nagase criteria for sequelae were 0.28, 0.28, and 0.17, respectively; the corresponding NPVs were 0.97, 0.95, and 0.98, respectively.

CONCLUSIONS

The effectiveness of the AESD CPR in a new cohort was lower than that in the derivation study. CPRs are not sufficient as diagnostic tests, but they are useful as screening tests. The Nagase criteria are the most effective for screening among the three CPRs due to their high NPV and swiftness.

Bayesian networks in healthcare: What is preventing their adoption?

There has been much research effort expended toward the use of Bayesian networks (BNs) in medical decision-making. However, because of the gap between developing an accurate BN and demonstrating its clinical usefulness, this has not resulted in any widespread BN adoption in clinical practice. This paper investigates this problem with the aim of finding an explanation and ways to address the problem through a comprehensive literature review of articles describing BNs in healthcare. Based on the literature collection that has been systematically narrowed down from 3810 to 116 most relevant articles, this paper analyses the benefits, barriers and facilitating factors (BBF) for implementing BN-based systems in healthcare using the ITPOSMO-BBF framework. A key finding is that works in the literature rarely consider barriers and even when these were identified they were not connected to facilitating factors. The main finding is that the barriers can be grouped into: (1) data inadequacies; (2) clinicians' resistance to new technologies; (3) lack of clinical credibility; (4) failure to demonstrate clinical impact; (5) absence of an acceptable predictive performance; and (6) absence of evidence for model's generalisability. The facilitating factors can be grouped into: (1) data collection improvements; (2) software and technological improvements; (3) having interpretable and easy to use BN-based systems; (4) clinical involvement in the development or review of the model; (5) investigation of model's clinical impact; (6) internal validation of the model's performance; and (7) external validation of the model. These groupings form a strong basis for a generic framework that could be used for formulating strategies for ensuring BN-based clinical decision-support system adoption in frontline care settings. The output of this review is expected to enhance the dialogue among researchers by providing a deeper understanding for the neglected issue of BN adoption in practice and promoting efforts for implementing BN-based systems.

"A catalyst for action": Factors for implementing clinical risk prediction models of infection in home care settings

OBJECTIVE

The study sought to outline how a clinical risk prediction model for identifying patients at risk of infection is perceived by home care nurses, and to inform how the output of the model could be integrated into a clinical workflow.

MATERIALS AND METHODS

This was a qualitative study using semi-structured interviews with 50 home care nurses. Interviews explored nurses' perceptions of clinical risk prediction models, their experiences using them in practice, and what elements are important for the implementation of a clinical risk prediction model focusing on infection. Interviews were audio-taped and transcribed, with data evaluated using thematic analysis.

RESULTS

Two themes were derived from the data: (1) informing nursing practice, which outlined how a clinical risk prediction model could inform nurse clinical judgment and be used to modify their care plan interventions, and (2) operationalizing the score, which summarized how the clinical risk prediction model could be incorporated in home care settings.

DISCUSSION

The findings indicate that home care nurses would find a clinical risk prediction model for infection useful, as long as it provided both context around the reasons why a patient was deemed to be at high risk and provided some guidance for action.

CONCLUSIONS

It is important to evaluate the potential feasibility and acceptability of a clinical risk prediction model, to inform the intervention design and implementation strategy. The results of this study can provide guidance for the development of the clinical risk prediction tool as an intervention for integration in home care settings.

Five-year risk of admission to long-term care home and death for older adults given a new diagnosis of dementia: a population-based retrospective cohort study

BACKGROUND

After diagnosis of a health condition, information about survival and potential transition from community into institutional care can be helpful for patients and care providers. We sought to describe the association between a new diagnosis of dementia and risk of admission to a long-term care home and death at 5 years.

METHODS

We conducted a population-based retrospective cohort study using linked health administrative databases. We identified individuals aged 65 years or older, living in the community, with a first documented diagnosis of dementia between Jan. 1, 2010, and Dec. 31, 2012, in Ontario, Canada. Dementia diagnosis was captured using diagnostic codes from hospital discharges, physician billings, assessments conducted for home care and long-term care, and dispensed prescriptions for cholinesterase inhibitors. Our primary outcome measures were 5-year risk of death and placement in a long-term care home, adjusted for sociodemographic and clinical factors.

RESULTS

We identified 108 757 individuals in our study cohort. By the end of 5 years, 24.4% remained alive in the community and 20.5% were living in a long-term care home. Of the 55.1% who died, about half (27.9%) were admitted to a long-term care home before death. Three risk factors were associated with increased odds of death: older age (age ≥ 90 yr; odds ratio [OR] 9.5, 95% confidence interval [CI] 8.8-10.2 [reference: age 65-69 yr]), male sex (OR 1.7, 95% CI 1.6-1.7), and the presence of organ failure, including chronic obstructive pulmonary disease (OR 1.7, 95% CI 1.7-1.8), congestive heart failure (OR 2.0, 95% CI 1.9-2.0) and renal failure (OR 1.7, 95% CI 1.6-1.8). Groups formed by combinations of these 3 factors had an observed 5-year risk of death varying between 22% and 91%.

INTERPRETATION

Among community-dwelling older adults with newly identified dementia in Ontario, the majority died or were admitted to a long-term care home within 5 years. This information may be helpful for discussions on prognosis and need for admission to long-term care.

Preoperative Prediction Nomogram Based on Integrated Profiling for Glioblastoma Multiforme in Glioma Patients

Introduction: Traditional classification that divided gliomas into glioblastoma multiformes (GBM) and lower grade gliomas (LGG) based on pathological morphology has been challenged over the past decade by improvements in molecular stratification, however, the reproducibility and diagnostic accuracy of glioma classification still remains poor. This study aimed to establish and validate a novel nomogram for the preoperative diagnosis of GBM by using integrated data combined with feasible baseline characteristics and preoperative tests.

Material and method: The models were established in a primary cohort that included 259 glioma patients who had undergone surgical resection and were pathologically diagnosed from March 2014 to May 2016 in the First Affiliated Hospital of Xi'an Jiaotong University. The preoperative data were used to construct three models by the best subset regression, the forward stepwise regression, and the least absolute shrinkage and selection operator, and to furthermore establish the nomogram among those models. The assessment of nomogram was carried out by the discrimination and calibration in internal cohorts and external cohorts.

Results and discussion: Out of all three models, model 2 contained eight clinical-related variables, which exhibited the minimum Akaike Information Criterion (173.71) and maximum concordance index (0.894). Compared with the other two models, the integrated discrimination index for model 2 was significantly improved, indicating that the nomogram obtained from model 2 was the most appropriate model. Likewise, the nomogram showed great calibration and significant clinical benefit according to calibration curves and the decision curve analysis.

Conclusion: In conclusion, our study showed a novel preoperative model that incorporated clinically relevant variables and imaging features with laboratory data that could be used for preoperative prediction in glioma patients, thus providing more reliable evidence for surgical decision-making.

Development and validation of a novel prediction score for cardiac tamponade in emergency department patients with pericardial effusion

Aims

Determining which patients with pericardial effusion require urgent intervention can be challenging. We sought to develop a novel, simple risk prediction score for patients with pericardial effusion.

Methods and results

Adult patients admitted through the emergency department (ED) with pericardial effusion were retrospectively evaluated. The overall cohort was divided into a derivation and validation cohort for the generation and validation of a novel risk score using logistic regression. The primary outcome was a pericardial drainage procedure or death attributed to cardiac tamponade within 24 h of ED arrival. Among 195 eligible patients, 102 (52%) experienced the primary outcome. Four variables were selected for the novel score: systolic blood pressure < 100 mmHg (1.5 points), effusion diameter [1–2 cm (0 points), 2–3 cm (1.5 points), >3 cm (2 points)], right ventricular diastolic collapse (2 points), and mitral inflow velocity variation > 25% (1 point). The need for pericardial drainage within 24 h was stratified as low (<2 points), intermediate (2–4 points), or high (≥4 points), which corresponded to risks of 8.1% [95% confidence interval (CI) 3.0–16.8%], 63.8% [95% CI 50.1–76.0%], and 93.7% [95% CI 84.5–98.2%]. The area under the curve of the simplified score was 0.94 for the derivation and 0.91 for the validation cohort.

Conclusion

Among ED patients with pericardial effusion, a four-variable prediction score consisting of systolic blood pressure, effusion diameter, right ventricular collapse, and mitral inflow velocity variation can accurately predict the need for urgent pericardial drainage. Prospective validation of this novel score is warranted.

Emergency admission risk stratification tools in UK primary care: a cross-sectional survey of availability and use

Background

Stratifying patient populations by risk of adverse events was believed to support preventive care for those identified, but recent evidence does not support this. Emergency admission risk stratification (EARS) tools have been widely promoted in UK policy and GP contracts.

Aim

To describe availability and use of EARS tools across the UK, and identify factors perceived to influence implementation.

Design and setting

Cross-sectional survey in UK.

Method

Online survey of 235 organisations responsible for UK primary care: 209 clinical commissioning groups (CCGs) in England; 14 health boards in Scotland; seven health boards in Wales; and five local commissioning groups (LCGs) in Northern Ireland. Analysis results are presented using descriptive statistics for closed questions and by theme for open questions.

Results

Responses were analysed from 171 (72.8%) organisations, of which 148 (86.5%) reported that risk tools were available in their areas. Organisations identified 39 different EARS tools in use. Promotion by NHS commissioners, involvement of clinical leaders, and engagement of practice managers were identified as the most important factors in encouraging use of tools by general practices. High staff workloads and information governance were identified as important barriers. Tools were most frequently used to identify individual patients, but also for service planning. Nearly 40% of areas using EARS tools reported introducing or realigning services as a result, but relatively few reported use for service evaluation.

Conclusion

EARS tools are widely available across the UK, although there is variation by region. There remains a need to align policy and practice with research evidence.

The Urgent Surgery Elderly Mortality risk score: a simple mortality score

INTRODUCTION

an increasing number of elderly patients undergo urgent abdominal surgery and this population has a higher risk of mortality. The main objective of the study was to identify mortality-associated factors in elderly patients undergoing abdominal surgery and to design a mortality scoring tool, the Urgent Surgery Elderly Mortality risk score (the USEM score).

PATIENTS AND METHODS

this was a retrospective study using a prospective database. Patients > 65 years old that underwent urgent abdominal surgery were included. Risk factors for 30-day mortality were identified using multivariate regression analysis and weights assigned using the odds ratios (OR). A mortality score was derived from the aggregate of weighted scores. Model calibration and discrimination were judged using the receiver operating characteristics curves and the Hosmer-Lemeshow test.

RESULTS

in the present study, 4,255 patients were included with an 8.5% mortality rate. The risk factors significantly associated with mortality were American Society of Anesthesiologists (ASA) score, age, preoperative diagnosis (OR: 37.82 for intestinal ischemia, OR: 5.01 for colorectal perforation, OR: 6.73 for intestinal obstruction), surgical wound classification and open or laparoscopic surgery. A risk score was devised from these data for the estimation of the probability of survival in each patient. The area under the ROC curve (AUROC) for this score was 0.84 (95% CI: 0.82-0.86) and the AUROC correct was 0.83 (0.81-0.85).

CONCLUSIONS

a simple score that uses five clinical variables predicts 30-day mortality. This model can assist surgeons in the initial evaluation of an elderly patient undergoing urgent abdominal surgery.

Development and validation of delirium prediction model for critically ill adults parameterized to ICU admission acuity

Background

Risk prediction models allow clinicians to forecast which individuals are at a higher risk for developing a particular outcome. We developed and internally validated a delirium prediction model for incident delirium parameterized to patient ICU admission acuity.

Methods

This retrospective, observational, fourteen medical-surgical ICU cohort study evaluated consecutive delirium-free adults surviving hospital stay with ICU length of stay (LOS) greater than or equal to 24 hours with both an admission APACHE II score and an admission type (e.g., elective post-surgery, emergency post-surgery, non-surgical) in whom delirium was assessed using the Intensive Care Delirium Screening Checklist (ICDSC). Risk factors included in the model were readily available in electric medical records. Least absolute shrinkage and selection operator logistic (LASSO) regression was used for model development. Discrimination was determined using area under the receiver operating characteristic curve (AUC). Internal validation was performed by cross-validation. Predictive performance was determined using measures of accuracy and clinical utility was assessed by decision-curve analysis.

Results

A total of 8,878 patients were included. Delirium incidence was 49.9% (n = 4,431). The delirium prediction model was parameterized to seven patient cohorts, admission type (3 cohorts) or mean quartile APACHE II score (4 cohorts). All parameterized cohort models were well calibrated. The AUC ranged from 0.67 to 0.78 (95% confidence intervals [CI] ranged from 0.63 to 0.79). Model accuracy varied across admission types; sensitivity ranged from 53.2% to 63.9% while specificity ranged from 69.0% to 74.6%. Across mean quartile APACHE II scores, sensitivity ranged from 58.2% to 59.7% while specificity ranged from 70.1% to 73.6%. The clinical utility of the parameterized cohort prediction model to predict and prevent incident delirium was greater than preventing incident delirium by treating all or none of the patients.

Conclusions

Our results support external validation of a prediction model parameterized to patient ICU admission acuity to predict a patients’ risk for ICU delirium. Classification of patients’ risk for ICU delirium by admission acuity may allow for efficient initiation of prevention measures based on individual risk profiles.

The Promise of Clinical Decision Support Systems Targetting Low-Resource Settings

Low-resource clinical settings are plagued by low physician-to-patient ratios and a shortage of high-quality medical expertise and infrastructure. Together, these phenomena lead to over-burdened healthcare systems that under-serve the needs of the community. Alleviating this burden can be undertaken by the introduction of clinical decision support systems (CDSSs); systems that support stakeholders (ranging from physicians to patients) within the clinical setting in their day-to-day activities. Such systems, which have proven to be effective in the developed world, remain to be under-explored in low-resource settings. This review attempts to summarize the research focused on clinical decision support systems that either target stakeholders within low-resource clinical settings or diseases commonly found in such environments. When categorizing our findings according to disease applications, we find that CDSSs are predominantly focused on dealing with bacterial infections and maternal care, do not leverage deep learning, and have not been evaluated prospectively. Together, these highlight the need for increased research in this domain in order to impact a diverse set of medical conditions and ultimately improve patient outcomes.

Robotic Gastrectomy for Gastric Adenocarcinoma in the USA: Insights and Oncologic Outcomes in 220 Patients

BACKGROUND

While multiple Asian and a few Western retrospective series have demonstrated the feasibility and safety of robotic-assisted gastrectomy for gastric cancer, its reliability for thorough resection, especially for locoregional disease, has not yet been firmly established, and reported learning curves vary widely. To support wider implementation of robotic gastrectomy, we evaluated the learning curve for this approach, assessed its oncologic feasibility, and created a selection model predicting the likelihood of conversion to open surgery in a US patient population.

PATIENTS AND METHODS

We retrospectively reviewed data on all consecutive patients who underwent robotic gastrectomy at a high-volume institution between May 2012 and March 2019.

RESULTS

Of the 220 patients with gastric cancer selected to undergo curative-intent robotic gastrectomy, surgery was completed using robotics in 159 (72.3%). The median number of removed lymph nodes was 28, and ≥ 15 lymph nodes were removed in 94% of procedures. Surgical time decreased steadily over the first 60-80 cases. Complications were generally minor: 7% of patients experienced complications of grade 3 or higher, with an anastomotic leak rate of 2% and mortality rate 0.9%. Factors predicting conversion to open surgery included neoadjuvant chemotherapy, BMI ≥ 31 kg/m², and tumor size ≥ 6 cm.

CONCLUSIONS

These findings support the safety and oncologic feasibility of robotic gastrectomy for selected patients with gastric cancer. Proficiency can be achieved by 20 cases and mastery by 60-80 cases. Ideal candidates for this approach are patients with few comorbidities, BMI < 31 kg/m², and tumors < 6 cm.