Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

The Ability of Clinical Decision Rules to Detect Peripheral Arterial Disease: A Narrative Review

Peripheral arterial disease (PAD) is a common cause of lower extremity wound. Consequently, PAD leads to a cause of leg amputation nowadays, especially in diabetic patients. In general practice (GP), confrontation with PAD prevention is a challenge. In general, ankle-brachial index (ABI) measurement can be used as a PAD diagnostic tool, but this takes some time. The tool is not generally available and this need to train healthcare workers to perform. Multiple independent predictors developed the diagnostic prediction model known as clinical decision rules (CDRs) to identify patients with high-risk PAD. This might therefore limit the number of patients (only high-risk patients) to refer for ABI evaluation. This narrative review summarized existing CDRs for PAD.

Heterogeneity in mortality risk prediction: a study of vulnerable adults in the Canadian longitudinal study on aging

BACKGROUND

Mortality prediction models are essential for clinical decision-making, but their performance may vary across patient subgroups. This study aimed to evaluate how a general mortality prediction model performs across subgroups defined by vulnerability factors and to test whether model improvements could improve prediction accuracy.

METHODS

We analyzed data from 49,266 participants in the Canadian Longitudinal Study on Aging. A general mortality prediction model (Model A) was developed using Cox proportional hazard regression with LASSO, incorporating variables spanning sociodemographic factors, lifestyle habits, comorbidities, and physical/cognitive function measures. Performance was evaluated across subgroups defined by age, frailty, multimorbidity, cognitive function, and functional impairment using discrimination (c-index), calibration, and Brier scores. We tested two additional strategies: incorporating subgroup-specific variables (Model B) and developing tailored models for different mortality risk categories (Models C1, C2, C3).

RESULTS

Over a median 6-year follow-up, 7.5% (3672) participants died. The general model performed well overall (c-index: 0.82, 95% CI 0.80-0.84; Brier: 0.036, 95% CI 0.032-0.040), but performance varied across subgroups. It was lower in frail individuals (c-index: 0.73, 95% CI 0.71-0.75; Brier: 0.12, 95% CI 0.11-0.13) and those with multiple chronic conditions (c-index: 0.76, 95% CI 0.75-0.78; Brier: 0.08, 95% CI 0.07-0.08), with risk underestimated in these groups. Neither incorporating subgroup variables nor developing risk-stratified models significantly improved performance.

CONCLUSION

Important variability in performance, particularly in vulnerable groups, highlights the limitations of a one-size-fits-all and underscores the need for more granular predictive models that account for subpopulation-specific characteristics to enhance mortality risk prediction.

Comparing the predictive discrimination of machine learning models for ordinal outcomes: A case study of dehydration prediction in patients with acute diarrhea

Many comparisons of statistical regression and machine learning algorithms to build clinical predictive models use inadequate methods to build regression models and do not have proper independent test sets on which to externally validate the models. Proper comparisons for models of ordinal categorical outcomes do not exist. We set out to compare model discrimination for four regression and machine learning methods in a case study predicting the ordinal outcome of severe, some, or no dehydration among patients with acute diarrhea presenting to a large medical center in Bangladesh using data from the NIRUDAK study derivation and validation cohorts. Proportional Odds Logistic Regression (POLR), penalized ordinal regression (RIDGE), classification trees (CART), and random forest (RF) models were built to predict dehydration severity and compared using three ordinal discrimination indices: ordinal c-index (ORC), generalized c-index (GC), and average dichotomous c-index (ADC). Performance was evaluated on models developed on the training data, on the same models applied to an external test set and through internal validation with three bootstrap algorithms to correct for overoptimism. RF had superior discrimination on the original training data set, but its performance was more similar to the other three methods after internal validation using the bootstrap. Performance for all models was lower on the prospective test dataset, with particularly large reduction for RF and RIDGE. POLR had the best performance in the test dataset and was also most efficient, with the smallest final model size. Clinical prediction models for ordinal outcomes, just like those for binary and continuous outcomes, need to be prospectively validated on external test sets if possible because internal validation may give a too optimistic picture of model performance. Regression methods can perform as well as more automated machine learning methods if constructed with attention to potential nonlinear associations. Because regression models are often more interpretable clinically, their use should be encouraged.

Adverse drug reactions and events in an Ageing PopulaTion risk Prediction (ADAPTiP) tool: the development and validation of a model for predicting adverse drug reactions and events in older patients

PURPOSE

Older people are at an increased risk of developing adverse drug reactions (ADR) and adverse drug events (ADE). This study aimed to develop and validate a risk prediction model (ADAPTiP) for ADR/ADE in older populations.

METHODS

We used the adverse drug reactions in an Ageing PopulaTion (ADAPT) cohort (N = 798; 361 ADR-related admissions; 437 non-ADR-related admissions), a cross-sectional study designed to examine the prevalence and risk factors for ADR-related hospital admissions in patients aged ≥ 65 years. Twenty predictors (categorised as sociodemographic-related, functional ability-related, disease-related, and medication-related) were considered in the development of the model. The model was developed using multivariable logistic regression and was internally validated by fivefold cross-validation. The model was externally validated in a separate prospective cohort from the Centre for Primary Care Research (CPCR) study of ADES. The cross-validated and externally validated model performance was evaluated by discrimination and calibration.

RESULTS

The final prediction model, ADAPTiP, included nine predictors: age, chronic lung disease, the primary presenting complaints of respiratory, bleeding and gastrointestinal disorders and syncope on hospital admission and antithrombotics, diuretics, and renin-angiotensin-aldosterone system drug classes. ADAPTiP demonstrated good performance with cross-validated area under the curve of 0.75 [95% CI 0.72;79] and 0.83 [95% CI 0.80;0.87] in the external validation.

CONCLUSION

Using accessible information from medical records, ADAPTiP can help clinicians to identify those older people at risk of an ADR/ADE who should be monitored and/or have their medications reviewed to avoid potentially harmful prescribing.

The revised Canadian Bleeding (CAN-BLEED) score for risk stratification of bleeding trauma patients: a mixed retrospective-prospective cohort study

BACKGROUND

Traumatic hemorrhage is a significant cause of morbidity and mortality. There is considerable interest in risk stratification tools to aid with early activation of intervention pathways for bleeding patients. In this study, we refine the Canadian Bleeding (CAN-BLEED) score for the prediction of major interventions in bleeding trauma patients.

METHODS

We conducted a mixed retrospective-prospective cohort study. We included a retrospective cohort from the CAN-BLEED derivation study, from September 2014 to September 2017. We also conducted a prospective cohort from May 2019 to August 2021 and included both datasets for refinement of the CAN-BLEED score. The primary outcome was major intervention, defined by a composite of massive transfusion, embolization, or surgery for hemostasis. Predictors were pre-specified based on previous validation work. We used a stepdown procedure and regression coefficients to create a clinical risk stratification score. We used bootstrap internal validation to assess optimism-corrected performance.

RESULTS

We included 1368 patients in the overall cohort. Incidence of penetrating injury was 23% and median injury severity score was 17. The overall incidence of the need for major intervention was 17%. The revised score included 8 variables: systolic blood pressure, heart rate, lactate, penetrating mechanism, pelvic instability, Focused Abdominal Sonography for Trauma positive for free fluid, computed tomography positive for free fluid, or contrast extravasation. The C-statistic for the simplified score is 0.89. A score cut-off of less than 2 points yielded a 97% (94-98%) sensitivity in ruling out the need for major intervention.

CONCLUSION

The revised CAN-BLEED score offers a clinically intuitive and internally validated tool with excellent performance in identifying patients requiring major intervention for traumatic bleeding. Further efforts are required to evaluate its performance with an external validation.

Identifying children who develop severe chronic kidney disease using primary care records

BACKGROUND

Understanding whether symptoms suggestive of chronic kidney disease (CKD) are reported to primary care before diagnosis may provide opportunities for earlier detection, thus supporting strategies to prevent progression and improve long-term outcomes. Our aim was to determine whether symptoms/signs or consultation frequency recorded in primary care could predict a subsequent diagnosis of chronic kidney disease in children.

METHODS

We undertook a case-control study within Clinical Practice Research Datalink. Cases were children <21 years with an incident code for severe CKD during the study period (January 2000-September 2018). Controls were matched on age (+/-3 years), sex, and practice-level kidney function testing rate. Conditional logistic regression modelling was used to identify symptoms predictive of severe CKD and differences in consultation frequency in 24- and 6-month timeframes before the index date.

RESULTS

Symptoms predictive of severe CKD in the 24 months before the index date included growth concerns (OR 7.4, 95% CI 3.5, 15.4), oedema (OR 5.7, 95% CI 2.9, 11.2) and urinary tract infection (OR 3.3, 95% CI 2.1, 5.4); within 6 months of the index date, effect estimates and specificity strengthened although sensitivity decreased. Overall, positive predictive value of symptoms was low. Cases consulted more frequently than controls in both timeframes. In combination, symptoms and consultation frequency demonstrated modest discrimination for CKD (c-statistic after bootstrapping 0.70, 95% CI 0.66, 0.73).

CONCLUSION

Despite increased consultation frequency and several symptoms being associated with severe chronic kidney disease, the positive predictive value of symptoms is low given disease rarity making earlier diagnosis challenging.

The Calgary Kids' Hand Rule: External Validation of a Prediction Model to Triage Pediatric Hand Fractures

Background: The Calgary Kids' Hand Rule (CKHR) is a clinical prediction rule intended to guide referral decisions for pediatric hand fractures presenting to the emergency department, identifying "complex" fractures that require surgical referral and optimizing care through better matching of patients' needs to provider expertise. The objective of this study was to externally validate the CKHR in two different tertiary pediatric hospitals in Canada. Methods: We partnered with British Columbia Children's Hospital (BCCH) and the Children's Hospital of Eastern Ontario (CHEO) to externally validate the CKHR using data from retrospective cohorts of pediatric hand fractures (via electronic medical record and x-ray review). Model performance was evaluated at each site using sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and the C-statistic. Results: A total of 954 hand fractures were included in the analysis (524 at BCCH and 430 at CHEO. At BCCH, the CKHR had a sensitivity of 91.1% (133 predicted complex out of 146 total complex fractures), specificity of 71.4% (269 predicted simple out of 377 total simple fractures), and C-statistic of .81, 95% CI [0.78-0.84]. At CHEO, the CKHR had a sensitivity of 98.3%, specificity of 30.2%, and C-statistic of .64, 95% CI [0.61-0.67]. Conclusion: The CKHR performed well at two different tertiary care centres with high sensitivity, supporting its ability to facilitate hand fracture triage in other populations without further modification. This work should be followed by rigorous implementation analysis to determine its impact on patient care.

The definition of predictor and outcome variables in mortality prediction models: a scoping review and quality of reporting study

BACKGROUND AND OBJECTIVES

Mortality prediction models are promising tools for guiding clinical decision-making and resource allocation in intensive care units (ICUs). Clearly specified predictor and outcome variables are necessary to enable external validation and safe clinical application of prediction models. The objective of this study was to identify the predictor and outcome variables used in different mortality prediction models in the ICU and investigate their reporting.

METHODS

For this scoping review, MEDLINE, EMBASE, Web of Science, and the Cochrane Central Register of Controlled Trials were searched. Studies developed within a general ICU population reporting on prediction models with mortality as a primary or secondary outcome were eligible. The selection criteria were adopted from a review by Keuning et al. Predictor and outcome variables, variable characteristics (defined as units, definitions, moments of measurement, and methods of measurement), and publication details (defined as first author, year of publication and title) were extracted from the included studies. Predictor and outcome variable categories were demographics, chronic disease, care logistics, acute diagnosis, clinical examination and physiological derangement, laboratory assessment, additional diagnostics, support and therapy, risk scores, and (mortality) outcomes.

RESULTS

A total of 56 mortality prediction models, containing 204 unique predictor and outcome variables, were included. The predictor variables most frequently included in the models were age (40 times), admission type (27 times), and mechanical ventilation (21 times). We observed that single variables were measured with different units, according to different definitions, at a different moment, and with a different method of measurement in different studies. The reporting of the unit was mostly complete (98% overall, 95% in the laboratory assessment category), whereas the definition of the variable (74% overall, 63% in the chronic disease category) and method of measurement (70% overall, 34% in the demographics category) were most often lacking.

CONCLUSION

Accurate and transparent reporting of predictor and outcome variables is paramount to enhance reproducibility, model performance in different contexts, and validity. Since unclarity about the required input data may introduce bias and thereby affect model performance, this study advocates that prognostic ICU models can be improved by transparent and clear reporting of predictor and outcome variables and their characteristics.

Exploring the acceptability of a risk prediction tool for cardiometabolic risk (gestational diabetes and hypertensive disorders of pregnancy) for use in early pregnancy: A qualitative study

PROBLEM/ BACKGROUND

The acceptability of providing women with personalised cardiometabolic risk information using risk prediction tools early in pregnancy is not well understood.

AIM

To explore women's and healthcare professionals' perspectives of the acceptability of a prognostic, composite risk prediction tool for cardiometabolic risk (gestational diabetes and/or hypertensive disorders of pregnancy) for use in early pregnancy.

METHODS

Semi-structured interviews were conducted to explore the acceptability of cardiometabolic risk prediction tools, preferences for risk communication and considerations for implementation into antenatal care. The Theoretical Framework of Acceptability informed interview questions. Transcripts were thematically analysed.

FINDINGS

Women ≤24 weeks' gestation (n = 13) and healthcare professionals (n = 8), including midwives (n = 2), general practitioners (n = 2), obstetricians (n = 2), an endocrinologist (n = 1) and cardiologist (n = 1) participated. Participants indicated that providing personalised risk information is only appropriate when preventative measures can be initiated to mitigate risks. Differentiating the risk for each condition (single risk outputs) was often preferred to composite risk outputs to enable targeted monitoring and management. Defining conditions and risks to mother/baby, visually depicting personalised risk scores, and providing clear, patient-centred clinical management plans were recommended. Supportive clinical policy changes, staff engagement/training, and integration into electronic health records were suggested to facilitate uptake into routine antenatal care.

CONCLUSION

Women and healthcare professionals suggested that early pregnancy cardiometabolic risk prediction tools may be acceptable when preventative interventions are available to reduce risks. Risk prediction tools with integrated patient-centred education materials may promote timely access and engagement with preventative interventions to optimise women's current and future health.

External Validation of Brief Resolved Unexplained Events Prediction Rules for Serious Underlying Diagnosis

Importance

The American Academy of Pediatrics (AAP) higher-risk criteria for brief resolved unexplained events (BRUE) have a low positive predictive value (4.8%) and misclassify most infants as higher risk (>90%). New BRUE prediction rules from a US cohort of 3283 infants showed improved discrimination; however, these rules have not been validated in an external cohort.

Objective

To externally validate new BRUE prediction rules and compare them with the AAP higher-risk criteria.

Design, Setting, and Participants

This was a retrospective multicenter cohort study conducted from 2017 to 2021 and monitored for 90 days after index presentation. The setting included infants younger than 1 year with a BRUE identified through retrospective chart review from 11 Canadian hospitals. Study data were analyzed from March 2022 to March 2024.

Exposures

The BRUE prediction rules.

Main Outcome and Measure

The primary outcome was a serious underlying diagnosis, defined as conditions where a delay in diagnosis could lead to increased morbidity or mortality.

Results

Of 1042 patients (median [IQR] age, 41 [13-84] days; 529 female [50.8%]), 977 (93.8%) were classified as higher risk by the AAP criteria. A total of 79 patients (7.6%) had a serious underlying diagnosis. For this outcome, the AAP criteria demonstrated a sensitivity of 100.0% (95% CI, 95.4%-100.0%), a specificity of 6.7% (95% CI, 5.2%-8.5%), a positive likelihood ratio (LR+) of 1.07 (95% CI, 1.05-1.09), and an AUC of 0.53 (95% CI, 0.53-0.54). The BRUE prediction rule for discerning serious diagnoses displayed an AUC of 0.60 (95% CI, 0.54-0.67; calibration intercept: 0.60), which improved to an AUC of 0.71 (95% CI, 0.65-0.76; P < .001; calibration intercept: 0.00) after model revision. Event recurrence was noted in 163 patients (15.6%). For this outcome, the AAP criteria yielded a sensitivity of 99.4% (95% CI, 96.6%-100.0%), a specificity of 7.3% (95% CI, 5.7%-9.2%), an LR+ of 1.07 (95% CI, 1.05-1.10), and an AUC of 0.58 (95% CI, 0.56-0.58). The AUC of the prediction rule stood at 0.67 (95% CI, 0.62-0.72; calibration intercept: 0.15).

Conclusions and Relevance

Results of this multicenter cohort study show that the BRUE prediction rules outperformed the AAP higher-risk criteria on external geographical validation, and performance improved after recalibration. These rules provide clinicians and families with a more precise tool to support decision-making, grounded in individual risk tolerance.

In Silico Evaluation of Algorithm-Based Clinical Decision Support Systems: Protocol for a Scoping Review

BACKGROUND

Integrating algorithm-based clinical decision support (CDS) systems poses significant challenges in evaluating their actual clinical value. Such CDS systems are traditionally assessed via controlled but resource-intensive clinical trials.

OBJECTIVE

This paper presents a review protocol for preimplementation in silico evaluation methods to enable broadened impact analysis under simulated environments before clinical trials.

METHODS

We propose a scoping review protocol that follows an enhanced Arksey and O'Malley framework and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines to investigate the scope and research gaps in the in silico evaluation of algorithm-based CDS models-specifically CDS decision-making end points and objectives, evaluation metrics used, and simulation paradigms used to assess potential impacts. The databases searched are PubMed, Embase, CINAHL, PsycINFO, Cochrane, IEEEXplore, Web of Science, and arXiv. A 2-stage screening process identified pertinent articles. The information extracted from articles was iteratively refined. The review will use thematic, trend, and descriptive analyses to meet scoping aims.

RESULTS

We conducted an automated search of the databases above in May 2023, with most title and abstract screenings completed by November 2023 and full-text screening extended from December 2023 to May 2024. Concurrent charting and full-text analysis were carried out, with the final analysis and manuscript preparation set for completion in July 2024. Publication of the review results is targeted from July 2024 to February 2025. As of April 2024, a total of 21 articles have been selected following a 2-stage screening process; these will proceed to data extraction and analysis.

CONCLUSIONS

We refined our data extraction strategy through a collaborative, multidisciplinary approach, planning to analyze results using thematic analyses to identify approaches to in silico evaluation. Anticipated findings aim to contribute to developing a unified in silico evaluation framework adaptable to various clinical workflows, detailing clinical decision-making characteristics, impact measures, and reusability of methods. The study's findings will be published and presented in forums combining artificial intelligence and machine learning, clinical decision-making, and health technology impact analysis. Ultimately, we aim to bridge the development-deployment gap through in silico evaluation-based potential impact assessments.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/63875.

Derivation and validation of a clinical predictive model for longer duration diarrhea among pediatric patients in Kenya using machine learning algorithms

BACKGROUND

Despite the adverse health outcomes associated with longer duration diarrhea (LDD), there are currently no clinical decision tools for timely identification and better management of children with increased risk. This study utilizes machine learning (ML) to derive and validate a predictive model for LDD among children presenting with diarrhea to health facilities.

METHODS

LDD was defined as a diarrhea episode lasting ≥ 7 days. We used 7 ML algorithms to build prognostic models for the prediction of LDD among children < 5 years using de-identified data from Vaccine Impact on Diarrhea in Africa study (N = 1,482) in model development and data from Enterics for Global Health Shigella study (N = 682) in temporal validation of the champion model. Features included demographic, medical history and clinical examination data collected at enrolment in both studies. We conducted split-sampling and employed K-fold cross-validation with over-sampling technique in the model development. Moreover, critical predictors of LDD and their impact on prediction were obtained using an explainable model agnostic approach. The champion model was determined based on the area under the curve (AUC) metric. Model calibrations were assessed using Brier, Spiegelhalter's z-test and its accompanying p-value.

RESULTS

There was a significant difference in prevalence of LDD between the development and temporal validation cohorts (478 [32.3%] vs 69 [10.1%]; p < 0.001). The following variables were associated with LDD in decreasing order: pre-enrolment diarrhea days (55.1%), modified Vesikari score(18.2%), age group (10.7%), vomit days (8.8%), respiratory rate (6.5%), vomiting (6.4%), vomit frequency (6.2%), rotavirus vaccination (6.1%), skin pinch (2.4%) and stool frequency (2.4%). While all models showed good prediction capability, the random forest model achieved the best performance (AUC [95% Confidence Interval]: 83.0 [78.6-87.5] and 71.0 [62.5-79.4]) on the development and temporal validation datasets, respectively. While the random forest model showed slight deviations from perfect calibration, these deviations were not statistically significant (Brier score = 0.17, Spiegelhalter p-value = 0.219).

CONCLUSIONS

Our study suggests ML derived algorithms could be used to rapidly identify children at increased risk of LDD. Integrating ML derived models into clinical decision-making may allow clinicians to target these children with closer observation and enhanced management.

Prediction models for outcomes in people with low back pain receiving conservative treatment: a systematic review

OBJECTIVES

To identify, critically appraise and evaluate the performance measures of the available prediction models for outcomes in people with low back pain (LBP) receiving conservative treatment.

STUDY DESIGN AND SETTING

In this systematic review, literature searches were conducted in Embase, Medline, and cumulative index of nursing and allied health literature from their inception until February 2024. Studies containing follow-up assessment (eg, prospective cohort studies, registry-based studies) investigating prediction models of outcomes (eg, pain intensity and disability) for people with LBP receiving conservative treatment were included. Two independent reviewers performed the study selection, the data extraction using the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies, and risk of bias assessment using the Prediction model Risk of Bias Assessment. Findings of individual studies were reported narratively taking into account the discrimination and calibration measures of the prediction models.

RESULTS

Seventy-five studies developing or investigating the validity of 216 models were included in this review. Most prediction models investigated people receiving physiotherapy treatment and most models included sociodemographic variables, clinical features, and self-reported measures as predictors. The discriminatory capacity of the internal validity of the 27 prediction models for pain intensity varied greatly showing a c-statistic ranging from 0.48 to 0.94. Similarly, the discriminatory capacity for 31 models for disability had the same pattern showing a c-statistic ranging from 0.48 to 0.86. The calibration measures of the internal validity of the prediction models predicting pain intensity and disability showed to be adequate. Only one of 3 studies testing the external validity of models to predict pain intensity and disability and reported both discrimination and calibration measures, which showed to be inadequate. The prediction models predicting the secondary outcomes (eg, self-reported recovery, quality of life, return to work) showed varied performance measures for internal validity, and only 2 studies tested the external validity of models although they did not provide performance the performance measures.

CONCLUSION

Several prediction models have been developed for people with LBP receiving conservative treatment; however, most show inadequate discriminatory validity. A few studies externally validated the prediction models and future studies should focus on testing this before implementing in clinical practice.

Injury severity bias in missing prehospital vital signs: Prevalence and implications for trauma registries

BACKGROUND

Vital signs are important factors in assessing injury severity and guiding trauma resuscitation, especially among severely injured patients. Despite this, physiological data are frequently missing from trauma registries. This study aimed to evaluate the extent of missing prehospital data in a hospital-based trauma registry and to assess the associations between prehospital physiological data completeness and indicators of injury severity.

METHODS

A retrospective review was conducted on all adult trauma patients brought directly to a level 1 trauma center in Toronto, Ontario by paramedics from January 1, 2015, to December 31, 2019. The proportion of missing data was evaluated for each variable and patterns of missingness were assessed. To investigate the associations between prehospital data completeness and injury severity factors, descriptive and unadjusted logistic regression analyses were performed.

RESULTS

A total of 3,528 patients were included. We considered prehospital data missing if any of heart rate, systolic blood pressure, respiratory rate or oxygen saturation were incomplete. Each individual variable was missing from the registry in approximately 20 % of patients, with oxygen saturation missing most frequently (n = 831; 23.6 %). Over 25 % (n = 909) of patients were missing at least one prehospital vital sign, of which 69.1 % (n = 628) were missing all four of these variables. Patients with incomplete data were more severely injured, had higher mortality, and more frequently received lifesaving interventions such as blood transfusion and intubation. Patients were most likely to have missing prehospital physiological data if they died in the trauma bay (unadjusted OR: 9.79; 95 % CI: 6.35-15.10), did not survive to discharge (unadjusted OR: 3.55; 95 % CI: 2.76-4.55), or had a prehospital GCS less than 9 (OR: 3.24; 95 % CI: 2.59-4.06).

CONCLUSION

In this single center trauma registry, key prehospital variables were frequently missing, particularly among more severely injured patients. Patients with missing data had higher mortality, more severe injury characteristics and received more life-saving interventions in the trauma bay, suggesting an injury severity bias in prehospital vital sign missingness. To ensure the validity of research based on trauma registry data, patterns of missingness must be carefully considered to ensure missing data is appropriately addressed.

Movement Clearing Screens for Military Service Member Musculoskeletal Injury Risk Identification

CONTEXT

Pain during movement screens is a risk factor for musculoskeletal injury (MSKI). Movement screens often require specialized or clinical expertise and large amounts of time to administer.

OBJECTIVE

Evaluate if self-reported pain (1) with movement clearing screens is a risk factor for any MSKI, (2) with movement clearing screens is a risk factor for body region-specific MSKIs, and (3) with a greater number of movement clearing screens progressively increases MSKI risk.

DESIGN

Retrospective cohort study.

SETTING

Field-based.

PATIENTS OR OTHER PARTICIPANTS

Military service members (n = 4222).

MAIN OUTCOME MEASURE(S)

Active-duty service members self-reported pain during movement clearing screens (Shoulder Clearing, Spinal Extension, Squat-Jump-Land). Musculoskeletal injury data were abstracted up to 180 days post-screening. A traffic light model grouped service members if they self-reported pain during 0 (Green), 1 (Amber), 2 (Red), or 3 (Black) movement clearing screens. Cox proportional hazards models adjusted for age, gender, body mass index, and prior MSKI determined the relationships between pain during movement clearing screens with any and body region-specific MSKIs.

RESULTS

Service members self-reporting pain during the Shoulder Clearing (adjusted hazard ratio and 95% confidence interval [HRadj (95% CI)] = 1.58 [1.37, 1.82]), Spinal Extension (HRadj = 1.48 [1.28, 1.87]), or Squat-Jump-Land (HRadj = 2.04 [1.79, 2.32]) tests were more likely to experience any MSKI than service members reporting no pain. Service members with pain during the Shoulder Clearing (HRadj = 3.28 [2.57, 4.19]), Spinal Extension (HRadj = 2.80 [2.26, 3.49]), or Squat-Jump-Land (HRadj = 2.07 [1.76, 2.43]) tests were more likely to experience an upper extremity, spine, back, and torso, or lower extremity MSKI, respectively, than service members reporting no pain. The Amber (HRadj = 1.69 [1.48, 1.93]), Red (HRadj = 2.07 [1.73, 2.48]), and Black (HRadj = 2.31 [1.81, 2.95]) cohorts were more likely to experience an MSKI than the Green cohort.

CONCLUSIONS

Self-report movement clearing screens in combination with a traffic light model provide clinician- and nonclinician-friendly expedient means to identify service members at MSKI risk.

Modifiable prognostic factors of high societal costs among people on sick leave due to musculoskeletal disorders: a replication study

BACKGROUND

Musculoskeletal disorders are an extensive burden to society, yet few studies have explored and replicated modifiable prognostic factors associated with high societal costs. This study aimed to replicate previously identified associations between nine modifiable prognostic factors and high societal costs among people on sick leave due to musculoskeletal disorders.

METHODS

Pooled data from a three-arm randomised controlled trial with 6 months of follow-up were used, including 509 participants on sick leave due to musculoskeletal disorders in Norway. Consistent with the identification study, the primary outcome was societal costs dichotomised as high (top 25th percentile) or low. Societal costs included healthcare utilization (primary, secondary, and tertiary care) and productivity loss (absenteeism, work assessment allowance and disability benefits) collected from public records. Binary unadjusted and adjusted logistic regression analyses were used to replicate previously identified associations between each modifiable prognostic factor and having high costs.

RESULTS

Adjusted for selected covariates, a lower degree of return-to-work expectancy was associated with high societal costs in both the identification and replication sample. Depressive symptoms and health literacy showed no prognostic value in both the identification and replication sample. There were inconsistent results with regards to statistical significance across the identification and replication sample for pain severity, self-perceived health, sleep quality, work satisfaction, disability, and long-lasting disorder expectation. Similar results were found when high costs were related to separately healthcare utilization and productivity loss.

CONCLUSION

This study successfully replicated the association between return-to-work expectancy and high societal costs among people on sick leave due to musculoskeletal disorders. Other factors showed no prognostic value or inconsistent results.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03871712, 12th of March 2019.

Predictive modelling of linear growth faltering among pediatric patients with Diarrhea in Rural Western Kenya: an explainable machine learning approach

INTRODUCTION

Stunting affects one-fifth of children globally with diarrhea accounting for an estimated 13.5% of stunting. Identifying risk factors for its precursor, linear growth faltering (LGF), is critical to designing interventions. Moreover, developing new predictive models for LGF using more recent data offers opportunity to enhance model accuracy, interpretability and capture new insights. We employed machine learning (ML) to derive and validate a predictive model for LGF among children enrolled with diarrhea in the Vaccine Impact on Diarrhea in Africa (VIDA) study and the Enterics for Global Heath (EFGH) - Shigella study in rural western Kenya.

METHODS

We used 7 diverse ML algorithms to retrospectively build prognostic models for the prediction of LGF (≥ 0.5 decrease in height/length for age z-score [HAZ]) among children 6-35 months. We used de-identified data from the VIDA study (n = 1,106) combined with synthetic data (n = 8,894) in model development, which entailed split-sampling and K-fold cross-validation with over-sampling technique, and data from EFGH-Shigella study (n = 655) for temporal validation. Potential predictors (n = 65) included demographic, household-level characteristics, illness history, anthropometric and clinical data were identified using boruta feature selection with an explanatory model analysis used to enhance interpretability.

RESULTS

The prevalence of LGF in the development and temporal validation cohorts was 187 (16.9%) and 147 (22.4%), respectively. Feature selection identified the following 6 variables used in model development, ranked by importance: age (16.6%), temperature (6.0%), respiratory rate (4.1%), SAM (3.4%), rotavirus vaccination (3.3%), and skin turgor (2.1%). While all models showed good prediction capability, the gradient boosting model achieved the best performance (area under the curve % [95% Confidence Interval]: 83.5 [81.6-85.4] and 65.6 [60.8-70.4]) on the development and temporal validation datasets, respectively.

CONCLUSION

Our findings accentuate the enduring relevance of established predictors of LGF whilst demonstrating the practical utility of ML algorithms for rapid identification of at-risk children.

A Blueprint for Clinical-Driven Medical Device Development: The Feverkidstool Application to Identify Children With Serious Bacterial Infection

Clinical decision rules (CDRs) integrated into applications enhance diagnostic and treatment prediction support for clinicians, necessitating Confirmité Europeenne (CE)-mark certification to enter the European market. We describe the development of a CDR as a medical device, focusing on challenges from a physician's perspective exemplified by the Feverkidstool (FKT), a validated CDR for febrile children. We pursued a local process, aligned with the CE-marking process, to develop the FKT as in-house developed device. We aimed to provide a blueprint for colleagues. Medical device development, conforming the medical device regulation and performed by a multidisciplinary team, encompassed 5 stages: market scan, design, production, verification and validation and conformity assessment. Regulatory processes were continuously updated. The market scan identified a need for the FKT compared with existing applications. A prototype was designed in stage 2, further adjusted and improved based on the qualitative and quantitative results of stages 2-4. Lastly, stage 5 confirmed FKT's performance and safety. Medical device development presents challenges for physicians, requiring collaboration for technical, regulatory, and financial expertise. Multidisciplinary teamwork also poses challenges, including uncertainties regarding responsibility and timelines. After CE certification, adapting to evolving needs and ensuring data privacy highlights the ongoing nature of medical device development.

Prediction models for COVID-19 disease outcomes

SARS-CoV-2 has caused over 6.9 million deaths and continues to produce lasting health consequences. COVID-19 manifests broadly from no symptoms to death. In a retrospective cross-sectional study, we developed personalized risk assessment models that predict clinical outcomes for individuals with COVID-19 and inform targeted interventions. We sequenced viruses from SARS-CoV-2-positive nasopharyngeal swab samples between July 2020 and July 2022 from 4450 individuals in Missouri and retrieved associated disease courses, clinical history, and urban-rural classification. We integrated this data to develop machine learning-based predictive models to predict hospitalization, ICU admission, and long COVID.The mean age was 38.3 years (standard deviation = 21.4) with 55.2% (N = 2453) females and 44.8% (N = 1994) males (not reported, N = 4). Our analyses revealed a comprehensive set of predictors for each outcome, encompassing human, environment, and virus genome-wide genetic markers. Immunosuppression, cardiovascular disease, older age, cardiac, gastrointestinal, and constitutional symptoms, rural residence, and specific amino acid substitutions were associated with hospitalization. ICU admission was associated with acute respiratory distress syndrome, ventilation, bacterial co-infection, rural residence, and non-wild type SARS-CoV-2 variants. Finally, long COVID was associated with hospital admission, ventilation, and female sex.Overall, we developed risk assessment models that offer the capability to identify patients with COVID-19 necessitating enhanced monitoring or early interventions. Of importance, we demonstrate the value of including key elements of virus, host, and environmental factors to predict patient outcomes, serving as a valuable platform in the field of personalized medicine with the potential for adaptation to other infectious diseases.

A beginner's guide to supervised analysis for mass cytometry data in cancer biology

Mass cytometry enables deep profiling of biological samples at single-cell resolution. This technology is more than relevant in cancer research due to high cellular heterogeneity and complexity. Downstream analysis of high-dimensional datasets increasingly relies on machine learning (ML) to extract clinically relevant information, including supervised algorithms for classification and regression purposes. In cancer research, they are used to develop predictive models that will guide clinical decision making. However, the development of supervised algorithms faces major challenges, such as sufficient validation, before being translated into the clinics. In this work, we provide a framework for the analysis of mass cytometry data with a specific focus on supervised algorithms and practical examples of their applications. We also raise awareness on key issues regarding good practices for researchers curious to implement supervised ML on their mass cytometry data. Finally, we discuss the challenges of supervised ML application to cancer research.

Grasping the big picture: impact analysis of screening tools for timely referral for device-aided therapies

Several screening tools are available to assist general neurologists in the timely identification of patients with advanced Parkinson's disease (PD) who may be eligible for referral for a device-aided therapy (DAT). However, it should be noted that not all of these clinical decision rules have been developed and validated in a thorough and consistent manner. Furthermore, only a limited number of head-to-head comparisons have been performed. Available studies suggest that D-DATS has a higher positive predictive value and higher specificity than the 5-2-1 criteria, while the sensitivity of both screening tools is similar. However, unanswered questions remain regarding the validity of the decision rules, such as whether the diagnostic performance measures from validation studies are generalizable to other populations. Ultimately, the question is whether a screening tool will effectively and efficiently improve the quality of life of patients with PD. To address this key question, an impact analysis should be performed. The authors intend to set up a multinational cluster randomised controlled trial to compare the D-DATS and 5-2-1 criteria on the downstream consequences of implementing these screening tools, with a particular focus on the impact on disability and quality of life.

Prediction models for diagnosis and prognosis of the colonization or infection of multidrug-resistant organisms in adults: a systematic review, critical appraisal, and meta-analysis

BACKGROUND

Prediction models help to target patients at risk of multidrug-resistant organism (MDRO) colonization or infection and could serve as tools informing clinical practices to prevent MDRO transmission and inappropriate empiric antibiotic therapy. However, there is limited evidence to identify which among the available models are of low risk of bias and suitable for clinical application.

OBJECTIVES

To identify, describe, appraise, and summarise the performance of all prognostic and diagnostic models developed or validated for predicting MDRO colonization or infection.

DATA SOURCES

Six electronic literature databases and clinical registration databases were searched until April 2022.

STUDY ELIGIBILITY CRITERIA

Development and validation studies of any multivariable prognostic and diagnostic models to predict MDRO colonization or infection in adults.

PARTICIPANTS

Adults (≥ 18 years old) without MDRO colonization or infection (in prognostic models) or with unknown or suspected MDRO colonization or infection (in diagnostic models).

ASSESSMENT OF RISK OF BIAS

The Prediction Model Risk of Bias Assessment Tool was used to assess the risk of bias. Evidence certainty was assessed using the Grading of Recommendations Assessment, Development, and Evaluation approach.

METHODS OF DATA SYNTHESIS

Meta-analyses were conducted to summarize the discrimination and calibration of the models' external validations conducted in at least two non-overlapping datasets.

RESULTS

We included 162 models (108 studies) developed for diagnosing (n = 135) and predicting (n = 27) MDRO colonization or infection. Models exhibited a high-risk of bias, especially in statistical analysis. High-frequency predictors were age, recent invasive procedures, antibiotic usage, and prior hospitalization. Less than 25% of the models underwent external validations, with only seven by independent teams. Meta-analyses for one diagnostic and two prognostic models only produced very low to low certainty of evidence.

CONCLUSIONS

The review comprehensively described the models for identifying patients at risk of MDRO colonization or infection. We cannot recommend which models are ready for application because of the high-risk of bias, limited validations, and low certainty of evidence from meta-analyses, indicating a clear need to improve the conducting and reporting of model development and external validation studies to facilitate clinical application.

Machine Learning-Based Predictive Models for Patients with Venous Thromboembolism: A Systematic Review

BACKGROUND

 Venous thromboembolism (VTE) is a chronic disorder with a significant health and economic burden. Several VTE-specific clinical prediction models (CPMs) have been used to assist physicians in decision-making but have several limitations. This systematic review explores if machine learning (ML) can enhance CPMs by analyzing extensive patient data derived from electronic health records. We aimed to explore ML-CPMs' applications in VTE for risk stratification, outcome prediction, diagnosis, and treatment.

METHODS

 Three databases were searched: PubMed, Google Scholar, and IEEE electronic library. Inclusion criteria focused on studies using structured data, excluding non-English publications, studies on non-humans, and certain data types such as natural language processing and image processing. Studies involving pregnant women, cancer patients, and children were also excluded. After excluding irrelevant studies, a total of 77 studies were included.

RESULTS

 Most studies report that ML-CPMs outperformed traditional CPMs in terms of receiver operating area under the curve in the four clinical domains that were explored. However, the majority of the studies were retrospective, monocentric, and lacked detailed model architecture description and external validation, which are essential for quality audit. This review identified research gaps and highlighted challenges related to standardized reporting, reproducibility, and model comparison.

CONCLUSION

 ML-CPMs show promise in improving risk assessment and individualized treatment recommendations in VTE. Apparently, there is an urgent need for standardized reporting and methodology for ML models, external validation, prospective and real-world data studies, as well as interventional studies to evaluate the impact of artificial intelligence in VTE.

Prognostic prediction models for adverse birth outcomes: A systematic review

Background

Despite progress in reducing maternal and child mortality worldwide, adverse birth outcomes such as preterm birth, low birth weight (LBW), small for gestational age (SGA), and stillbirth continue to be a major global health challenge. Developing a prediction model for adverse birth outcomes allows for early risk detection and prevention strategies. In this systematic review, we aimed to assess the performance of existing prediction models for adverse birth outcomes and provide a comprehensive summary of their findings.

Methods

We used the Population, Index prediction model, Comparator, Outcome, Timing, and Setting (PICOTS) approach to retrieve published studies from PubMed/MEDLINE, Scopus, CINAHL, Web of Science, African Journals Online, EMBASE, and Cochrane Library. We used WorldCat, Google, and Google Scholar to find the grey literature. We retrieved data before 1 March 2022. Data were extracted using CHecklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies. We assessed the risk of bias with the Prediction Model Risk of Bias Assessment tool. We descriptively reported the results in tables and graphs.

Results

We included 115 prediction models with the following outcomes: composite adverse birth outcomes (n = 6), LBW (n = 17), SGA (n = 23), preterm birth (n = 71), and stillbirth (n = 9). The sample sizes ranged from composite adverse birth outcomes (n = 32-549), LBW (n = 97-27 233), SGA (n = 41-116 070), preterm birth (n = 31-15 883 784), and stillbirth (n = 180-76 629). Only nine studies were conducted on low- and middle-income countries. 10 studies were externally validated. Risk of bias varied across studies, in which high risk of bias was reported on prediction models for SGA (26.1%), stillbirth (77.8%), preterm birth (31%), LBW (23.5%), and composite adverse birth outcome (33.3%). The area under the receiver operating characteristics curve (AUROC) was the most used metric to describe model performance. The AUROC ranged from 0.51 to 0.83 in studies that reported predictive performance for preterm birth. The AUROC for predicting SGA, LBW, and stillbirth varied from 0.54 to 0.81, 0.60 to 0.84, and 0.65 to 0.72, respectively. Maternal clinical features were the most utilised prognostic markers for preterm and LBW prediction, while uterine artery pulsatility index was used for stillbirth and SGA prediction.

Conclusions

A varied prognostic factors and heterogeneity between studies were found to predict adverse birth outcomes. Prediction models using consistent prognostic factors, external validation, and adaptation of future risk prediction models for adverse birth outcomes was recommended at different settings.

Registration

PROSPERO CRD42021281725.

Protein Biomarkers in Lung Cancer Screening: Technical Considerations and Feasibility Assessment

Lung cancer remains the leading cause of cancer-related deaths worldwide, mainly due to late diagnosis and the presence of metastases. Several countries around the world have adopted nation-wide LDCT-based lung cancer screening that will benefit patients, shifting the stage at diagnosis to earlier stages with more therapeutic options. Biomarkers can help to optimize the screening process, as well as refine the TNM stratification of lung cancer patients, providing information regarding prognostics and recommending management strategies. Moreover, novel adjuvant strategies will clearly benefit from previous knowledge of the potential aggressiveness and biological traits of a given early-stage surgically resected tumor. This review focuses on proteins as promising biomarkers in the context of lung cancer screening. Despite great efforts, there are still no successful examples of biomarkers in lung cancer that have reached the clinics to be used in early detection and early management. Thus, the field of biomarkers in early lung cancer remains an evident unmet need. A more specific objective of this review is to present an up-to-date technical assessment of the potential use of protein biomarkers in early lung cancer detection and management. We provide an overview regarding the benefits, challenges, pitfalls and constraints in the development process of protein-based biomarkers. Additionally, we examine how a number of emerging protein analytical technologies may contribute to the optimization of novel robust biomarkers for screening and effective management of lung cancer.

Risk Assessment Tool in Predicting the Therapeutic Outcomes of Antiseizure Medication in Adults with Epilepsy

Aim

Identifying a patient's risk for poor outcomes after starting antiseizure medication (ASM) therapy is crucial in managing epilepsy pharmacologically. To date, there is a lack of designated tools to assess such risks.

Purpose

To develop and validate a risk assessment tool for the therapeutic outcomes of ASM therapy.

Patients and Methods

A cross-sectional study was carried out in a hospital-based specialist clinic from September 2022 to August 2023. Data was analyzed from patients' medical records and face-to-face assessments. The seizure control domain was determined from the patients' medical records while seizure severity (SS) and adverse effects (AE) of ASM were assessed using the Seizure Severity Questionnaire and the Liverpool Adverse Event Profile respectively. The developed tool was devised from prediction models using logistic and linear regressions. Concurrent validity and interrater reliability methods were employed for validity assessments.

Results

A total of 397 patients were included in the analysis. For seizure control, the identified predictors include ≥10 years' epilepsy duration (OR:1.87,95% CI:1.10-3.17), generalized onset (OR:7.42,95% CI:2.95-18.66), focal onset seizure (OR:8.24,95% CI:2.98-22.77), non-adherence (OR:3.55,95% CI:1.52-8.27) and having ≥3 ASM (OR:3.29,95% CI:1.32-8.24). Younger age at epilepsy onset (≤40) (OR:3.29,95% CI:1.32-8.24) and neurological deficit (OR:3.55,95% CI:1.52-8.27) were significant predictors for SS. For AE, the positive predictors were age >35 (OR:0.12,95% CI:0.03-0.20), <13 years epilepsy duration (OR:2.89,95% CI:0.50-5.29) and changes in ASM regimen (OR:2.93,95% CI: 0.24-5.62). The seizure control domain showed a good discriminatory ability with a c-index of 0.711. From the Bonferroni (ANOVA) analysis, only SS predicted scores generated a linear plot against the mean of the actual scores. The AE domain was omitted from the final tool because it did not meet the requirements for validity assessment.

Conclusion

This newly developed tool (RAS-TO) is a promising tool that could help healthcare providers in determining optimal treatment strategies for adults with epilepsy.

External validation of Finnish diabetes risk score (FINDRISC) and Latin American FINDRISC for screening of undiagnosed dysglycemia: Analysis in a Peruvian hospital health care workers sample

AIMS

To evaluate the external validity of Finnish diabetes risk score (FINDRISC) and Latin American FINDRISC (LAFINDRISC) for undiagnosed dysglycemia in hospital health care workers.

METHODS

We carried out a cross-sectional study on health workers without a prior history of diabetes mellitus (DM). Undiagnosed dysglycemia (prediabetes or diabetes mellitus) was defined using fasting glucose and two-hour oral glucose tolerance test. LAFINDRISC is an adapted version of FINDRISC with different waist circumference cut-off points. We calculated the area under the receptor operational characteristic curve (AUROC) and explored the best cut-off point.

RESULTS

We included 549 participants in the analysis. The frequency of undiagnosed dysglycemia was 17.8%. The AUROC of LAFINDRISC and FINDRISC were 71.5% and 69.2%; p = 0.007, respectively. The optimal cut-off for undiagnosed dysglycemiaaccording to Index Youden was ≥ 11 in LAFINDRISC (Sensitivity: 78.6%; Specificity: 51.7%) and ≥12 in FINDRISC (Sensitivity: 70.4%; Specificity: 53.9%).

CONCLUSION

The discriminative capacity of both questionnaires is good for the diagnosis of dysglycemia in the healthcare personnel of the María Auxiliadora hospital. The LAFINDRISC presented a small statistical difference, nontheless clinically similar, since there was no difference by age or sex. Further studies in the general population are required to validate these results.

Point-of-Care Noninvasive Prediction of Liver-Related Events in Patients With Nonalcoholic Fatty Liver Disease

BACKGROUND & AIMS

Individual risk prediction of liver-related events (LRE) is needed for clinical assessment of nonalcoholic fatty liver disease (NAFLD)/nonalcoholic steatohepatitis (NASH) patients. We aimed to provide point-of-care validated liver stiffness measurement (LSM)-based risk prediction models for the development of LRE in patients with NAFLD, focusing on selecting patients for clinical trials at risk of clinical events.

METHODS

Two large multicenter cohorts were evaluated, 2638 NAFLD patients covering all LSM values as the derivation cohort and 679 more advanced patients as the validation cohort. We used Cox regression to develop and validate risk prediction models based on LSM alone, and the ANTICIPATE and ANTICIPATE-NASH models for clinically significant portal hypertension. The main outcome of the study was the rate of LRE in the first 3 years after initial assessment.

RESULTS

The 3 predictive models had similar performance in the derivation cohort with a very high discriminative value (c-statistic, 0.87-0.91). In the validation cohort, the LSM-LRE alone model had a significant inferior discrimination (c-statistic, 0.75) compared with the other 2 models, whereas the ANTICIPATE-NASH-LRE model (0.81) was significantly better than the ANTICIPATE-LRE model (0.79). In addition, the ANTICIPATE-NASH-LRE model presented very good calibration in the validation cohort (integrated calibration index, 0.016), and was better than the ANTICIPATE-LRE model.

CONCLUSIONS

The ANTICIPATE-LRE models, and especially the ANTICIPATE-NASH-LRE model, could be valuable validated clinical tools to individually assess the risk of LRE at 3 years in patients with NAFLD/NASH.

Stretching and muscle-performance exercises for chronic nonspecific neck pain: who may benefit most?

BACKGROUND

Although exercise is the mainstay of treatment for neck pain (NP), uncertainty remains over optimal decision-making concerning who may benefit most from such, particularly in the long term.

OBJECTIVE

To identify the subgroup of patients with nonspecific NP most likely to benefit from stretching and muscle-performance exercises.

METHODS

This was a secondary analysis of treatment outcomes of 70 patients (10 of whom dropped out) with a primary complaint of nonspecific NP in one treatment arm of a prospective, randomized, controlled trial. All patients performed the exercises, twice weekly for 6 weeks, and a home exercise program. Blinded outcome measurements were collected at baseline, after the 6-week program, and at a 6-month follow-up. Patients rated their perceived recovery on a 15-point global rating of change scale; a rating of "quite a bit better" (+5) or higher was defined as a successful outcome. Clinical predictor variables were developed via logistic regression analysis to classify patients with NP that may benefit from exercise-based treatment.

RESULTS

NP duration since onset≤6 months, no cervicogenic headache, and shoulder protraction were independent predictor variables. The pretest probability of success was 47% after the 6-week intervention and 40% at the 6-month follow-up. The corresponding posttest probabilities of success for participants with all three variables were 86% and 71%, respectively; such participants were likely to recover.

CONCLUSION

The clinical predictor variables developed in this study may identify patients with nonspecific NP likely to benefit most from stretching and muscle-performance exercises in the short and long terms.

Evaluating classification tools for the prediction of in-vitro microbial pyruvate yield from organic carbon sources

The laboratory-scale (in-vitro) microbial fermentation based on screening of process parameters (factors) and statistical validation of parameters (responses) using regression analysis. The recent trends have shifted from full factorial design towards more complex response surface methodology designs such as Box-Behnken design, Central Composite design. Apart from the optimisation methodologies, the listed designs are not flexible enough in deducing properties of parameters in terms of class variables. Machine learning algorithms have unique visualisations for the dataset presented with appropriate learning algorithms. The classification algorithms cannot be applied on all datasets and selection of classifier is essential in this regard. To resolve this issue, factor-response relationship needs to be evaluated as dataset and subsequent preprocessing could lead to appropriate results. The aim of the current study was to investigate the data-mining accuracy on the dataset developed using in-vitro pyruvate production using organic sources for the first time. The attributes were subjected to comparative classification on various classifiers and based on accuracy, multilayer perceptron (neural network algorithm) was selected as classifier. As per the results, the model showed significant results for prediction of classes and a good fit. The learning curve developed also showed the datasets converging and were linearly separable.

Identifying Predictor Variables for a Composite Risk Prediction Tool for Gestational Diabetes and Hypertensive Disorders of Pregnancy: A Modified Delphi Study

A composite cardiometabolic risk prediction tool will support the systematic identification of women at increased cardiometabolic risk during pregnancy to enable early screening and intervention. This study aims to identify and select predictor variables for a composite risk prediction tool for cardiometabolic risk (gestational diabetes mellitus and/or hypertensive disorders of pregnancy) for use in the first trimester. A two-round modified online Delphi study was undertaken. A prior systematic literature review generated fifteen potential predictor variables for inclusion in the tool. Multidisciplinary experts (n = 31) rated the clinical importance of variables in an online survey and nominated additional variables for consideration (Round One). An online meeting (n = 14) was held to deliberate the importance, feasibility and acceptability of collecting variables in early pregnancy. Consensus was reached in a second online survey (Round Two). Overall, 24 variables were considered; 9 were eliminated, and 15 were selected for inclusion in the tool. The final 15 predictor variables related to maternal demographics (age, ethnicity/race), pre-pregnancy history (body mass index, height, history of chronic kidney disease/polycystic ovarian syndrome, family history of diabetes, pre-existing diabetes/hypertension), obstetric history (parity, history of macrosomia/pre-eclampsia/gestational diabetes mellitus), biochemical measures (blood glucose levels), hemodynamic measures (systolic blood pressure). Variables will inform the development of a cardiometabolic risk prediction tool in subsequent research. Evidence-based, clinically relevant and routinely collected variables were selected for a composite cardiometabolic risk prediction tool for early pregnancy.

Clinical prediction models in children that use repeated measurements with time-varying covariates: a scoping review

BACKGROUND

Emerging evidence suggests that clinical prediction models that use repeated (time-varying) measurements within each patient may have higher predictive accuracy than models that use patient information from a single measurement.

OBJECTIVE

To determine the breadth of the published literature reporting the development of clinical prediction models in children that use time-varying predictors.

DATA SOURCES

MEDLINE, EMBASE and Cochrane databases.

ELIGIBILITY CRITERIA

We included studies reporting the development of a multivariable clinical prediction model in children, with or without validation, to predict a repeatedly measured binary or time-to-event outcome and utilizing at least one repeatedly measured predictor.

SYNTHESIS METHODS

We categorized included studies by the method used to model time-varying predictors.

RESULTS

Of 99 clinical prediction model studies that had a repeated measurements data structure, only 27 (27%) used methods that incorporated the repeated measurements as time-varying predictors in a single model. Among these 27 time-varying prediction model studies, we grouped model types into nine categories: time-dependent Cox regression, generalized estimating equations, random effects model, landmark model, joint model, neural network, K-nearest neighbor, support vector machine and tree-based algorithms. Where there was comparison of time-varying models to single measurement models, using time-varying predictors improved predictive accuracy.

CONCLUSIONS

Various methods have been used to develop time-varying prediction models in children, but there is a paucity of pediatric time-varying models in the literature. Incorporating time-varying covariates in pediatric prediction models may improve predictive accuracy. Future research in pediatric prediction model development should further investigate whether incorporation of time-varying covariates improves predictive accuracy.

Receiver Operating Characteristic (ROC) Curves: The Basics and Beyond

Diagnostic tests and clinical prediction rules are frequently used to help estimate the probability of a disease or outcome. How well a test or rule distinguishes between disease or no disease (discrimination) can be measured by plotting a receiver operating characteristic (ROC) curve and calculating the area under it (AUROC). In this paper, we review the features of ROC curves and interpretation of ROC curves and AUROC values. We highlight 5 underappreciated features of ROC curves: (1) the slope of the ROC curve over a test result interval is the likelihood ratio for that interval; (2) the optimal cutoff for calling a test positive depends not only on the shape of the ROC curve, but also on the pretest probability of disease and relative harms of false-positive and false-negative results; (3) the AUROC measures discrimination only, not the accuracy of the predicted probabilities; (4) the AUROC is not a good measure of discrimination if the slope of the ROC curve is not consistently decreasing; and (5) the AUROC can be increased by including a large number of people correctly identified as being at very low risk for the outcome of interest. We illustrate this last concept using 3 published studies.

Comparing penalization methods for linear models on large observational health data

OBJECTIVE

This study evaluates regularization variants in logistic regression (L1, L2, ElasticNet, Adaptive L1, Adaptive ElasticNet, Broken adaptive ridge [BAR], and Iterative hard thresholding [IHT]) for discrimination and calibration performance, focusing on both internal and external validation.

MATERIALS AND METHODS

We use data from 5 US claims and electronic health record databases and develop models for various outcomes in a major depressive disorder patient population. We externally validate all models in the other databases. We use a train-test split of 75%/25% and evaluate performance with discrimination and calibration. Statistical analysis for difference in performance uses Friedman's test and critical difference diagrams.

RESULTS

Of the 840 models we develop, L1 and ElasticNet emerge as superior in both internal and external discrimination, with a notable AUC difference. BAR and IHT show the best internal calibration, without a clear external calibration leader. ElasticNet typically has larger model sizes than L1. Methods like IHT and BAR, while slightly less discriminative, significantly reduce model complexity.

CONCLUSION

L1 and ElasticNet offer the best discriminative performance in logistic regression for healthcare predictions, maintaining robustness across validations. For simpler, more interpretable models, L0-based methods (IHT and BAR) are advantageous, providing greater parsimony and calibration with fewer features. This study aids in selecting suitable regularization techniques for healthcare prediction models, balancing performance, complexity, and interpretability.

Medical calculators derived synthetic cohorts: a novel method for generating synthetic patient data

This study shows that we can use synthetic cohorts created from medical risk calculators to gain insights into how risk estimations, clinical reasoning, data-driven subgrouping, and the confidence in risk calculator scores are connected. When prediction variables aren't evenly distributed in these synthetic cohorts, they can be used to group similar cases together, revealing new insights about how cohorts behave. We also found that the confidence in predictions made by these calculators can vary depending on patient characteristics. This suggests that it might be beneficial to include a "normalized confidence" score in future versions of these calculators for healthcare professionals. We plan to explore this idea further in our upcoming research.

Predictors contributing to the estimation of pulmonary tuberculosis among adults in a resource-limited setting: A systematic review of diagnostic predictions

Background

Although tuberculosis is highly prevalent in low- and middle-income countries, millions of cases remain undetected using current diagnostic methods. To address this problem, researchers have proposed prediction rules.

Objective

We analyzed existing prediction rules for the diagnosis of pulmonary tuberculosis and identified factors with a moderate to high strength of association with the disease.

Methods

We conducted a comprehensive search of relevant databases (MEDLINE/PubMed, Cochrane Library, Science Direct, Global Health for Reports, and Google Scholar) up to 14 November 2022. Studies that developed diagnostic algorithms for pulmonary tuberculosis in adults from low and middle-income countries were included. Two reviewers performed study screening, data extraction, and quality assessment. The study quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2. We performed a narrative synthesis.

Results

Of the 26 articles selected, only half included human immune deficiency virus-positive patients. In symptomatic human immune deficiency virus patients, radiographic findings and body mass index were strong predictors of pulmonary tuberculosis, with an odds ratio of >4. However, in human immune deficiency virus-negative individuals, the biomarkers showed a moderate association with the disease. In symptomatic human immune deficiency virus patients, a C-reactive protein level ⩾10 mg/L had a sensitivity and specificity of 93% and 40%, respectively, whereas a trial of antibiotics had a specificity of 86% and a sensitivity of 43%. In smear-negative patients, anti-tuberculosis treatment showed a sensitivity of 52% and a specificity of 63%.

Conclusions

The performance of predictors and diagnostic algorithms differs among patient subgroups, such as in human immune deficiency virus-positive patients, radiographic findings, and body mass index were strong predictors of pulmonary tuberculosis. However, in human immune deficiency virus-negative individuals, the biomarkers showed a moderate association with the disease. A few models have reached the World Health Organization's recommendation. Therefore, more work should be done to strengthen the predictive models for tuberculosis screening in the future, and they should be developed rigorously, considering the heterogeneity of the population in clinical work.

Prediction Models for Successful External Cephalic Version: An Updated Systematic Review

OBJECTIVE

 To review the decision aids currently available or being developed to predict a patient's odds that their external cephalic version (ECV) will be successful.

STUDY DESIGN

 We searched PubMed/MEDLINE, Cochrane Central, and ClinicalTrials.gov from 2015 to 2022. Articles from a pre-2015 systematic review were also included. We selected English-language articles describing or evaluating models (prediction rules) designed to predict an outcome of ECV for an individual patient. Acceptable model outcomes included cephalic presentation after the ECV attempt and whether the ECV ultimately resulted in a vaginal delivery. Two authors independently performed article selection following PRISMA 2020 guidelines. Since 2015, 380 unique records underwent title and abstract screening, and 49 reports underwent full-text review. Ultimately, 17 new articles and 8 from the prior review were included. Of the 25 articles, 22 proposed one to two models each for a total of 25 models, while the remaining 3 articles validated prior models without proposing new ones.

RESULTS

 Of the 17 new articles, 10 were low, 6 moderate, and 1 high risk of bias. Almost all articles were from Europe (11/25) or Asia (10/25); only one study in the last 20 years was from the United States. The models found had diverse presentations including score charts, decision trees (flowcharts), and equations. The majority (13/25) had no form of validation and only 5/25 reached external validation. Only the Newman-Peacock model (United States, 1993) was repeatedly externally validated (Pakistan, 2012 and Portugal, 2018). Most models (14/25) were published in the last 5 years. In general, newer models were designed more robustly, used larger sample sizes, and were more mathematically rigorous. Thus, although they await further validation, there is great potential for these models to be more predictive than the Newman-Peacock model.

CONCLUSION

 Only the Newman-Peacock model is ready for regular clinical use. Many newer models are promising but require further validation.

KEY POINTS

· 25 ECV prediction models have been published; 14 were in the last 5 years.. · The Newman-Peacock model is currently the only one with sufficient validation for clinical use.. · Many newer models appear to perform better but await further validation..

A Systematic Review of Predictor Composition, Outcomes, Risk of Bias, and Validation of COVID-19 Prognostic Scores

BACKGROUND

Numerous prognostic scores have been published to support risk stratification for patients with coronavirus disease 2019 (COVID-19).

METHODS

We performed a systematic review to identify the scores for confirmed or clinically assumed COVID-19 cases. An in-depth assessment and risk of bias (ROB) analysis (Prediction model Risk Of Bias ASsessment Tool [PROBAST]) was conducted for scores fulfilling predefined criteria ([I] area under the curve [AUC)] ≥ 0.75; [II] a separate validation cohort present; [III] training data from a multicenter setting [≥2 centers]; [IV] point-scale scoring system).

RESULTS

Out of 1522 studies extracted from MEDLINE/Web of Science (20/02/2023), we identified 242 scores for COVID-19 outcome prognosis (mortality 109, severity 116, hospitalization 14, long-term sequelae 3). Most scores were developed using retrospective (75.2%) or single-center (57.1%) cohorts. Predictor analysis revealed the primary use of laboratory data and sociodemographic information in mortality and severity scores. Forty-nine scores were included in the in-depth analysis. The results indicated heterogeneous quality and predictor selection, with only five scores featuring low ROB. Among those, based on the number and heterogeneity of validation studies, only the 4C Mortality Score can be recommended for clinical application so far.

CONCLUSIONS

The application and translation of most existing COVID scores appear unreliable. Guided development and predictor selection would have improved the generalizability of the scores and may enhance pandemic preparedness in the future.

Enhanced Risk Stratification for Children and Young Adults with B-Cell Acute Lymphoblastic Leukemia: A Children's Oncology Group Report

Current strategies to treat pediatric acute lymphoblastic leukemia rely on risk stratification algorithms using categorical data. We investigated whether using continuous variables assigned different weights would improve risk stratification. We developed and validated a multivariable Cox model for relapse-free survival (RFS) using information from 21199 patients. We constructed risk groups by identifying cutoffs of the COG Prognostic Index (PICOG) that maximized discrimination of the predictive model. Patients with higher PICOG have higher predicted relapse risk. The PICOG reliably discriminates patients with low vs. high relapse risk. For those with moderate relapse risk using current COG risk classification, the PICOG identifies subgroups with varying 5-year RFS. Among current COG standard-risk average patients, PICOG identifies low and intermediate risk groups with 96% and 90% RFS, respectively. Similarly, amongst current COG high-risk patients, PICOG identifies four groups ranging from 96% to 66% RFS, providing additional discrimination for future treatment stratification. When coupled with traditional algorithms, the novel PICOG can more accurately risk stratify patients, identifying groups with better outcomes who may benefit from less intensive therapy, and those who have high relapse risk needing innovative approaches for cure.

Prospective validity of a clinical prediction rule for response to non-surgical multidisciplinary management of knee osteoarthritis in tertiary care: a multisite prospective longitudinal study

OBJECTIVES

We tested a previously developed clinical prediction tool-a nomogram consisting of four patient measures (lower patient-expected benefit, lower patient-reported knee function, greater knee varus angle and severe medial knee radiological degeneration) that were related to poor response to non-surgical management of knee osteoarthritis. This study sought to prospectively evaluate the predictive validity of this nomogram to identify patients most likely to respond poorly to non-surgical management of knee osteoarthritis.

DESIGN

Multisite prospective longitudinal study.

SETTING

Advanced practice physiotherapist-led multidisciplinary service across six tertiary hospitals.

PARTICIPANTS

Participants with knee osteoarthritis deemed appropriate for trial of non-surgical management following an initial assessment from an advanced practice physiotherapist were eligible for inclusion.

INTERVENTIONS

Baseline clinical nomogram scores were collected before a trial of individualised non-surgical management commenced.

PRIMARY OUTCOME MEASURE

Clinical outcome (Global Rating of Change) was collected 6 months following commencement of non-surgical management and dichotomised to responder (a little better to a very great deal better) or poor responder (almost the same to a very great deal worse). Clinical nomogram accuracy was evaluated from receiver operating characteristics curve analysis and area under the curve, and sensitivity/specificity and positive/negative likelihood ratios were calculated.

RESULTS

A total of 242 participants enrolled. Follow-up scores were obtained from 210 participants (87% response rate). The clinical nomogram demonstrated an area under the curve of 0.70 (p<0.001), with greatest combined sensitivity 0.65 and specificity 0.64. The positive likelihood ratio was 1.81 (95% CI 1.32 to 2.36) and negative likelihood ratio 0.55 (95% CI 0.41 to 0.75).

CONCLUSIONS

The knee osteoarthritis clinical nomogram prediction tool may have capacity to identify patients at risk of poor response to non-surgical management. Further work is required to determine the implications for service delivery, feasibility and impact of implementing the nomogram in clinical practice.

Modifiable prognostic factors of high societal costs among people on sick leave due to musculoskeletal disorders: findings from an occupational cohort study

OBJECTIVES

The objective was to identify modifiable prognostic factors of high societal costs among people on sick leave due to musculoskeletal disorders, and to identify modifiable prognostic factors of high costs related to separately healthcare utilisation and productivity loss.

DESIGN

A prospective cohort study with a 1-year follow-up.

PARTICIPANTS AND SETTING

A total of 549 participants (aged 18-67 years) on sick leave (≥ 4 weeks) due to musculoskeletal disorders in Norway were included.

OUTCOME MEASURES AND METHOD

The primary outcome was societal costs aggregated for 1 year of follow-up and dichotomised as high or low, defined by the top 25th percentile. Secondary outcomes were high costs related to separately healthcare utilisation and productivity loss aggregated for 1 year of follow-up. Healthcare utilisation was collected from public records and included primary, secondary and tertiary healthcare use. Productivity loss was collected from public records and included absenteeism, work assessment allowance and disability pension. Nine modifiable prognostic factors were selected based on previous literature. Univariable and multivariable binary logistic regression analyses were performed to identify associations (crude and adjusted for selected covariates) between each modifiable prognostic factor and having high costs.

RESULTS

Adjusted for selected covariates, six modifiable prognostic factors associated with high societal costs were identified: pain severity, disability, self-perceived health, sleep quality, return to work expectation and long-lasting disorder expectation. Depressive symptoms, work satisfaction and health literacy showed no prognostic value. More or less similar results were observed when high costs were related to separately healthcare utilisation and productivity loss.

CONCLUSION

Factors identified in this study are potential target areas for interventions which could reduce high societal costs among people on sick leave due to musculoskeletal disorders. However, future research aimed at replicating these findings is warranted.

TRIAL REGISTRATION NUMBER

NCT04196634, 12 December 2019.

Prediction of 30-day, 90-day, and 1-year mortality after colorectal cancer surgery using a data-driven approach

PURPOSE

To develop prediction models for short-term mortality risk assessment following colorectal cancer surgery.

METHODS

Data was harmonized from four Danish observational health databases into the Observational Medical Outcomes Partnership Common Data Model. With a data-driven approach using the Least Absolute Shrinkage and Selection Operator logistic regression on preoperative data, we developed 30-day, 90-day, and 1-year mortality prediction models. We assessed discriminative performance using the area under the receiver operating characteristic and precision-recall curve and calibration using calibration slope, intercept, and calibration-in-the-large. We additionally assessed model performance in subgroups of curative, palliative, elective, and emergency surgery.

RESULTS

A total of 57,521 patients were included in the study population, 51.1% male and with a median age of 72 years. The model showed good discrimination with an area under the receiver operating characteristic curve of 0.88, 0.878, and 0.861 for 30-day, 90-day, and 1-year mortality, respectively, and a calibration-in-the-large of 1.01, 0.99, and 0.99. The overall incidence of mortality were 4.48% for 30-day mortality, 6.64% for 90-day mortality, and 12.8% for 1-year mortality, respectively. Subgroup analysis showed no improvement of discrimination or calibration when separating the cohort into cohorts of elective surgery, emergency surgery, curative surgery, and palliative surgery.

CONCLUSION

We were able to train prediction models for the risk of short-term mortality on a data set of four combined national health databases with good discrimination and calibration. We found that one cohort including all operated patients resulted in better performing models than cohorts based on several subgroups.

Diagnostic performance of clinical prediction rules to detect group A beta-haemolytic streptococci in people with acute pharyngitis: a systematic review

OBJECTIVE

To assess and compare the diagnostic performance of Clinical Prediction Rules (CPRs) developed to detect group A Beta-haemolytic streptococci in people with acute pharyngitis (or sore throat).

STUDY DESIGN

A systematic review.

METHODS

We searched PubMed, Embase and Web of Science (inception-September 2022) for studies deriving and/or validating CPRs comprised of ≥2 predictors from an individual's history or physical examination. Two authors independently screened articles, extracted data and assessed risk of bias in included studies. A meta-analysis was not possible due to heterogeneity. Instead we compared the performance of CPRs when they were validated in the same study population (head-to-head comparisons). We used a modified grading of recommendations, assessment, development, and evaluations (GRADE) approach to assess certainty of the evidence.

RESULTS

We included 63 studies, all judged at high risk of bias. Of 24 derived CPRs, 7 were externally validated (in 46 external validations). Five validation studies provided data for head-to-head comparison of four pairs of CPRs. Very low certainty evidence favoured the Centor CPR over the McIsaac (2 studies) and FeverPain CPRs (1 study) and found the Centor CPR was equivalent to the Walsh CPR (1 study). The AbuReesh and Steinhoff 2005 CPRs had a similar poor discriminative ability (1 study). Within and between study comparisons suggested the performance of the Centor CPR may be better in adults (>18 years).

CONCLUSION

Very low certainty evidence suggests a better performance of the Centor CPR. When deciding about antibiotic prescribing for pharyngitis patients, involving patients in a shared decision making discussion about the likely benefits and harms, including antibiotic resistance, is recommended. Further research of higher rigour, which compares CPRs across multiple settings, is needed.

Retrospective validation study of an artificial neural network-based preoperative decision-support tool for noninvasive lymph node staging (NILS) in women with primary breast cancer (ISRCTN14341750)

BACKGROUND

Surgical sentinel lymph node biopsy (SLNB) is routinely used to reliably stage axillary lymph nodes in early breast cancer (BC). However, SLNB may be associated with postoperative arm morbidities. For most patients with BC undergoing SLNB, the findings are benign, and the procedure is currently questioned. A decision-support tool for the prediction of benign sentinel lymph nodes based on preoperatively available data has been developed using artificial neural network modelling.

METHODS

This was a retrospective geographical and temporal validation study of the noninvasive lymph node staging (NILS) model, based on preoperatively available data from 586 women consecutively diagnosed with primary BC at two sites. Ten preoperative clinicopathological characteristics from each patient were entered into the web-based calculator, and the probability of benign lymph nodes was predicted. The performance of the NILS model was assessed in terms of discrimination with the area under the receiver operating characteristic curve (AUC) and calibration, that is, comparison of the observed and predicted event rates of benign axillary nodal status (N0) using calibration slope and intercept. The primary endpoint was axillary nodal status (discrimination, benign [N0] vs. metastatic axillary nodal status [N+]) determined by the NILS model compared to nodal status by definitive pathology.

RESULTS

The mean age of the women in the cohort was 65 years, and most of them (93%) had luminal cancers. Approximately three-fourths of the patients had no metastases in SLNB (N0 74% and 73%, respectively). The AUC for the predicted probabilities for the whole cohort was 0.6741 (95% confidence interval: 0.6255-0.7227). More than one in four patients (n = 151, 26%) were identified as candidates for SLNB omission when applying the predefined cut-off for lymph node-negative status from the development cohort. The NILS model showed the best calibration in patients with a predicted high probability of healthy axilla.

CONCLUSION

The performance of the NILS model was satisfactory. In approximately every fourth patient, SLNB could potentially be omitted. Considering the shift from postoperatively to preoperatively available predictors in this validation study, we have demonstrated the robustness of the NILS model. The clinical usability of the web interface will be evaluated before its clinical implementation.

TRIAL REGISTRATION

Registered in the ISRCTN registry with study ID ISRCTN14341750. Date of registration 23/11/2018.

Diagnostic scoring systems for tuberculous pleural effusion in patients with lymphocyte-predominant exudative pleural profile: A development study

Background

Diagnosing tuberculous pleural effusion (TPE) in patients presenting with Lymphocyte-Predominant Exudative pleural effusion (LPE) is challenging, due to the poor clinical utility of TB culture. Adenosine deaminase (ADA) has been recommended for diagnosis, but its high cost and limited availability hinder its clinical utility. We aim to develop diagnostic prediction tools for Thai patients with LPE in scenarios where pleural fluid ADA is available but yields negative results and in situations where pleural fluid ADA is not available.

Methods

Two diagnostic prediction tools were developed using retrospective data from patients with LPE at Surin Hospital. Model 1 is for ADA-negative results, and Model 2 is for situations where pleural fluid ADA testing is unavailable. The models were derived using multivariable logistic regression and presented as two clinical scoring systems: round-up and count scoring. The score cut-point that achieves a positive predictive value (PPV) comparable to the post-test probability of a pleural fluid ADA at a cut-point of 40 U/L was used as a threshold for initiating anti-TB treatment.

Results

A total of 359 patients were eligible for analysis, with 166 diagnosed with TPE and 193 diagnosed with non-TPE. Age <40 years, fever, pleural fluid protein ≥5 g/dL, male gender, pleural fluid color, and pleural fluid ADA ≥20 U/L were identified as final predictors. Both models demonstrated excellent discriminative ability (AuROC: 0.85 to 0.89). The round-up scoring demonstrated PPV above 90% at cut-off points of 4 and 4.5, while the count scoring achieved cut-off points of 3 and 4 for Model 1 (Lex-2P2A) and Model 2 (Lex-2P-MAC), respectively.

Conclusion

These diagnostic tools offer valuable assistance in differentiating between TPE and non-TPE in LPE patients with negative pleural fluid ADA (Lex-2P2A) and in settings where pleural fluid ADA testing is not available (Lex-2P-MAC). Implementing these diagnostic scores may have the potential to improve TPE diagnosis and facilitate prompt initiation of treatment.

Intelligent digital tools for screening of brain connectivity and dementia risk estimation in people affected by mild cognitive impairment: the AI-Mind clinical study protocol

More than 10 million Europeans show signs of mild cognitive impairment (MCI), a transitional stage between normal brain aging and dementia stage memory disorder. The path MCI takes can be divergent; while some maintain stability or even revert to cognitive norms, alarmingly, up to half of the cases progress to dementia within 5 years. Current diagnostic practice lacks the necessary screening tools to identify those at risk of progression. The European patient experience often involves a long journey from the initial signs of MCI to the eventual diagnosis of dementia. The trajectory is far from ideal. Here, we introduce the AI-Mind project, a pioneering initiative with an innovative approach to early risk assessment through the implementation of advanced artificial intelligence (AI) on multimodal data. The cutting-edge AI-based tools developed in the project aim not only to accelerate the diagnostic process but also to deliver highly accurate predictions regarding an individual's risk of developing dementia when prevention and intervention may still be possible. AI-Mind is a European Research and Innovation Action (RIA H2020-SC1-BHC-06-2020, No. 964220) financed between 2021 and 2026. First, the AI-Mind Connector identifies dysfunctional brain networks based on high-density magneto- and electroencephalography (M/EEG) recordings. Second, the AI-Mind Predictor predicts dementia risk using data from the Connector, enriched with computerized cognitive tests, genetic and protein biomarkers, as well as sociodemographic and clinical variables. AI-Mind is integrated within a network of major European initiatives, including The Virtual Brain, The Virtual Epileptic Patient, and EBRAINS AISBL service for sensitive data, HealthDataCloud, where big patient data are generated for advancing digital and virtual twin technology development. AI-Mind's innovation lies not only in its early prediction of dementia risk, but it also enables a virtual laboratory scenario for hypothesis-driven personalized intervention research. This article introduces the background of the AI-Mind project and its clinical study protocol, setting the stage for future scientific contributions.

Development and validation of a prognostic nomogram for patients with laryngeal cancer with synchronous or metachronous lung cancer

BACKGROUND

The objective of this study was to examine the independent prognostic factors of laryngeal cancer with synchronous or metachronous lung cancer (LCSMLC), and to generate and verify a clinical prediction model.

METHODS

In this study, laryngeal cancer alone and LCSMLC were defined using the Surveillance, Epidemiology, and End Results (SEER) database. Risk factors of patients with LCSMLC were analyzed through univariate and multivariate logistic regression analysis. Independent prognostic factors were selected by Cox regression analyses, on the basis of which a nomogram was constructed using R code. Kaplan-Meier survival analyses were applied to test the application of a risk stratification system. Finally, we conducted a comparison of the American Joint Committee on Cancer (AJCC) staging system of laryngeal cancer with the new model of nomogram and risk stratification. For further validation of the nomogram, data from patients at two Chinese independent institutions were also analyzed.

RESULTS

According to the eligibility criteria, 32 429 patients with laryngeal cancer alone and 641 patients with LCSMLC from the SEER database (the training cohort) and additional 61 patients from two Chinese independent institutions (the external validation cohort) were included for final analyses. Compared with patients with laryngeal cancer who did not have synchronous or metachronous lung cancer, age, sex, race, primary site of laryngeal cancer, grade, and stage were risk factors for LCSMLC, while marriage, surgery, radiation therapy, and chemotherapy are not their risk factors. Age, two cancers' interval, pathological type, stage, surgery, radiation, primary lung site, and primary throat site were independent prognostic predictors of LCSMLC. The risk stratification system of high-, medium-, and low-risk groups significantly distinguished the prognosis in different patients with LCSMLC, regardless of the training cohort or the validation cohort. Compared with the 6th AJCC TNM stage of laryngeal cancer, the new model of nomogram and risk stratification showed an improved net benefit.

CONCLUSIONS

Age, sex, race, primary site of laryngeal cancer, grade, and stage were risk factors for LCSMLC. An individualized clinical prognostic predictive model by nomogram was generated and validated, which showed superior prediction ability for LCSMLC.

Development and validation of a clinical prediction rule for acute appendicitis in children in primary care

BACKGROUND

Recognising acute appendicitis in children presenting with acute abdominal pain in primary care is challenging. General practitioners (GPs) may benefit from a clinical prediction rule.

OBJECTIVES

To develop and validate a clinical prediction rule for acute appendicitis in children presenting with acute abdominal pain in primary care.

METHODS

In a historical cohort study data was retrieved from GP electronic health records included in the Integrated Primary Care Information database. We assigned children aged 4-18 years presenting with acute abdominal pain (≤ 7 days) to development (2010-2012) and validation (2013-2016) cohorts, using acute appendicitis within six weeks as the outcome. Multiple logistic regression was used to develop a prediction model based on predictors with > 50% data availability derived from existing rules for secondary care. We performed internal and external temporal validation and derived a point score to stratify risk of appendicitis into three groups, i.e. low-risk, medium-risk and high-risk.

RESULTS

The development and validation cohorts included 2,041 and 3,650 children, of whom 95 (4.6%) and 195 (5.3%) had acute appendicitis. The model included male sex, pain duration (<24, 24-48, > 48 h), nausea/vomiting, elevated temperature (≥ 37.3 °C), abnormal bowel sounds, right lower quadrant tenderness, and peritoneal irritation. Internal and temporal validation showed good discrimination (C-statistics: 0.93 and 0.90, respectively) and excellent calibration. In the three groups, the risks of acute appendicitis were 0.5%, 7.5%, and 41%.

CONCLUSION

Combined with further testing in the medium-risk group, the prediction rule could improve clinical decision making and outcomes.

External validation and updating of prognostic prediction models for nonrecovery among older adults seeking primary care for back pain

ABSTRACT

Prognostic prediction models for 3 different definitions of nonrecovery were developed in the Back Complaints in the Elders study in the Netherlands. The models' performance was good (optimism-adjusted area under receiver operating characteristics [AUC] curve ≥0.77, R2 ≥0.3). This study aimed to assess the external validity of the 3 prognostic prediction models in the Norwegian Back Complaints in the Elders study. We conducted a prospective cohort study, including 452 patients aged ≥55 years, seeking primary care for a new episode of back pain. Nonrecovery was defined for 2 outcomes, combining 6- and 12-month follow-up data: Persistent back pain (≥3/10 on numeric rating scale) and persistent disability (≥4/24 on Roland-Morris Disability Questionnaire). We could not assess the third model (self-reported nonrecovery) because of substantial missing data (>50%). The models consisted of biopsychosocial prognostic factors. First, we assessed Nagelkerke R2 , discrimination (AUC) and calibration (calibration-in-the-large [CITL], slope, and calibration plot). Step 2 was to recalibrate the models based on CITL and slope. Step 3 was to reestimate the model coefficients and assess if this improved performance. The back pain model demonstrated acceptable discrimination (AUC 0.74, 95% confidence interval: 0.69-0.79), and R2 was 0.23. The disability model demonstrated excellent discrimination (AUC 0.81, 95% confidence interval: 0.76-0.85), and R2 was 0.35. Both models had poor calibration (CITL <0, slope <1). Recalibration yielded acceptable calibration for both models, according to the calibration plots. Step 3 did not improve performance substantially. The recalibrated models may need further external validation, and the models' clinical impact should be assessed.