Methods matter: clinical prediction models will benefit sports medicine practice, but only if they are properly developed and validated

Integration of longitudinal load-bearing tissue MRI radiomics and neural network to predict knee osteoarthritis incidence

Background

Load-bearing structural degradation is crucial in knee osteoarthritis (KOA) progression, yet limited prediction models use load-bearing tissue radiomics for radiographic (structural) KOA incident.

Purpose

We aim to develop and test a Load-Bearing Tissue plus Clinical variable Radiomic Model (LBTC-RM) to predict radiographic KOA incidents.

Study design

Risk prediction study.

Methods

The 700 knees without radiographic KOA at baseline were included from Osteoarthritis Initiative cohort. We selected 2164 knee MRIs during 4-year follow-up. LBTC-RM, which integrated MRI features of meniscus, femur, tibia, femorotibial cartilage, and clinical variables, was developed in total development cohort (n = 1082, 542 cases vs. 540 controls) using neural network algorithm. Final predictive model was tested in total test cohort (n = 1082, 534 cases vs. 548 controls), which integrated data from five visits: baseline (n = 353, 191 cases vs. 162 controls), 3 years prior KOA (n = 46, 19 cases vs. 27 controls), 2 years prior KOA (n = 143, 77 cases vs. 66 controls), 1 year prior KOA (n = 220, 105 cases vs. 115 controls), and at KOA incident (n = 320, 156 cases vs. 164 controls).

Results

In total test cohort, LBTC-RM predicted KOA incident with AUC (95 % CI) of 0.85 (0.82-0.87); with LBTC-RM aid, performance of resident physicians for KOA prediction were improved, with specificity, sensitivity, and accuracy increasing from 50 %, 60 %, and 55 %-72 %, 73 %, and 72 %, respectively. The LBTC-RM output indicated an increased KOA risk (OR: 20.6, 95 % CI: 13.8-30.6, p < .001). Radiomic scores of load-bearing tissue raised KOA risk (ORs: 1.02-1.9) from 4-year prior KOA whereas 3-dimensional feature score of medial meniscus decreased the OR (0.99) of KOA incident at KOA confirmed. The 2-dimensional feature score of medial meniscus increased the ORs (1.1-1.2) of KOA symptom score from 2-year prior KOA.

Conclusions

We provided radiomic features of load-bearing tissue to improved KOA risk level assessment and incident prediction. The model has potential clinical applicability in predicting KOA incidents early, enabling physicians to identify high-risk patients before significant radiographic evidence appears. This can facilitate timely interventions and personalized management strategies, improving patient outcomes.

The Translational Potential of this Article

This study presents a novel approach integrating longitudinal MRI-based radiomics and clinical variables to predict knee osteoarthritis (KOA) incidence using machine learning. By leveraging deep learning for auto-segmentation and machine learning for predictive modeling, this research provides a more interpretable and clinically applicable method for early KOA detection. The introduction of a Radiomics Score System enhances the potential for radiomics as a virtual image-based biopsy tool, facilitating non-invasive, personalized risk assessment for KOA patients. The findings support the translation of advanced imaging and AI-driven predictive models into clinical practice, aiding early diagnosis, personalized treatment planning, and risk stratification for KOA progression. This model has the potential to be integrated into routine musculoskeletal imaging workflows, optimizing early intervention strategies and resource allocation for high-risk populations. Future validation across diverse cohorts will further enhance its clinical utility and generalizability.

Prognostic factors associated with changes in knee pain outcomes, identified from initial primary care consultation data. A systematic literature review

BACKGROUND

Data collected during initial primary care consultations could be a source of baseline prognostic factors associated with changes in outcome measures for patients with knee pain.

OBJECTIVES

To identify, appraise and synthesize studies investigating prognostic factors associated with changes in outcome for people presenting with knee pain in primary care.

METHODS

EMBASE, CINAHL, AMED, MEDLINE and MedRxiv electronic databases were searched from inception to March 2021 and repeated in August 2022. Prospective cohort studies of adult participants with musculoskeletal knee pain assessing the association between putative prognostic factors and outcomes in primary care were included. The Quality in Prognostic Studies (QUIPS) tool and The Modified Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework, specific to prognostic reviews were used to appraise and synthesize the evidence respectively.

RESULTS

Eight studies were included. Eight knee pain outcomes were identified. Methodological and statistical heterogeneity resulted in qualitative analysis. All evidence was judged to be of low to very low quality. Bilateral knee pain (multivariable odds ratio (OR) range 2.60-2.74; 95%CI range 0.90-8.10, p value = 0.09) and a lower educational level (multivariable (OR) range 1.74-5.6; 95%CI range 1.16-16.20, p value = <0.001) were synonymously associated with persisting knee pain at 12-month follow up. A total of 37 univariable and 63 multivariable prognostic factors were statistically associated with outcomes (p ≤ 0.05) in single studies.

CONCLUSIONS

There was consensus from two independent studies that bilateral knee pain and lower educational level were associated with persistent knee pain. Many baseline factors were associated with outcome in individual studies but not consistently between studies. The current understanding, accuracy and reliability of the prognostic value of initial primary care consultation data for knee pain outcomes are limited. This review will provide an essential guide for candidate variable selection in future primary care prognostic confirmatory studies.Key messagesBilateral knee pain and lower educational level were associated with persistent knee pain.Many baseline factors were associated with outcome in individual studies but not consistently between studies.The current understanding, accuracy and reliability of the prognostic value of initial primary care consultation data for knee pain outcomes are limited.

The Trade Secret Taboo: Open Science Methods are Required to Improve Prediction Models in Sports Medicine and Performance

Clinical prediction models in sports medicine that utilize regression or machine learning techniques have become more widely published, used, and disseminated. However, these models are typically characterized by poor methodology and incomplete reporting, and an inadequate evaluation of performance, leading to unreliable predictions and weak clinical utility within their intended sport population. Before implementation in practice, models require a thorough evaluation. Strong replicable methods and transparency reporting allow practitioners and researchers to make independent judgments as to the model's validity, performance, clinical usefulness, and confidence it will do no harm. However, this is not reflected in the sports medicine literature. As shown in a recent systematic review of models for predicting sports injury models, most were typically characterized by poor methodology, incomplete reporting, and inadequate performance evaluation. Because of constraints imposed by data from individual teams, the development of accurate, reliable, and useful models is highly reliant on external validation. However, a barrier to collaboration is a desire to maintain a competitive advantage; a team's proprietary information is often perceived as high value, and so these 'trade secrets' are frequently guarded. These 'trade secrets' also apply to commercially available models, as developers are unwilling to share proprietary (and potentially profitable) development and validation information. In this Current Opinion, we: (1) argue that open science is essential for improving sport prediction models and (2) critically examine sport prediction models for open science practices.

What Factors Do Clinicians, Coaches, and Athletes Perceive Are Associated With Recovery From Low Back Pain in Elite Athletes? A Concept Mapping Study

OBJECTIVE: Identify factors that elite sport clinicians, coaches, and athletes perceive are associated with low back pain (LBP) recovery. DESIGN: Concept mapping methodology. METHOD: Participants brainstormed, sorted (thematically), and rated (5-point Likert scales: importance and feasibility) statements in response to the prompt, "What factors are associated with the recovery of an elite athlete from low back pain?" Data cleaning, analysis (multidimensional scaling, hierarchical cluster analysis, and descriptive statistics), and visual representation (cluster map and Go-Zone graph) were conducted following concept mapping guidelines. RESULTS: Participants (brainstorming, n = 56; sorting, n = 34; and rating, n = 33) comprised 75% clinicians, 15% coaches, and 10% athletes and represented 13 countries and 17 sports. Eighty-two unique and relevant statements were brainstormed. Sorting resulted in 6 LBP recovery-related themes: (1) coach and clinician relationships, (2) inter-disciplinary team factors, (3) athlete psychological factors, (4) athlete rehabilitation journey, (5) athlete non-modifiable risk factors, and (6) athlete physical factors. Participants rated important recovery factors as follows: athlete empowerment and psychology, coach-athlete and athlete-clinician relationships, care team communication, return-to-sport planning, and identifying red flags. CONCLUSION: Factors perceived as important to LBP recovery in elite athletes align with the biopsychosocial model of community LBP management. Clinicians should consider that an athlete's psychology, relationships, care team communication, and rehabilitation plan may be as important to their LBP recovery as the formulation of a diagnosis or the medications or exercises prescribed. J Orthop Sports Phys Ther 2023;53(10):1-16. Epub 10 August 2023. doi:10.2519/jospt.2023.11982.

Players', Head Coaches', And Medical Personnels' Knowledge, Understandings and Perceptions of Injuries and Injury Prevention in Elite-Level Women's Football in Ireland

BACKGROUND

To manage injuries effectively, players, head coaches, and medical personnel need to have excellent knowledge, attitudes, and behaviours in relation to the identification of risk factors for injuries, the implementation of injury prevention initiatives, as well as the implementation of effective injury management strategies. Understanding the injury context, whereby specific personal, environmental, and societal factors can influence the implementation of injury prevention initiatives and injury management strategies is critical to player welfare. To date, no qualitative research investigating the context of injuries, has been undertaken in elite-level women's football. The aim of our study was to explore the knowledge, attitudes, and behaviours of players, head coaches, and medical personnel in the Irish Women's National League (WNL) to injury prevention and injury management.

METHODS

We used qualitative research methods to explore the knowledge, attitudes, and behaviours of players, head coaches, and medical personnel in the Irish WNL to injury prevention and injury management. Semi-structured interviews were undertaken with 17 players, 8 medical personnel, and 7 head coaches in the Irish WNL. The data were analysed using thematic analysis. Our study is located within an interpretivist, constructivist research paradigm.

RESULTS

The participants had incomplete knowledge of common injuries in elite-level football, and many held beliefs about risk factors for injuries, such as menstrual cycle stage, which lacked evidence to support them. Jumping and landing exercises were commonly used to reduce the risk of injuries but evidence-based injury prevention exercises and programmes such as the Nordic hamstring curl, Copenhagen adduction exercise, and the FIFA 11+ were rarely mentioned. Overall, there was dissatisfaction amongst players with their medical care and strength and conditioning (S & C) support, with resultant inadequate communication between players, head coaches, and medical personnel.

CONCLUSION

Poor quality and availability of medical care and S & C support were considered to be a major obstacle in the effective implementation of injury risk reduction strategies and successful return-to-sport practices. More original research is required in elite-level women's football to explore injury risk factors, injury prevention initiatives, and contextual return-to-sport strategies, so that players, head coaches, and medical personnel can use evidence that is both up-to-date and specific to their environment.

Development of an Injury Burden Prediction Model in Professional Baseball Pitchers

Background

Baseball injuries are a significant problem and have increased in incidence over the last decade. Reporting injury incidence only gives context to rate but not in relation to severity or injury time loss.

Hypothesis/Purpose

The purpose of this study was to 1) incorporate both modifiable and non-modifiable factors to develop an arm injury burden prediction model in Minor League Baseball (MiLB) pitchers; and 2) understand how the model performs separately on elbow and shoulder injury burden.

Study Design

Prospective longitudinal study.

Methods

The study was conducted from 2013 to 2019 on MiLB pitchers. Pitchers were evaluated in spring training arm for shoulder range of motion and injuries were followed throughout the season. A model to predict arm injury burden was produced using zero inflated negative binomial regression. Internal validation was performed using ten-fold cross validation. Subgroup analyses were performed for elbow and shoulder separately. Model performance was assessed with root mean square error (RMSE), model fit (R2), and calibration with 95% confidence intervals (95% CI).

Results

Two-hundred, ninety-seven pitchers (94 injuries) were included with an injury incidence of 1.15 arm injuries per 1000 athletic exposures. Median days lost to an arm injury was 58 (11, 106). The final model demonstrated good prediction ability (RMSE: 11.9 days, R2: 0.80) and a calibration slope of 0.98 (95% CI: 0.92, 1.04). A separate elbow model demonstrated weaker predictive performance (RMSE: 21.3; R2: 0.42; calibration: 1.25 [1.16, 1.34]), as did a separate shoulder model (RMSE: 17.9; R2: 0.57; calibration: 1.01 [0.92, 1.10]).

Conclusions

The injury burden prediction model demonstrated excellent performance. Caution should be advised with predictions between one to 14 days lost to arm injury. Separate elbow and shoulder prediction models demonstrated decreased performance. The inclusion of both modifiable and non-modifiable factors into a comprehensive injury burden model provides the most accurate prediction of days lost in professional pitchers.

Level of Evidence

2.

A systematic review and meta-analysis demonstrating Klotho as an emerging exerkine

Klotho is an anti-aging protein with several therapeutic roles in the pathophysiology of different organs, such as the skeletal muscle and kidneys. Available evidence suggests that exercise increases Klotho levels, regardless of the condition or intervention, shedding some light on this anti-aging protein as an emergent and promising exerkine. Development of a systematic review and meta-analysis in order to verify the role of different exercise training protocols on the levels of circulating soluble Klotho (S-Klotho) protein. A systematic search of the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE through PubMed, EMBASE, CINAHL, CT.gov, and PEDro. Randomized and quasi-randomized controlled trials that investigated effects of exercise training on S-Klotho levels. We included 12 reports in the analysis, comprising 621 participants with age ranging from 30 to 65 years old. Klotho concentration increased significantly after chronic exercise training (minimum of 12 weeks) (Hedge' g [95%CI] 1.3 [0.69-1.90]; P < 0.0001). Moreover, exercise training increases S-Klotho values regardless of the health condition of the individual or the exercise intervention, with the exception of combined aerobic + resistance training. Furthermore, protocol duration and volume seem to influence S-Klotho concentration, since the effect of the meta-analysis changes when subgrouping these variables. Altogether, circulating S-Klotho protein is altered after chronic exercise training and it might be considered an exerkine. However, this effect may be influenced by different training configurations, including protocol duration, volume, and intensity.

Just How Confident Can We Be in Predicting Sports Injuries? A Systematic Review of the Methodological Conduct and Performance of Existing Musculoskeletal Injury Prediction Models in Sport

BACKGROUND

An increasing number of musculoskeletal injury prediction models are being developed and implemented in sports medicine. Prediction model quality needs to be evaluated so clinicians can be informed of their potential usefulness.

OBJECTIVE

To evaluate the methodological conduct and completeness of reporting of musculoskeletal injury prediction models in sport.

METHODS

A systematic review was performed from inception to June 2021. Studies were included if they: (1) predicted sport injury; (2) used regression, machine learning, or deep learning models; (3) were written in English; (4) were peer reviewed.

RESULTS

Thirty studies (204 models) were included; 60% of studies utilized only regression methods, 13% only machine learning, and 27% both regression and machine learning approaches. All studies developed a prediction model and no studies externally validated a prediction model. Two percent of models (7% of studies) were low risk of bias and 98% of models (93% of studies) were high or unclear risk of bias. Three studies (10%) performed an a priori sample size calculation; 14 (47%) performed internal validation. Nineteen studies (63%) reported discrimination and two (7%) reported calibration. Four studies (13%) reported model equations for statistical predictions and no machine learning studies reported code or hyperparameters.

CONCLUSION

Existing sport musculoskeletal injury prediction models were poorly developed and have a high risk of bias. No models could be recommended for use in practice. The majority of models were developed with small sample sizes, had inadequate assessment of model performance, and were poorly reported. To create clinically useful sports musculoskeletal injury prediction models, considerable improvements in methodology and reporting are urgently required.

Including Modifiable and Nonmodifiable Factors Improves Injury Risk Assessment in Professional Baseball Pitchers

OBJECTIVES

To (1) evaluate an injury risk model that included modifiable and nonmodifiable factors into an arm injury risk prediction model in Minor League Baseball (MiLB) pitchers and (2) compare model performance separately for predicting the incidence of elbow and shoulder injuries.

DESIGN

Prospective cohort.

METHODS

A 10-year MiLB injury risk study was conducted. Pitchers were evaluated during preseason, and pitches and arm injuries were documented prospectively. Nonmodifiable variables included arm injury history, professional experience, arm dominance, year, and humeral torsion. Modifiable variables included BMI, pitch count, total range of motion, and horizontal adduction. We compared modifiable, nonmodifiable, and combined model performance by R2, calibration (best = 1.00), and discrimination (area under the curve [AUC]; higher number is better). Sensitivity analysis included only arm injuries sustained in the first 90 days.

RESULTS

In this study, 407 MiLB pitchers (141 arm injuries) were included. Arm injury incidence was 0.27 injuries per 1000 pitches. The arm injury model (calibration 1.05 [0.81-1.30]; AUC: 0.74 [0.69-0.80]) had improved performance compared to only using modifiable predictors (calibration: 0.91 [0.68-1.14]; AUC: 0.67 [0.62-0.73]) and only shoulder range of motion (calibration: 0.52 [0.29, 0.75]; AUC: 0.52 [0.46, 58]). Elbow injury model demonstrated improved performance (calibration: 1.03 [0.76-1.33]; AUC: 0.76 [0.69-0.83]) compared to the shoulder injury model (calibration: 0.46 [0.22-0.69]; AUC: 0.62 [95% CI: 0.55, 0.69]). The sensitivity analysis demonstrated improved model performance compared to the arm injury model.

CONCLUSION

Arm injury risk is influenced by modifiable and nonmodifiable risk factors. The most accurate way to identify professional pitchers who are at risk for arm injury is to use a model that includes modifiable and nonmodifiable risk factors. J Orthop Sports Phys Ther 2022;52(9):630-640. Epub: 9 July 2022. doi:10.2519/jospt.2022.11072.

Forecasting football injuries by combining screening, monitoring and machine learning

Identifying players or circumstances associated with an increased risk of injury is fundamental for successful risk management in football. So far, time-constant and volatile risk factors are generally considered separately in either a screening (constant) or a monitoring (volatile) approach each resulting in a restricted set of explanatory variables. Consequently, improvements in predictive accuracy may be expected when screening and monitoring data are combined, especially when analysed with current machine learning (ML) techniques. This trial was designed as a prospective observational cohort study aiming to forecast non-contact time-loss injuries in male professional football (soccer). Injuries were registered according to the Fuller consensus. Gradient boosting with ROSE upsampling within a leave-one-out cross-validation was used for data analysis. The hierarchical data structure was considered throughout. Different splits of the original dataset were used to probe the robustness of results. Data of 88 players from 4 teams and 51 injuries could be analysed. The cross-validated performance of the gradient boosted model (ROC area under the curve 0.61) was promising and higher compared to models without integration of screening data. Importantly, holdout test set performance was similar (ROC area under the curve 0.62) indicating prospect of generalizability to new cases. However, the variation of predictive accuracy and feature importance with different splits of the original dataset reflects the relatively low number of events. It is concluded that ML-based injury forecasting based on the integration of screening and monitoring data is promising. However, external prospective verification and continued model development are required.

Prediction models for the risk of total knee replacement: development and validation using data from multicentre cohort studies

Background

Few prognostic prediction models for total knee replacement are available, and the role of radiographic findings in predicting its use remains unclear. We aimed to develop and validate predictive models for total knee replacement and to assess whether adding radiographic findings improves predictive performance.

Methods

We identified participants with recent knee pain (in the past 3 months) in the Multicenter Osteoarthritis Study (MOST) and the Osteoarthritis Initiative (OAI). The baseline visits of MOST were initiated in 2003 and of OAI were initiated in 2004. We developed two predictive models for the risk of total knee replacement within 60 months of follow-up by fitting Cox proportional hazard models among participants in MOST. The first model included sociodemographic and anthropometric factors, medical history, and clinical measures (referred to as the clinical model). The second model added radiographic findings into the predictive model (the radiographic model). We evaluated each model's discrimination and calibration performance and assessed the incremental value of radiographic findings using both category-free net reclassification improvement (NRI) and integrated discrimination improvement (IDI). We tuned the models and externally validated them among participants in OAI.

Findings

We included 2658 participants from MOST (mean age 62·4 years [SD 8·1], 1646 [61·9%] women) in the training dataset and 4060 participants from OAI (mean age 60·9 years [9·1], 2379 [58·6%] women) in the validation dataset. 290 (10·9%) participants in the training dataset and 174 (4·3%) in the validation dataset had total knee replacement. The retained predictive variables included in the clinical model were age, sex, race, history of knee arthroscopy, frequent knee pain, current use of analgesics, current use of glucosamine, body-mass index, and Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) pain score, and the most predictive factors were age, race, and WOMAC pain score. The retained predictive variables in the radiographic model were age, sex, race, frequent knee pain, current use of analgesics, WOMAC pain score, and Kellgren-Lawrence grade, and the most predictive factors were Kellgren-Lawrence grade, race, and age. The C-statistic was 0·79 (95% CI 0·76-0·81) for the clinical model and 0·87 (0·85-0·99) for the radiographic model in the training dataset. The calibration slope was 0·95 (95% CI 0·86-1·05) and 0·96 (0·87-1·04), respectively. Adding radiograph findings significantly improved predictive performance with an NRI of 0·43 (95% CI 0·38-0·50) and IDI of 0·14 (95% CI: 0·10-0·18). Both models, with tuned coefficients, showed a good predictive performance among participants in the validation dataset.

Interpretation

The risk of total knee replacement can be predicted based on common risk factors with good discrimination and calibration. Additionally, adding radiographic findings of knee osteoarthritis into the model substantially improves its predictive performance.

Funding

National Natural Science Foundation of China, National Key Research and Development Program, and Beijing Municipal Science & Technology Commission.