Variations in risk assessment models may contribute to the existing gap between venous thromboembolism prophylaxis guidelines and adherence

Evaluation of venous thromboembolism risk assessment models for hospital inpatients: the VTEAM evidence synthesis

Background

Pharmacological prophylaxis during hospital admission can reduce the risk of acquired blood clots (venous thromboembolism) but may cause complications, such as bleeding. Using a risk assessment model to predict the risk of blood clots could facilitate selection of patients for prophylaxis and optimise the balance of benefits, risks and costs.

Objectives

We aimed to identify validated risk assessment models and estimate their prognostic accuracy, evaluate the cost-effectiveness of different strategies for selecting hospitalised patients for prophylaxis, assess the feasibility of using efficient research methods and estimate key parameters for future research.

Design

We undertook a systematic review, decision-analytic modelling and observational cohort study conducted in accordance with Enhancing the QUAlity and Transparency Of health Research (EQUATOR) guidelines.

Setting

NHS hospitals, with primary data collection at four sites.

Participants

Medical and surgical hospital inpatients, excluding paediatric, critical care and pregnancy-related admissions.

Interventions

Prophylaxis for all patients, none and according to selected risk assessment models.

Main outcome measures

Model accuracy for predicting blood clots, lifetime costs and quality-adjusted life-years associated with alternative strategies, accuracy of efficient methods for identifying key outcomes and proportion of inpatients recommended prophylaxis using different models.

Results

We identified 24 validated risk assessment models, but low-quality heterogeneous data suggested weak accuracy for prediction of blood clots and generally high risk of bias in all studies. Decision-analytic modelling showed that pharmacological prophylaxis for all eligible is generally more cost-effective than model-based strategies for both medical and surgical inpatients, when valuing a quality-adjusted life-year at £20,000. The findings were more sensitive to uncertainties in the surgical population; strategies using risk assessment models were more cost-effective if the model was assumed to have a very high sensitivity, or the long-term risks of post-thrombotic complications were lower. Efficient methods using routine data did not accurately identify blood clots or bleeding events and several pre-specified feasibility criteria were not met. Theoretical prophylaxis rates across an inpatient cohort based on existing risk assessment models ranged from 13% to 91%.

Limitations

Existing studies may underestimate the accuracy of risk assessment models, leading to underestimation of their cost-effectiveness. The cost-effectiveness findings do not apply to patients with an increased risk of bleeding. Mechanical thromboprophylaxis options were excluded from the modelling. Primary data collection was predominately retrospective, risking case ascertainment bias.

Conclusions

Thromboprophylaxis for all patients appears to be generally more cost-effective than using a risk assessment model, in hospitalised patients at low risk of bleeding. To be cost-effective, any risk assessment model would need to be highly sensitive. Current evidence on risk assessment models is at high risk of bias and our findings should be interpreted in this context. We were unable to demonstrate the feasibility of using efficient methods to accurately detect relevant outcomes for future research.

Future work

Further research should evaluate routine prophylaxis strategies for all eligible hospitalised patients. Models that could accurately identify individuals at very low risk of blood clots (who could discontinue prophylaxis) warrant further evaluation.

Study registration

This study is registered as PROSPERO CRD42020165778 and Researchregistry5216.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: NIHR127454) and will be published in full in Health Technology Assessment; Vol. 28, No. 20. See the NIHR Funding and Awards website for further award information.

Effectiveness and cost effectiveness of pharmacological thromboprophylaxis for medical inpatients: decision analysis modelling study

Objective

To determine the balance of costs, risks, and benefits for different thromboprophylaxis strategies for medical patients during hospital admission.

Design

Decision analysis modelling study.

Setting

NHS hospitals in England.

Population

Eligible adult medical inpatients, excluding patients in critical care and pregnant women.

Interventions

Pharmacological thromboprophylaxis (low molecular weight heparin) for all medical inpatients, thromboprophylaxis for none, and thromboprophylaxis given to higher risk inpatients according to risk assessment models (Padua, Caprini, IMPROVE, Intermountain, Kucher, Geneva, and Rothberg) previously validated in medical cohorts.

Main outcome measures

Lifetime costs and quality adjusted life years (QALYs). Costs were assessed from the perspective of the NHS and Personal Social Services in England. Other outcomes assessed were incidence and treatment of venous thromboembolism, major bleeds including intracranial haemorrhage, chronic thromboembolic complications, and overall survival.

Results

Offering thromboprophylaxis to all medical inpatients had a high probability (>99%) of being the most cost effective strategy (at a threshold of £20 000 (€23 440; $25 270) per QALY) in the probabilistic sensitivity analysis, when applying performance data from the Padua risk assessment model, which was typical of that observed across several risk assessment models in a medical inpatient cohort. Thromboprophylaxis for all medical inpatients was estimated to result in 0.0552 additional QALYs (95% credible interval 0.0209 to 0.1111) while generating cost savings of £28.44 (-£47 to £105) compared with thromboprophylaxis for none. No other risk assessment model was more cost effective than thromboprophylaxis for all medical inpatients when assessed in deterministic analysis. Risk based thromboprophylaxis was found to have a high (76.6%) probability of being the most cost effective strategy only when assuming a risk assessment model with very high sensitivity is available (sensitivity 99.9% and specificity 23.7% v base case sensitivity 49.3% and specificity 73.0%).

Conclusions

Offering pharmacological thromboprophylaxis to all eligible medical inpatients appears to be the most cost effective strategy. To be cost effective, any risk assessment model would need to have a very high sensitivity resulting in widespread thromboprophylaxis in all patients except those at the very lowest risk, who could potentially avoid prophylactic anticoagulation during their hospital stay.

Comparative study of two models predicting the risk of deep vein thrombosis progression in spinal trauma patients after operation

OBJECTIVE

Patients with preoperative deep vein thrombosis (DVT) exhibit a notable incidence of postoperative deep vein thrombosis progression (DVTp), which bears a potential for silent, severe consequences. Consequently, the development of a predictive model for the risk of postoperative DVTp among spinal trauma patients is important.

METHODS

Data of 161 spinal traumatic patients with preoperative DVT, who underwent spine surgery after admission, were collected from our hospital between January 2016 and December 2022. The least absolute shrinkage and selection operator (LASSO) combined with multivariable logistic regression analysis was applied to select variables for the development of the predictive logistic regression models. One logistic regression model was formulated simply with the Caprini risk score (Model A), while the other model incorporated not only the previously screened variables but also the age variable (Model B). The model's capability was evaluated using sensitivity, specificity, positive predictive value, negative predictive value, accuracy, F1 score, and receiver operating characteristic (ROC) curve. Nomograms simplified and visually presented Model B for the clinicians and patients to understand the predictive model. The decision curve was used to analyze the clinical value of Model B.

RESULTS

A total of 161 DVT patients were enrolled in this study. Postoperative DVTp occurred in 48 spinal trauma patients, accounting for 29.81% of the total patient enrolled. Model A inadequately predicted postoperative DVTp in spinal trauma patients, with ROC AUC values of 0.595 for the training dataset and 0.593 for the test dataset. Through the application of LASSO regression and multivariable logistic regression, a screening process was conducted for seven risk factors: D-dimer, blood platelet, hyperlipidemia, blood group, preoperative anticoagulant, spinal cord injury, lower extremity varicosities. Model B demonstrated superior and consistent predictive performance, with ROC AUC values of 0.809 for the training dataset and 0.773 for the test dataset. According to the calibration curves and decision curve analysis, Model B could accurately predict the probability of postoperative DVTp after spine surgery. The nomograms enhanced the interpretability of Model B in charts and graphs.

CONCLUSIONS

In summary, we established a logistic regression model for the accurate predicting of postoperative deep vein thrombosis progression in spinal trauma patients, utilizing D-dimer, blood platelet, hyperlipidemia, blood group, preoperative anticoagulant, spinal cord injury, lower extremity varicosities, and age as predictive factors. The proposed model outperformed a logistic regression model based simply on CRS. The proposed model has the potential to aid frontline clinicians and patients in identifying and intervening in postoperative DVTp among traumatic patients undergoing spinal surgery.

Generalizability of COBRA: A Parsimonious Perioperative Venous Thromboembolism Risk Assessment Model

INTRODUCTION

Standardized use of venous thromboembolism (VTE) risk assessment models (RAMs) in surgical patients has been limited, in part due to the cumbersome workflow addition required to use available models. The COBRA score-capturing cancer diagnosis, (old) age, body mass index, race, and American Society of Anesthesiologists Physical Status score-has been reported as a potentially automatable VTE RAM that circumvents the cumbersome workflow addition that most RAMs represent. We aimed to test the ability of the COBRA model to effectively risk-stratify patients across various populations.

METHODS

Patients were included from the 2014-2019 American College of Surgeons National Surgical Quality Improvement Program (NSQIP) Participant Use Data File for two hospitals, representing colorectal, endocrine, breast, transplant, plastic, and general surgery services. COBRA score was calculated for each patient using preoperative characteristics. We calculated negative predictive value (NPV) for VTE outcomes and compared the COBRA score to NSQIP's expected VTE rate for all patients, between the two hospitals, and between subspecialty service lines.

RESULTS

Of the 10,711 patients included, those with COBRA <4 (31%) had projected median VTE rate of 0.21% (interquartile range, 0.09-0.68%; mean, 0.54%). Patients with higher scores (69%) had median rate of 0.88% (0.26-2.07%; 1.46%); relative rate 2.7. The median projected VTE rates for patients identified as low risk were 0.21% and 0.16% and as high risk were 0.87% and 0.89% at hospitals one and 2, respectively. The median projected VTE rates for patients identified as low risk were 0.17%, 0.61%, and 0.08% and as high risk were 0.52%, 1.43%, and 0.18% among general, colorectal, and endocrine surgery patients, respectively. COBRA had NPV of 0.995 and sensitivity of 0.871 as compared to NPV 0.997 and sensitivity 0.857 of the NSQIP model.

CONCLUSIONS

The COBRA score is concordant with the traditional gold standard NSQIP VTE RAM and demonstrates interhospital and service-specific generalizability, although performance was limited in especially low-risk patients. The model adequately risk-stratifies surgical patients preoperatively, potentially providing clinical decision support for perioperative workflows.

An abbreviated Caprini model for VTE risk assessment in trauma

The Caprini risk assessment model (RAM) is widely used to assess risk of venous thromboembolism (VTE). However, it is cumbersome with 31 variables and poses challenges with inter-rater reliability. This study aimed to determine if an abbreviated model could perform similarly in VTE risk assessment. We performed a retrospective review of trauma patients ≥ 18 years old and admitted for over 24 h at a Level I trauma center from January 1, 2018, to December 31, 2018. Demographic and clinical data were analyzed to generate Caprini scores. Using a p-value cutoff of < 0.05, the individual components of the original Caprini RAM most highly associated with VTE were identified and used to calculate an abbreviated Caprini score. Logistic regression assessed odds of inpatient VTE with the original or abbreviated Caprini RAMs. Receiver operating characteristic curves and c-statistics were generated to assess discriminatory ability. The study sample included 1279 patients. Ten risk factors were included in the abbreviated model (recent major surgery, length of surgery > 2 h, transfusion, restricted mobility > 72 h, central venous catheter, current major surgery, age, history of VTE, hip or leg fracture, and serious trauma). Compared to the original, the abbreviated model had a similar odds ratio (1.17 vs 1.07, both p-values < 0.001), c-statistic (0.747 vs 0.753), sensitivity (0.73 vs 0.76) and specificity (0.62 vs 0.61). An abbreviated Caprini RAM performs similarly to the original, may streamline workflow and allow for automation in electronic health records, potentially enhancing its use in resource limited settings.

Development of a Novel Preoperative Venous Thromboembolism Risk Assessment Model

INTRODUCTION

Standardization of preoperative venous thromboembolism (VTE) risk assessment remains challenging due to variation in risk assessment models (RAMs) and the cumbersome workflow addition that most RAMs represent. We aimed to develop a parsimonious RAM that is automatable and actionable within the preoperative workflow.

METHODS

We performed a case-controlled review of all 18 VTE cases reported over a 12-month period and 171 matched controls included in an institutional National Surgical Quality Improvement Project (NSQIP) data set. We examined the predictive value of the Caprini, Padua, and NSQIP RAMs. We identified the 5 most impactful risk factors in VTE development by contribution to the known RAMs. We compared the predictive ability of cancer, age, body mass index, black race, and American Society of Anesthesiologists Physical Status (ASA-PS) score, to the Caprini, Padua, and NSQIP RAMs for VTE outcomes. Finally, we evaluated concordance between each of the models.

RESULTS

The Caprini Score was found to be 88.9% sensitive and 32.7% specific using a threshold of 5. The Padua score was found to be 61.1% sensitive and 47.4% specific using a threshold of 4. The novel 5-factor RAM was found to be 94.4% sensitive and 38.0% specific using a threshold of 4. The Caprini and Padua models were discordant in 26% of patients.

DISCUSSION

Cumbersome manual data entry contributes to the ongoing challenge of standardized VTE risk assessment and prophylaxis. Universally documented information and patient demographics can be utilized to create clinical decision support tools that can improve the efficiency of perioperative workflow and improve the quality of care.