Prognostic prediction models and clinical tools based on consensus to support patient prioritization for clinical pharmacy services in hospitals: A scoping review

Development and external validation of a prognostic model for time to readmission or death in multimorbid patients

OBJECTIVE

To develop and externally validate a prognostic model built on important factors predisposing multimorbid patients to all-cause readmission and/or death. In addition to identify patients who may benefit most from a comprehensive clinical pharmacist intervention.

METHODS

A multivariable prognostic model was developed based on data from a randomised controlled trial investigating the effect of pharmacist-led medicines management on readmission rate in multimorbid, hospitalised patients. The derivation set comprised 386 patients randomised in a 1:1 manner to the intervention group, i.e. with a pharmacist included in their multidisciplinary treatment team, or the control group receiving standard care at the ward. External validation of the model was performed using data from an independent cohort, in which 100 patients were randomised to the same intervention, or standard care. The setting was an internal medicines ward at a university hospital in Norway.

RESULTS

The number of patients who were readmitted or had died within 18 months after discharge was 297 (76.9 %) in the derivation set, i.e. the randomized controlled trial, and 69 (71.1 %) in the validation set, i.e. the independent cohort. Charlson comorbidity index (CCI; low, moderate or high), previous hospital admissions within the previous six months and heart failure were the strongest prognostic factors and were included in the final model. The efficacy of the pharmaceutical intervention did not prove significant in the model. A prognostic index (PI) was constructed to estimate the hazard of readmission or death (low, intermediate or high-risk groups). Overall, the external validation replicated the result. We were unable to identify a subgroup of the multimorbid patients with better efficacy of the intervention.

CONCLUSIONS

A prognostic model including CCI, previous admissions and heart failure can be used to obtain valid estimates of risk of readmission and death in patients with multimorbidity.

Development, content validation and standardization of an adult patient prioritization tool for hospital clinical pharmacy services

BACKGROUND

Patient prioritization is a effective strategy to identify high risk patients for targeted Clinical Pharmacy Service (CPS) in hospital pharmacy. However, there is a paucity of patient prioritization tool to use in clinical practice.

OBJECTIVES

Describe the development, content validation and standardization of an adult patient prioritization tool for hospital CPS named, PrioFarClinH.

METHODS

The tool was developed using a stepwise design multi: Scoping Review to identify prioritization criteria/sub-criteria; Delphi technique to obtain consensus under the identified criteria/sub-criteria; Survey with pharmacists evaluating applicability of the criteria/sub-criteria obtained from Delphi; Definition of criteria/sub-criteria to be included in PrioFarClinH attribution of scores. Content validation was performed by a panel of experts evaluating relevance, feasibility, clarity and adequacy of the score. Content Validity Index (CVI) was calculated. Standardization occurred through a retrospective observational study carried out at 24 and 72 h and median of the patient's hospital stay. An intragroup norm was performed, determining percentile ranks of the instrument's total scores. Patients with a P90 score were classified with a high level of prioritization for CPS.

RESULTS

PrioFarClinH is divided into three sections, with prioritization criteria for health issues; therapeutic classes; laboratory parameters. It comprises 51 criteria with specific scores with simple total calculation. None of the criteria presented CVI <0.78, maintaining the items from the initial version of PrioFarClinH. The scores were adjusted per suggestions from the panel of judges. Data were collected from 393 patients. The P90 percentile in the three hospitalization stages (24 h, 72 h, and median) was found, respectively, in the following scores: 18.0, 20.0, and 22.6.

CONCLUSIONS

PrioFarClinH is a comprehensive tool to target and to prioritize adults patients most likely to benefit from CPS. Evidence for adequate content validity was provided. However, further validation of this tool is necessary to establish tool performance.

The adverse inpatient medication event and frailty (AIME-frail) risk prediction model

BACKGROUND

Medication harm affects between 5 and 15% of hospitalised patients, with approximately half of the harm events considered preventable through timely intervention. The Adverse Inpatient Medication Event (AIME) risk prediction model was previously developed to guide a systematic approach to patient prioritisation for targeted clinician review, but frailty was not tested as a candidate predictor variable.

AIM

To evaluate the predictive performance of an updated AIME model, incorporating a measure of frailty, when applied to a new multisite cohort of hospitalised adult inpatients.

METHODS

A retrospective cohort study was conducted at two tertiary Australian hospitals on patients discharged between 1st January and April 31, 2020. Data were extracted from electronic medical records (EMRs) and clinical coding databases. Medication harm was identified using ICD-10 Y-codes and confirmed by senior pharmacist review of medical records. The Hospital Frailty Risk Score (HFRS) was calculated for each patient. Logistic regression analysis was used to construct a modified AIME model. Candidate variables of the original AIME model, together with new variables including HFRS were tested. Performance of the final model was reported using area under the curve (AUC) and decision curve analysis (DCA).

RESULTS

A total of 4089 patient admissions were included, with a mean age ± standard deviation (SD) of 64 years (±19 years), 2050 patients (50%) were males, and mean HFRS was 6.2 (±5.9). 184 patients (4.5%) experienced one or more medication harm events during hospitalisation. The new AIME-Frail risk model incorporated 5 of the original variables: length of stay (LOS), anti-psychotics, antiarrhythmics, immunosuppressants, and INR greater than 3, as well as 5 new variables: HFRS, anticoagulants, antibiotics, insulin, and opioid use. The AUC was 0.79 (95% CI: 0.76-0.83) which was superior to the original model (AUC = 0.70, 95% CI: 0.65-0.74) with a sensitivity of 69%, specificity of 81%, positive predictive value of 0.14 (95% CI: 0.10-0.17) and negative predictive value of 0.98 (95% CI: 0.97-0.99). The DCA identified the model as having potential clinical utility between the probability thresholds of 0.05-0.4.

CONCLUSION

The inclusion of a frailty measure improved the predictive performance of the AIME model. Screening inpatients using the AIME-Frail tool could identify more patients at high-risk of medication harm who warrant timely clinician review.

The pursuit of accurate predictive models of the bioactivity of small molecules

Property prediction is a key interest in chemistry. For several decades there has been a continued and incremental development of mathematical models to predict properties. As more data is generated and accumulated, there seems to be more areas of opportunity to develop models with increased accuracy. The same is true if one considers the large developments in machine and deep learning models. However, along with the same areas of opportunity and development, issues and challenges remain and, with more data, new challenges emerge such as the quality and quantity and reliability of the data, and model reproducibility. Herein, we discuss the status of the accuracy of predictive models and present the authors' perspective of the direction of the field, emphasizing on good practices. We focus on predictive models of bioactive properties of small molecules relevant for drug discovery, agrochemical, food chemistry, natural product research, and related fields.

Increasing the number of pharmacist-led medication order reviews using the available workforce: a retrospective study

BACKGROUND

Due to staffing constraints, several hospitals have defined targeting strategies for pharmacist-led medication order review, leaving non-targeted patients exposed to potential harmful drug-related problems (DRPs). Using targeting criteria to stratify medication order review level (level 1 (L1): orders, basic patient characteristics; level 2 (L2) or comprehensive medication order review: orders, patient characteristics, medical records, laboratory results) could make it possible to save time and increase the overall number of medication order reviews. This study aims to define targeting criteria to stratify medication order review level and estimate the time saved for the performance of additional medication order reviews.

METHOD

This retrospective single-centre study included all medication order reviews performed in 2020; DRPs were collected to assess the medication order review level required to detect them. Logistic regressions were performed to define patient characteristics associated with L2. These targeting criteria were applied to the cohort to estimate the time saved and the number of additional medication order reviews which could have been performed using this approach.

RESULTS

2478 DRPs were reported; 54.2% (1343/2748) could have been detected using an L1 medication order review (representing 48.2% of the patients (829/1721)). L2 medication order reviews were significantly associated with age ≥65 years, male, and renal clearance <60 mL/min (OR≥75yo=1.79; OR65-74yo=1.74; ORfemale=0.74; OR30-59mL/min=1.67; OR<30mL/min=2.62; p<0.05). Sex being a confounding factor, only age and renal clearance were used as targeting criteria. The time saved was estimated at 274 hours per year, leading to an additional 1720 medication order reviews (54 hospital beds).

CONCLUSION

The proposed approach would maintain a satisfying level of safety and quality for patients, by performing an L2 medication order review for targeted patients based on age and renal clearance, while improving medication order review coverage with an L1 medication order review for non-targeted patients, using the available workforce.

Systematic Review of Risk Factors Assessed in Predictive Scoring Tools for Drug-Related Problems in Inpatients

Drug-related problems (DRP, defined as adverse drug events/reactions and medication errors) are a common threat for patient safety. With the aim to aid improved allocation of specialist resources and to improve detection and prevention of DRP, numerous predictive scoring tools have been proposed. The external validation and evidence for the transferability of these tools still faces limitations. However, the proposed scoring tools include partly overlapping sets of similar factors, which may allow a new approach to estimate the external usability and validity of individual risk factors. Therefore, we conducted this systematic review and analysis. We identified 14 key studies that assessed 844 candidate risk factors for inclusion into predictive scoring tools. After consolidation to account for overlapping terminology and variable definitions, we assessed each risk factor in the number of studies it was assessed, and, if it was found to be a significant predictor of DRP, whether it was included in a final scoring tool. The latter included intake of ≥ 8 drugs, drugs of the Anatomical Therapeutic Chemical (ATC) class N, ≥1 comorbidity, an estimated glomerular filtration rate (eGFR) <30 mL/min and age ≥60 years. The methodological approach and the individual risk factors presented in this review may provide a new starting point for improved risk assessment.

Consensus on the criteria for patient prioritization in hospital clinical pharmacy services: a Delphi study

BACKGROUND

Hospital pharmacists cannot provide extensive clinical pharmacy service to every inpatient because the demand for these services can exceed pharmacists' available work time. A way to solve this issue is hospital pharmacists to prioritize their clinical tasks. Tool prioritization could determine which patients would benefit from clinical pharmacists' input.

AIM

Establishing consensus on which criteria are relevant for prioritizing patients for clinical pharmacy services.

METHOD

The Delphi method was performed with criteria identified from a previously published Scoping Review. The panel of experts included hospital pharmacists, who evaluated the clinical significance of criteria in a three-round Delphi panel from July to December 2020. They determined the need for sub-criteria and evaluated their clinical significance. The experts also analyzed the criteria/sub-criteria as to their applicability in clinical practice. Consensus criteria were defined as 70% or more participants scoring the criteria as critical and 15% or fewer scoring the criteria as unimportant.

RESULTS

A total of 19 criteria and 230 sub-criteria were included for evaluation by panel experts based on scoping review. Twenty-nine, 22, and 17 experts participated per round, respectively. After completing the three rounds, experts suggested the inclusion of one criterion, the exclusion of one criterion, and the inclusion of 29 sub-criteria. The final list consisted of 18 criteria and 177 sub-criteria, divided into 28 groups.

CONCLUSION

The result was comprehensive and coherent, potentially contributing to developing an instrument for prioritizing hospitalized patients for clinical pharmacy services.

Fully independent validation and updating of a clinical pharmacy prioritizing risk score in an infectious disease hospital ward

AIM

To validate and update the risk score originally developed at Hospital de Clínicas de Porto Alegre, verifying its performance in an infectious disease population.

METHODS

This is an observational study with consecutive selection of admission in a ward of participants with infectious diseases. Predictors were age, number of medications, intravenous drugs, potentially dangerous drugs, renal dysfunction, liver dysfunction, use of nasogastric tube, nasogastric tube, gastrostomy feeding, jejunostomy feeding, oral enteral tube, total parenteral nutrition, cardiac or pulmonary dysfunction and immunosuppression. Outcome was defined as preventable prescription incidents by a clinical pharmacist. A GEE model was fit to make predictions each week.

RESULTS

219 patients participated in the study. 79.25% of the participants had prescription incidents in the first week of admission. Predictors of the updated model were number of drugs prescribed, number of intravenous drugs, use of tubes, truncated age at 36 years and week of hospitalization. The performance of the original model was poor. The updated model's descrimination and calibration were moderate (overall AUC 0.74). A calculator to apply the model is available at https://pedrobrasil.shinyapps.io/INDWELL/.

CONCLUSION

The updated risk score enabled the user to make predictions at admission and throughout the weeks, allowing for a prioritized weekly update for clinical pharmacy intervention. The updated model has a moderate and satisfactory performance for infectious disease patients.

Clinical usefulness of prediction tools to identify adult hospitalized patients at risk of drug-related problems: A systematic review of clinical prediction models and risk assessment tools

AIMS

This study aimed to review systematically all available prediction tools identifying adult hospitalized patients at risk of drug-related problems, and to synthesize the evidence on performance and clinical usefulness.

METHODS

PubMed, Scopus, Web of Science, Embase, and CINAHL databases were searched for relevant studies. Titles, abstracts and full-text studies were sequentially screened for inclusion by two independent reviewers. The Prediction Model Risk of Bias Assessment Tool (PROBAST) and the Revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklists were used to assess risk of bias and applicability of prediction tools. A narrative synthesis was performed.

RESULTS

A total of 21 studies were included, 14 of which described the development of new prediction tools (four risk assessment tools and ten clinical prediction models) and six studies were validation based and one an impact study. There were variations in tool development processes, outcome measures and included predictors. Overall, tool performance had limitations in reporting and consistency, with the discriminatory ability based on area under the curve receiver operating characteristics (AUROC) ranging from poor to good (0.62-0.81), sensitivity and specificity ranging from 57.0% to 89.9% and 30.2% to 88.0%, respectively. The Medicines Optimisation Assessment tool and Assessment of Risk tool were prediction tools with the lowest risk of bias and low concern for applicability. Studies reporting external validation and impact on patient outcomes were scarce.

CONCLUSION

Most prediction tools have limitations in development and validation processes, as well as scarce evidence of clinical usefulness. Future studies should attempt to either refine currently available tools or apply a rigorous process capturing evidence of acceptance, usefulness, performance and outcomes.

External validation of the Medication Risk Score in polypharmacy patients in general practice: A tool for prioritizing patients at greatest risk of potential drug-related problems

Drug-related problems are important causes of patient harm and increased healthcare costs. To assist general practitioners in prioritizing patients in need of a critical medication review, we aimed to assess the ability of the Medication Risk Score (MERIS) to stratify patients with polypharmacy in general practice according to their risk of drug-related problems. We conducted a cross-sectional multi-centre external validation study. Patients receiving more than five concomitant medications (polypharmacy) were eligible. The outcome was potentially serious drug-related problems as evaluated by expert consensus. Performance was assessed in terms of calibration and discrimination indices. Of 497 patients, 489 were included in the main analysis. The median age (interquartile range) was 70.5 years (60-79). In total, 372 potentially serious drug-related problems were observed in 253 patients (52%). The MERIS was well calibrated above a score level of 10. The area under the receiver operating characteristic curve was 0.70 (95% confidence interval: 0.65-0.74). The performance of the MERIS was fair in patients with polypharmacy in general practice. Given the scale of drug-related problems and the lack of efficient prioritization tools in this setting, the MERIS could be a useful risk indicator to complement usual practice.