Reducing non-attendance in outpatient appointments: predictive model development, validation, and clinical assessment

Decision analysis framework for predicting no-shows to appointments using machine learning algorithms

BACKGROUND

No-show to medical appointments has significant adverse effects on healthcare systems and their clients. Using machine learning to predict no-shows allows managers to implement strategies such as overbooking and reminders targeting patients most likely to miss appointments, optimizing the use of resources.

METHODS

In this study, we proposed a detailed analytical framework for predicting no-shows while addressing imbalanced datasets. The framework includes a novel use of z-fold cross-validation performed twice during the modeling process to improve model robustness and generalization. We also introduce Symbolic Regression (SR) as a classification algorithm and Instance Hardness Threshold (IHT) as a resampling technique and compared their performance with that of other classification algorithms, such as K-Nearest Neighbors (KNN) and Support Vector Machine (SVM), and resampling techniques, such as Random under Sampling (RUS), Synthetic Minority Oversampling Technique (SMOTE) and NearMiss-1. We validated the framework using two attendance datasets from Brazilian hospitals with no-show rates of 6.65% and 19.03%.

RESULTS

From the academic perspective, our study is the first to propose using SR and IHT to predict the no-show of patients. Our findings indicate that SR and IHT presented superior performances compared to other techniques, particularly IHT, which excelled when combined with all classification algorithms and led to low variability in performance metrics results. Our results also outperformed sensitivity outcomes reported in the literature, with values above 0.94 for both datasets.

CONCLUSION

This is the first study to use SR and IHT methods to predict patient no-shows and the first to propose performing z-fold cross-validation twice. Our study highlights the importance of avoiding relying on few validation runs for imbalanced datasets as it may lead to biased results and inadequate analysis of the generalization and stability of the models obtained during the training stage.

Mapping out the arguments for and against patient non-attendance fees in healthcare: an analysis of public consultation documents

BACKGROUND

Patients not attending their appointments without giving notice burden healthcare services. To reduce non-attendance rates, patient non-attendance fees have been introduced in various settings. Although some argue in narrow economic terms that behavioural change as a result of financial incentives is a voluntary transaction, charging patients for non-attendance remains controversial. This paper aims to investigate the controversies of implementing patient non-attendance fees.

OBJECTIVE

The aim was to map out the arguments in the Norwegian public debate concerning the introduction and use of patient non-attendance fees at public outpatient clinics.

METHODS

Public consultation documents (2009-2021) were thematically analysed (n=84). We used a preconceived conceptual framework based on the works of Grant to guide the analysis.

RESULTS

A broad range of arguments for and against patient non-attendance fees were identified, here referring to the acceptability of the fees' purpose, the voluntariness of the responses, the effects on the individual character and institutional norms and the perceived fairness and comparative effectiveness of patient non-attendance fees. Whereas the aim of motivating patients to keep their appointments to avoid poor utilisation of resources and increased waiting times was widely supported, principled and practical arguments against patient non-attendance fees were raised.

CONCLUSION

A narrow economic understanding of incentives cannot capture the breadth of arguments for and against patient non-attendance fees. Policy makers may draw on this insight when implementing similar incentive schemes. The study may also contribute to the general debate on ethics and incentives.

Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative

BACKGROUND

Appointment no shows are prevalent in safety-net healthcare systems. The efficacy and equitability of using predictive algorithms to selectively add resource-intensive live telephone outreach to standard automated reminders in such a setting is not known.

OBJECTIVE

To determine if adding risk-driven telephone outreach to standard automated reminders can improve in-person primary care internal medicine clinic no show rates without worsening racial and ethnic show-rate disparities.

DESIGN

Randomized controlled quality improvement initiative.

PARTICIPANTS

Adult patients with an in-person appointment at a primary care internal medicine clinic in a safety-net healthcare system from 1/1/2022 to 8/24/2022.

INTERVENTIONS

A random forest model that leveraged electronic health record data to predict appointment no show risk was internally trained and validated to ensure fair performance. Schedulers leveraged the model to place reminder calls to patients in the augmented care arm who had a predicted no show rate of 15% or higher.

MAINE MEASURES

The primary outcome was no show rate stratified by race and ethnicity.

KEY RESULTS

There were 5840 appointments with a predicted no show rate of 15% or higher. A total of 2858 had been randomized to the augmented care group and 2982 randomized to standard care. The augmented care group had a significantly lower no show rate than the standard care group (33% vs 36%, p < 0.01). There was a significant reduction in no show rates for Black patients (36% vs 42% respectively, p < 0.001) not reflected in white, non-Hispanic patients.

CONCLUSIONS

In this randomized controlled quality improvement initiative, adding model-driven telephone outreach to standard automated reminders was associated with a significant reduction of in-person no show rates in a diverse primary care clinic. The initiative reduced no show disparities by predominantly improving access for Black patients.

Predictive model-based interventions to reduce outpatient no-shows: a rapid systematic review

OBJECTIVE

Outpatient no-shows have important implications for costs and the quality of care. Predictive models of no-shows could be used to target intervention delivery to reduce no-shows. We reviewed the effectiveness of predictive model-based interventions on outpatient no-shows, intervention costs, acceptability, and equity.

MATERIALS AND METHODS

Rapid systematic review of randomized controlled trials (RCTs) and non-RCTs. We searched Medline, Cochrane CENTRAL, Embase, IEEE Xplore, and Clinical Trial Registries on March 30, 2022 (updated on July 8, 2022). Two reviewers extracted outcome data and assessed the risk of bias using ROB 2, ROBINS-I, and confidence in the evidence using GRADE. We calculated risk ratios (RRs) for the relationship between the intervention and no-show rates (primary outcome), compared with usual appointment scheduling. Meta-analysis was not possible due to heterogeneity.

RESULTS

We included 7 RCTs and 1 non-RCT, in dermatology (n = 2), outpatient primary care (n = 2), endoscopy, oncology, mental health, pneumology, and an magnetic resonance imaging clinic. There was high certainty evidence that predictive model-based text message reminders reduced no-shows (1 RCT, median RR 0.91, interquartile range [IQR] 0.90, 0.92). There was moderate certainty evidence that predictive model-based phone call reminders (3 RCTs, median RR 0.61, IQR 0.49, 0.68) and patient navigators reduced no-shows (1 RCT, RR 0.55, 95% confidence interval 0.46, 0.67). The effect of predictive model-based overbooking was uncertain. Limited information was reported on cost-effectiveness, acceptability, and equity.

DISCUSSION AND CONCLUSIONS

Predictive modeling plus text message reminders, phone call reminders, and patient navigator calls are probably effective at reducing no-shows. Further research is needed on the comparative effectiveness of predictive model-based interventions addressed to patients at high risk of no-shows versus nontargeted interventions addressed to all patients.

Social determinants of health and the prediction of missed breast imaging appointments

BACKGROUND

Predictive models utilizing social determinants of health (SDH), demographic data, and local weather data were trained to predict missed imaging appointments (MIA) among breast imaging patients at the Boston Medical Center (BMC). Patients were characterized by many different variables, including social needs, demographics, imaging utilization, appointment features, and weather conditions on the date of the appointment.

METHODS

This HIPAA compliant retrospective cohort study was IRB approved. Informed consent was waived. After data preprocessing steps, the dataset contained 9,970 patients and 36,606 appointments from 1/1/2015 to 12/31/2019. We identified 57 potentially impactful variables used in the initial prediction model and assessed each patient for MIA. We then developed a parsimonious model via recursive feature elimination, which identified the 25 most predictive variables. We utilized linear and non-linear models including support vector machines (SVM), logistic regression (LR), and random forest (RF) to predict MIA and compared their performance.

RESULTS

The highest-performing full model is the nonlinear RF, achieving the highest Area Under the ROC Curve (AUC) of 76% and average F1 score of 85%. Models limited to the most predictive variables were able to attain AUC and F1 scores comparable to models with all variables included. The variables most predictive of missed appointments included timing, prior appointment history, referral department of origin, and socioeconomic factors such as household income and access to caregiving services.

CONCLUSIONS

Prediction of MIA with the data available is inherently limited by the complex, multifactorial nature of MIA. However, the algorithms presented achieved acceptable performance and demonstrated that socioeconomic factors were useful predictors of MIA. In contrast with non-modifiable demographic factors, we can address SDH to decrease the incidence of MIA.