Risk Prediction Tool for Medical Appointment Attendance Among HIV-Infected Persons with Unsuppressed Viremia

Development of a machine learning prediction model for loss to follow-up in HIV care using routine electronic medical records in a low-resource setting

BACKGROUND

Despite the global commitment to ending AIDS by 2030, the loss of follow-up (LTFU) in HIV care remains a significant challenge. To address this issue, a data-driven clinical decision tool is crucial for identifying patients at greater risk of LTFU and facilitating personalized and proactive interventions. This study aimed to develop a prediction model to assess the future risk of LTFU in HIV care in Ethiopia.

METHODS

The study used a retrospective design in which machine learning (ML) methods were applied to the electronic medical records (EMRs) data of adult HIV-positive individuals who were newly enrolled in antiretroviral therapy between July 2019 and April 2024. The data were collected across eight randomly selected high-volume healthcare facilities. Six supervised ML classifiers-J48 decision tree, random forest, K-nearest neighbors, support vector machine, logistic regression, and naïve Bayes-were utilized for training via Weka 3.8.6 software. The performance of each algorithm was evaluated through a 10-fold cross-validation approach. Algorithm performance was compared via the corrected resampled t test (p < 0.05), and decision curve analysis (DCA) was used to assess the model's clinical utility.

RESULTS

A total of 3,720 individuals' EMR data were analyzed, with 2,575 (69.2%) classified as not LTFU and 1,145 (30.8%) classified as LTFU. On the basis of the ML feature selection process, six strong predictors of LTFU were identified: differentiated service delivery model, adherence, tuberculosis preventive therapy, follow-up period, nutritional status, and address information. The random forest algorithm showed superior performance, with an accuracy of 84.2%, a sensitivity of 82.4%, a specificity of 85.7%, a precision of 83.7%, an F1 score of 83.1%, and an area under the curve of 89.5%. The model demonstrated greater clinical utility, offering greater net benefit than both the 'intervention for all' approach and the 'intervention for none' approach, particularly at threshold probabilities of 10% and above.

CONCLUSIONS

This study developed a machine learning-based predictive model for assessing the future risk of LTFU in HIV care within low-resource settings. Notably, the model built via the random forest algorithm exhibited high accuracy and strong discriminative performance, highlighting its positive net benefit for clinical applications. Furthermore, ongoing external validation across diverse populations is important to ensure the model's reliability and generalizability.

Barriers to Retention in Care among Adults with HIV in Developed Countries: An Integrative Review

Focusing on factors hindering viral suppression is essential for improving the health outcomes of people with Human Immunodeficiency Virus (HIV) and working towards ending the HIV/AIDS epidemic. The aim of this integrative review is to create an overview of barriers to retention in care among adults with HIV living in developed countries. Based on a systematic literature search across EMBASE, PubMed, Scopus, CINAHL, and PsycInfo, 4,089 studies of various methodology were identified. A total of 52 studies met the inclusion criteria. Quality assessment was performed using the Mixed Method Appraisal Tool. Based on thematic analysis, the following five main themes were identified as most common barriers to retention in care: financial challenges, logistical challenges, stigma, mental health problems, and substance use. The integrative review highlights that various factors can hinder retention in care and underscores that strategies to promote retention in care should be person-centered and targeted the individual person's barriers to retention in care.

Accuracy of Provider Predictions of Viral Suppression Among Adolescents and Young Adults With HIV in an HIV Clinical Program

BACKGROUND

Providers caring for adolescents and young adults with HIV (AYA-HIV) mostly base their adherence counseling during clinical encounters on clinical judgment and expectations of patients' medication adherence. There is currently no data on provider predictions of viral suppression for AYA-HIV. We aimed to assess the accuracy of provider predictions of patients' viral suppression status compared to viral load results.

METHODS

Providers caring for AYA-HIV were asked to predict the likelihood of viral suppression of patients before a clinical encounter and give reasons for their predictions. Provider predictions were compared to actual viral load measurements of patients. Patient data were abstracted from electronic health records. The final analysis included 9 providers, 28 patients, and 34 observations of paired provider predictions and viral load results.

RESULTS

Provider prediction accuracy of viral suppression was low (59%, Cohen's Kappa = 0.16). Provider predictions of lack of viral suppression were based on nonadherence to medications, new patient status, or structural vulnerabilities (e.g., unstable housing). Anticipated viral suppression was based on medication adherence, history of viral suppression, and the presence of family or other social forms of support.

CONCLUSIONS

Providers have difficulty accurately predicting viral suppression among AYA-HIV and may base their counseling on incorrect assumptions. Rapid point-of-care viral load testing may provide opportunities to improve counseling provided during the clinical encounter.

Distance Matters: Investigating No-Shows in a Large Rural Provider Network

Background/Objective: No-shows have a negative effect on healthcare outcomes. It is unclear, however, whether patients' distance from the clinic is associated with higher no-show rates. To fill this knowledge gap, we examined the relationship between patients' distance from the clinic and no-shows in a rural provider network.Methods: Data from Marshfield Clinic Health System's scheduling system, including 263,464 recent patient appointments in 2021 were analyzed. The outcome was no-shows, defined as when patients missed an appointment (categorized as yes/no). The exposure was the distance to the clinic, measured in miles as a straight-line distance from the clinic in the patient's zip code to the facility where the appointment was held (classified as <5 miles, 5-10, 10-20; >20, and used as continuous). Covariates were patient demographics, appointments, providers, and insurance status. Chi-square and logistic regression were used with p-values ≤.05 considered statistically significant.Results: The no-show rate was 8.0%. Patients who lived <5 miles (8.3%) and >20 miles (8.2%) from the clinic had higher no-show rates than those who lived between 10-20 miles (8.0%) and 5-10 miles (7.6%), at P=0.001. In the adjusted model, the odds of no-show were similar between patients who did not show and those who did (OR:1.00,95%CI:1.00-1.00). No-shows were more likely among male patients compared to females (OR:1.14,95%CI:1.11-1.18), Spanish compared to English speakers (OR:1.34,95%CI:1.20-1.50), prior no-show compared to no prior no-show (OR:4.42,95%CI:4.27-4.48), >4 weeks lead time compared to <1 day (OR:5.45,95%CI:4.98-5.97), and Medicaid compared to non-Medicaid patients (OR:1.56,95%CI:1.49-1.63).Conclusion: Our analysis showed patients who lived <5 miles and >20 miles from the clinic had higher no-show rates. The odds of a no-show were comparable between patients who showed up and those who did not. Male patients, Spanish-speaking patients, patients with a history of no-shows, and Medicaid beneficiaries were more likely to miss their appointments. Understanding the impact of these variables on no-show rates can assist healthcare providers in developing strategies to improve patient access and reduce no-show rates. These findings imply that rural patients may face a variety of barriers when seeking healthcare, necessitating a comprehensive approach to addressing this issue.

Development of an evidence-based model for predicting patient, provider, and appointment factors that influence no-shows in a rural healthcare system

BACKGROUND

No-show appointments pose a significant challenge for healthcare providers, particularly in rural areas. In this study, we developed an evidence-based predictive model for patient no-shows at the Marshfield Clinic Health System (MCHS) rural provider network in Wisconsin, with the aim of improving overbooking approaches in outpatient settings and reducing the negative impact of no-shows in our underserved rural patient populations.

METHODS

Retrospective data (2021) were obtained from the MCHS scheduling system, which included 1,260,083 total appointments from 263,464 patients, as well as their demographic, appointment, and insurance information. We used descriptive statistics to associate variables with show or no-show status, logistic regression, and random forests utilized, and eXtreme Gradient Boosting (XGBoost) was chosen to develop the final model, determine cut-offs, and evaluate performance. We also used the model to predict future no-shows for appointments from 2022 and onwards.

RESULTS

The no-show rate was 6.0% in both the train and test datasets. The train and test datasets both yielded 5.98. Appointments scheduled further in advance (> 60 days of lead time) had a higher (7.7%) no-show rate. Appointments for patients aged 21-30 had the highest no-show rate (11.8%), and those for patients over 60 years of age had the lowest (2.9%). The model predictions yielded an Area Under Curve (AUC) of 0.84 for the train set and 0.83 for the test set. With the cut-off set to 0.4, the sensitivity was 0.71 and the positive predictive value was 0.18. Model results were used to recommend 1 overbook for every 6 at-risk appointments per provider per day.

CONCLUSIONS

Our findings demonstrate the feasibility of developing a predictive model based on administrative data from a predominantly rural healthcare system. Our new model distinguished between show and no-show appointments with high performance, and 1 overbook was advised for every 6 at-risk appointments. This data-driven approach to mitigating the impact of no-shows increases treatment availability in rural areas by overbooking appointment slots on days with an elevated risk of no-shows.

Implementation of Lost & Found, An Intervention to Reengage Patients Out of HIV Care: A Convergent Explanatory Sequential Mixed-Methods Analysis

Being out of HIV care (OOC) is associated with increased morbidity and mortality. We assessed implementation of Lost & Found, a clinic-based intervention to reengage OOC patients. OOC patients were identified using a nurse-validated, real-time OOC list within the electronic medical records (EMR) system. Nurses called OOC patients. Implementation occurred at the McGill University Health Centre from April 2018 to 2019. Results from questionnaires to nurses showed elevated scores for implementation outcomes throughout, but with lower, more variable scores during pre-implementation to month 3 [e.g., adoption subscales (scale: 1-5): range from pre-implementation to month 3, 3.7-4.9; thereafter, 4.2-4.9]. Qualitative results from focus groups with nurses were consistent with observed quantitative trends. Barriers concerning the EMR and nursing staff shortages explained reductions in fidelity. Strategies for overcoming barriers to implementation were crucial in early months of implementation. Intervention compatibility, information systems support, as well as nurses' team processes, knowledge, and skills facilitated implementation.

Baseline and Process Factors of Anti-Retroviral Therapy That Predict Loss to Follow-up Among People Living with HIV/AIDS in China: A Retrospective Cohort Study

We explored the predictors and predictive models of loss to follow-up (LTFU) during the first year of anti-retroviral therapy (ART). LTFU was defined as the failure to visit the clinic for antiretroviral drugs for ≥ 90 days after the last missed scheduled visit. Based on the electronic medical records of 5953 patients who were HIV positive and began ART between 2016 and 2019 in China, the LTFU rate was 7.24 (95% confidence interval 6.49-7.97) per 100 person-years during the first year of ART. ART baseline factors were associated with LTFU, but were non-optimal predictors. A model including ART process-related factors such as follow-up behaviors and physical health status had an area under the receiver operating characteristic curve of 73.4% for predicting LTFU. Therefore, the medical records of follow-up visits can be used to identify patients with a high risk of LTFU and allow interventions to be implemented proactively.

Factors Impacting Video Telehealth Appointment Completion During COVID-19 Pandemic Among People Living with HIV in a Community-Based Health System

As the threat of COVID-19 on vulnerable populations continues, mitigation protocols have escalated the use of telehealth platforms, secure 2-way video platforms with audio capabilities. The goal of the current study was to examine factors associated with successful completion of video telehealth appointments in HIV care. We utilized a random effects logistic model to assess characteristics of patient encounters that predicted completed telehealth visits. Results show that factors such as identifying as black (AOR = 0.30, 95% CI 0.23–0.40, p < 0.01), identifying as heterosexual (AOR = 0.40, 95% CI, 0.29–0.55, p < 0.01), identifying as Hispanic/Latinx (AOR = 0.67, 95% CI, 0.48–0.95), having public insurance (e.g., Ryan White funding, Medicare/Medicaid) (AOR = .25, 95% CI 0.19–0.33, p < .001), and having detectable viral load (AOR = .049, 95% CI, 0.31–0.76) are negatively associated with completion of telehealth appointments. Results suggest that greater efforts to address the digital divide are needed to increase access to video telehealth.

Age and Year of HIV Diagnosis are Associated with Perceptions of Discrimination and Internalized Stigma Among Sexual Minority Men Who Use Substances

Discrimination and internalized stigma are barriers to engagement in HIV self-care among men who have sex with men (MSM) living with HIV. However, differences in perceptions of discrimination and internalized stigmas by age, year of HIV-diagnosis, and race are poorly understood. We assessed differences in reported discrimination related to HIV, race, sexual orientation, and substance use and internalized stigmas among 202 MSM living with HIV who use substances. Younger participants reported higher levels of all types of discrimination and internalized stigmas (p-values < 0.001-0.030). Those diagnosed after the advent of antiretrovirals reported higher levels of discrimination related to HIV, sexual orientation, and substance use, as well as internalized stigma related to HIV and substance use (p-values 0.001-0.049). We explored perceived community HIV stigma, which accounted for associations involving age and year of diagnosis. Age, year of diagnosis, and race should be considered when assessing and intervening with stigma.

Substance Use Stigma, Avoidance Coping, and Missed HIV Appointments Among MSM Who Use Substances

Men who have sex with men (MSM) living with HIV who use substances have multiple stigmatized identities. Theory suggests that internalization of stigma may elicit avoidance behaviors associated with these stigmas, potentially resulting in suboptimal engagement in HIV care. We investigated interrelationships between internalized stigmas related to HIV, sexual orientation, and substance use; avoidance coping; and missed HIV appointments among 202 MSM living with HIV who use substances. Neither HIV nor sexual orientation-related internalized stigmas were associated with missed appointments, however, internalized substance use stigma (SUS) was associated (OR 1.47, 95% CI 1.15, 1.87). The relationship between internalized SUS and missed appointments was partially accounted for by avoidance coping (b = 0.12; bootstrap 95% CI 0.02, 0.25). To better understand the role of SUS, we assessed relationships between enacted and anticipated SUS and missed appointments (OR 2.08, 95% CI 1.52, 2.84 and OR 1.44, 95% CI 1.10, 1.88, respectively). Avoidance coping fully accounted for the relationship between anticipated SUS and missed appointments (b = 0.12; 95% CI 0.02, 0.25). Results suggest that avoidance strategies to manage anticipated SUS may result in substance using MSM forgoing HIV care appointments.

Reducing Missed Appointments for Patients With HIV: An Evidence-Based Approach

BACKGROUND

High rates of missed appointments for routine HIV care are associated with unsuppressed viremia, increasing morbidity.

LOCAL PROBLEM

The Clinic no-show rate ranged between 30% and 35%, and only 69% of patients were considered retained in care within a 24-month time frame.

METHODS

The Woodward Risk Prediction Tool was completed on all patients to stratify patient risk for missing the next appointment.

INTERVENTIONS

All patients were offered text message along with standard phone message appointment reminders, and patients who missed appointments were called within 24 hours to reschedule. Medium-risk patients received a previsit planning call to remove barriers to appointment attendance, and high-risk patients received a home visit from the peer navigator.

RESULTS

The project resulted in a 3.8% reduction rate in the overall no-show rate in the first 5 months of implementation. Using risk stratification and targeted interventions allowed valuable resources to be allocated where they were needed.

Patient No-Show Prediction: A Systematic Literature Review

Nowadays, across the most important problems faced by health centers are those caused by the existence of patients who do not attend their appointments. Among others, these patients cause loss of revenue to the health centers and increase the patients’ waiting list. In order to tackle these problems, several scheduling systems have been developed. Many of them require predicting whether a patient will show up for an appointment. However, obtaining these estimates accurately is currently a challenging problem. In this work, a systematic review of the literature on predicting patient no-shows is conducted aiming at establishing the current state-of-the-art. Based on a systematic review following the PRISMA methodology, 50 articles were found and analyzed. Of these articles, 82% were published in the last 10 years and the most used technique was logistic regression. In addition, there is significant growth in the size of the databases used to build the classifiers. An important finding is that only two studies achieved an accuracy higher than the show rate. Moreover, a single study attained an area under the curve greater than the 0.9 value. These facts indicate the difficulty of this problem and the need for further research.

Developing a triage tool for use in identifying people living with HIV who are at risk for non-retention in HIV care

Introduction:

Identifying PLHIV in HIV care who are at particular risk of non-retention in care is an important element in improving their HIV care outcomes. The purpose of this study was to develop a risk prediction tool to identify PLHIV at risk of non-retention in care over the course of the next year.

Method:

We used stepwise logistic regression to assess sociodemographic, clinical and behavioral predictors of non-retention in HIV care. Retention in care was defined as having evidence of at least two encounters with an HIV care provider (or CD4 or viral load lab tests as a proxy measure for the encounter), at least 3 months apart within a year. We validated the risk prediction tool internally using the bootstrap method.

Results:

The risk prediction tool included a total of six factors: age group, race, poverty level, homelessness, problematic alcohol/drug use and viral suppression status. The total risk score ranged from 0 to 17. Compared to those in the lowest quartile (0 risk score), those who were in the middle two quartiles (score 1–4) and those in the upper quartile (>4 risk score) were more likely not to be retained in care (odds ratio [OR] 1.63 [CI; 1.39–1.92] and OR 4.82 [CI; 4.04–5.78] respectively). The discrimination ability for the prediction model was 0.651.

Conclusion:

We found that increased risk for non-retention in care can be predicted with routinely available variables. Since the discrimination of the tool was low, future studies may need to include more prognostic factors in the risk prediction tool.

A type II implementation-effectiveness hybrid quasi-experimental pilot study of a clinical intervention to re-engage people living with HIV into care, 'Lost & Found': an implementation science protocol

Background

At the McGill University Health Centre (MUHC), 10% of patients living with HIV do not return for care annually. Currently, no formal system exists to re-engage out-of-care (OOC) patients. Lost & Found, developed using an implementation science approach, is an intervention to re-engage OOC patients. It is based on existing evidence-based interventions and will be adapted for use by nurses at the MUHC. The aims of this study are to simultaneously assess both implementation and effectiveness of Lost & Found in order to determine the viability of a future multisite stepped-wedge cluster randomised trial.

Methods

Lost & Found consists of two core elements: identifying and contacting OOC patients. Based on formative work involving MUHC nurses, and the use of a combined implementation framework (enhanced Replicating Effective Programs, Tailored Implementation for Chronic Diseases, and Proctor et al.’s implementation outcomes), we will adapt the intervention to our clinic. Adaptations include the creation of an OOC risk prediction tool, an automated real-time OOC list, and prioritization of high-risk OOC patients for re-engagement. Delivery and ongoing adaptation of the intervention will follow a three-pronged implementation strategy consisting of (1) promoting adaptability; (2) planning, engaging, executing, evaluating, and reflecting cycles; and (3) internal facilitation. This 15-month quasi-experimental pilot study adopts a type II implementation-effectiveness hybrid design. To evaluate implementation, a convergent parallel mixed-methods approach will guide the mixing of qualitative and quantitative data at time points throughout the study. In addition, descriptive and pre-post analyses, for each of the implementation and sustainability phases, will inform evaluations of the cumulative effectiveness and sustainability of the Lost & Found intervention.

Discussion

This study will provide preliminary evidence for (1) the utility of our chosen implementation strategies and (2) the effectiveness of the intervention. Ultimately, this information may be used to inform future re-engagement efforts using implementation science in other HIV care centres. In addition, the procedures and measurement tools developed for this study will be foundational to the development of a multi-site, randomised stepped wedge study that would provide more robust evidence in support of the Lost & Found intervention.

Early suboptimal ART adherence was associated with missed clinical visits in HIV-infected patients in Asia

Missed clinic visits can lead to poorer treatment outcomes in HIV-infected patients. Suboptimal antiretroviral therapy (ART) adherence has been linked to subsequent missed visits. Knowing the determinants of missed visits in Asian patients will allow for appropriate counselling and intervention strategies to ensure continuous engagement in care. A missed visit was defined as having no assessments within six months. Repeated measures logistic regression was used to analyse factors associated with missed visits. A total of 7100 patients were included from 12 countries in Asia with 2676 (37.7%) having at least one missed visit. Patients with early suboptimal self-reported adherence <95% were more likely to have a missed visit compared to those with adherence ≥95% (OR = 2.55, 95% CI(1.81-3.61)). Other factors associated with having a missed visit were homosexual (OR = 1.45, 95%CI(1.27-1.66)) and other modes of HIV exposure (OR = 1.48, 95%CI(1.27-1.74)) compared to heterosexual exposure; using PI-based (OR = 1.33, 95%CI(1.15-1.53) and other ART combinations (OR = 1.79, 95%CI(1.39-2.32)) compared to NRTI+NNRTI combinations; and being hepatitis C co-infected (OR = 1.27, 95%CI(1.06-1.52)). Patients aged >30 years (31-40 years OR = 0.81, 95%CI(0.73-0.89); 41-50 years OR = 0.73, 95%CI(0.64-0.83); and >50 years OR = 0.77, 95%CI(0.64-0.93)); female sex (OR = 0.81, 95%CI(0.72-0.90)); and being from upper middle (OR = 0.78, 95%CI(0.70-0.80)) or high-income countries (OR = 0.42, 95%CI(0.35-0.51)), were less likely to have missed visits. Almost 40% of our patients had a missed clinic visit. Early ART adherence was an indicator of subsequent clinic visits. Intensive counselling and adherence support should be provided at ART initiation in order to optimise long-term clinic attendance and maximise treatment outcomes.

Health Literacy and Demographic Disparities in HIV Care Continuum Outcomes

Studies evaluating the association between human immunodeficiency virus (HIV) infection continuum of care outcomes [antiretroviral (ART) adherence, retention in care, viral suppression] and health literacy have yielded conflicting results. Moreover, studies from the southern United States, a region of the country disproportionately affected by the HIV epidemic and low health literacy, are lacking. We conducted an observational cohort study among 575 people living with HIV (PLWH) at the Vanderbilt Comprehensive Care Clinic (Nashville, Tennessee). Health literacy was measured using the brief health literacy screen, a short tool which can be administered verbally by trained clinical personnel. Low health literacy was associated with a lack of viral suppression, but not with poor ART adherence or poor retention. Age and racial disparities in continuum of care outcomes persisted after accounting for health literacy, suggesting that factors in addition to health literacy must be addressed in order to improve outcomes for PLWH.

Health Care Utilization Behaviors Predict Disengagement From HIV Care: A Latent Class Analysis

Background

The traditional definition of engagement in HIV care in terms of only clinic attendance and viral suppression provides a limited understanding of how persons living with HIV (PLWH) interact with the health care system.

Methods

We conducted a retrospective analysis of patients with ≥1 HIV clinic visits at the Duke Adult Infectious Diseases Clinic between 2008 and 2013. Health care utilization was characterized by 4 indicators: clinic attendance in each half of the year (yes/no), number of emergency department (ED) visits/year (0, 1, or 2+), inpatient admissions/year (0, 1, 2+), and viral suppression (never, intermittent, always). Health care engagement patterns were modeled using latent class/latent transition analysis.

Results 

A total of 2288 patients (median age, 46.4 years; 59% black, 71% male) were included in the analysis. Three care engagement classes were derived from the latent class model: "adherent" "nonadherent," and "sick." Patients age ≤40 years were more likely to be in the nonadherent class (odds ratio, 2.64; 95% confidence interval, 1.38-5.04) than other cohort members. Whites and males were more likely to transition from nonadherent to adherent the following year. Nonadherent patients were significantly more likely to disengage from care the subsequent year than adherent patients (23.6 vs 0.2%, P < .001).

Conclusions

A broader definition of health care engagement revealed distinct and dynamic patterns among PLWH that would have been hidden had only previous HIV clinic attendance had been considered. These patterns may be useful for designing engagement-targeted interventions.