Online Reviews of Mental Health Treatment Facilities: Narrative Themes Associated With Positive and Negative Ratings

Disparities by Race and Urbanicity in Online Health Care Facility Reviews

Importance

Online review platforms offer valuable insights into patient satisfaction and the quality of health care services, capturing content and trends that traditional metrics might miss. The COVID-19 pandemic has disrupted health care services, influencing patient experiences.

Objective

To examine health care facility numerical ratings and patient experience reported on an online platform by facility type and area demographic characteristics after the COVID-19 pandemic (ie, post-COVID).

Design, Setting, and Participants

All reviews of US health care facilities posted on one online platform from January 1, 2014, to December 31, 2023, were obtained for this cross-sectional study. Analyses focused on facilities providing essential health benefits, which are service categories that health insurance plans must cover under the Affordable Care Act. Facility zip code tabulation area level demographic data were obtained from US census and rural-urban commuting area codes.

Main Outcomes and Measures

The primary outcome was the change in the percentage of positive reviews (defined as reviews with ≥4 of 5 stars) before and post-COVID. Secondary outcomes included the association between positive ratings and facility demographic characteristics (race and ethnicity and urbanicity), and thematic analysis of review content using latent Dirichlet allocation.

Results

A total of 1 445 706 reviews across 151 307 facilities were included. The percent of positive reviews decreased from 54.3% to 47.9% (P < .001) after March 2020. Rural areas, areas with a higher proportion of Black residents, and areas with a higher proportion of White residents experienced lower positive ratings post-COVID, while reviews in areas with a higher proportion of Hispanic residents were less negatively impacted (P < .001 for all comparisons). For example, logistic regression showed that rural areas had significantly lower odds of positive reviews post-COVID compared with urban areas (odds ratio, 0.77; 95% CI, 0.72-0.83). Latent Dirichlet allocation identified themes such as billing issues, poor customer service, and insurance handling that increased post-COVID among certain communities. For instance, areas with a higher proportion of Black residents and areas with a higher proportion of Hispanic residents reported increases in insurance and billing issues, while areas with a higher proportion of White residents reported increases in wait time among negative reviews.

Conclusions and Relevance

This serial cross-sectional study observed a significant decrease in positive reviews for health care facilities post-COVID. These findings underscore a disparity in patient experience, particularly in rural areas and areas with the highest proportions of Black and White residents.

Association Between Online Reviews of Substance Use Disorder Treatment Facilities and Drug-Induced Mortality Rates: Cross-Sectional Analysis

BACKGROUND

Drug-induced mortality across the United States has continued to rise. To date, there are limited measures to evaluate patient preferences and priorities regarding substance use disorder (SUD) treatment, and many patients do not have access to evidence-based treatment options. Patients and their families seeking SUD treatment may begin their search for an SUD treatment facility online, where they can find information about individual facilities, as well as a summary of patient-generated web-based reviews via popular platforms such as Google or Yelp. Web-based reviews of health care facilities may reflect information about factors associated with positive or negative patient satisfaction. The association between patient satisfaction with SUD treatment and drug-induced mortality is not well understood.

OBJECTIVE

The objective of this study was to examine the association between online review content of SUD treatment facilities and drug-induced state mortality.

METHODS

A cross-sectional analysis of online reviews and ratings of Substance Abuse and Mental Health Services Administration (SAMHSA)-designated SUD treatment facilities listed between September 2005 and October 2021 was conducted. The primary outcomes were (1) mean online rating of SUD treatment facilities from 1 star (worst) to 5 stars (best) and (2) average drug-induced mortality rates from the Centers for Disease Control and Prevention (CDC) WONDER Database (2006-2019). Clusters of words with differential frequencies within reviews were identified. A 3-level linear model was used to estimate the association between online review ratings and drug-induced mortality.

RESULTS

A total of 589 SAMHSA-designated facilities (n=9597 reviews) were included in this study. Drug-induced mortality was compared with the average. Approximately half (24/47, 51%) of states had below average ("low") mortality rates (mean 13.40, SD 2.45 deaths per 100,000 people), and half (23/47, 49%) had above average ("high") drug-induced mortality rates (mean 21.92, SD 3.69 deaths per 100,000 people). The top 5 themes associated with low drug-induced mortality included detoxification and addiction rehabilitation services (r=0.26), gratitude for recovery (r=-0.25), thankful for treatment (r=-0.32), caring staff and amazing experience (r=-0.23), and individualized recovery programs (r=-0.20). The top 5 themes associated with high mortality were care from doctors or providers (r=0.24), rude and insensitive care (r=0.23), medication and prescriptions (r=0.22), front desk and reception experience (r=0.22), and dissatisfaction with communication (r=0.21). In the multilevel linear model, a state with a 10 deaths per 100,000 people increase in mortality was associated with a 0.30 lower average Yelp rating (P=.005).

CONCLUSIONS

Lower online ratings of SUD treatment facilities were associated with higher drug-induced mortality at the state level. Elements of patient experience may be associated with state-level mortality. Identified themes from online, organically derived patient content can inform efforts to improve high-quality and patient-centered SUD care.

Experiences of Black Adults Evaluated in a Locked Psychiatric Emergency Unit: A Qualitative Study

OBJECTIVE

Evidence shows that Black individuals have higher rates of coercive emergency psychiatric interventions than other racialized groups, yet no studies have elevated the voices of Black patients undergoing emergency psychiatric evaluation. This qualitative study sought to explore the experiences of Black individuals who had been evaluated in a locked psychiatric emergency unit (PEU).

METHODS

Electronic health records were used to identify and recruit adult patients (ages ≥18 years) who self-identified as Black and who had undergone evaluation in a locked PEU at a large academic medical center. In total, 11 semistructured, one-on-one interviews were conducted by telephone, exploring experiences during psychiatric evaluation. Transcripts were analyzed with thematic analysis.

RESULTS

Participants shared experiences of criminalization, stigma, and vulnerability before and during their evaluation. Although participants described insight into their desire and need for treatment and identified helpful aspects of the care they received, they noted a mismatch between their expectations of treatment and the treatment received.

CONCLUSIONS

This study reveals six major patient-identified themes that supplement a growing body of quantitative evidence demonstrating that racialized minority groups endure disproportionate rates of coercive interventions during emergency psychiatric evaluation. Interdisciplinary systemic changes are urgently needed to address structural barriers to equitable psychiatric care.

Association Between Crowdsourced Health Care Facility Ratings and Mortality in US Counties

IMPORTANCE

Mortality across US counties varies considerably, from 252 to 1847 deaths per 100 000 people in 2018. Although patient satisfaction with health care is associated with patient- and facility-level health outcomes, the association between health care satisfaction and community-level health outcomes is not known.

OBJECTIVE

To examine the association between online ratings of health care facilities and mortality across US counties and to identify language specific to 1-star (lowest rating) and 5-star (highest rating) reviews in counties with high vs low mortality.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective population-based cross-sectional study examined reviews and ratings of 95 120 essential health care facilities across 1301 US counties. Counties that had at least 1 essential health care facility with reviews available on Yelp, an online review platform, were included. Essential health care was defined according to the 10 essential health benefits covered by Affordable Care Act insurance plans.

MAIN OUTCOMES AND MEASURES

The mean rating of essential health care facilities was calculated by county from January 1, 2015, to December 31, 2019. Ratings were on a scale of 1 to 5 stars, with 1 being the worst rating and 5 the best. County-level composite measures of health behaviors, clinical care, social and economic factors, and physical environment were obtained from the University of Wisconsin School of Medicine and Public Health County Health Rankings database. The 2018 age-adjusted mortality by county was obtained from the Centers for Disease Control and Prevention Wide-ranging Online Data for Epidemiological Research database. Multiple linear regression analysis was used to estimate the association between mean facility rating and mortality, adjusting for county health ranking variables. Words with frequencies of use that were significantly different across 1-star and 5-star reviews in counties with high vs low mortality were identified.

RESULTS

The 95 120 facilities meeting inclusion criteria were distributed across 1301 of 3142 US counties (41.4%). At the county level, a 1-point increase in mean rating was associated with a mean (SE) age-adjusted decrease of 18.05 (3.68) deaths per 100 000 people (P < .001). Words specific to 1-star reviews in high-mortality counties included told, rude, and wait, and words specific to 5-star reviews in low-mortality counties included Dr, pain, and professional.

CONCLUSIONS AND RELEVANCE

This study found that, at the county level, higher online ratings of essential health care facilities were associated with lower mortality. Equivalent online ratings did not necessarily reflect equivalent experiences of care across counties with different mortality levels, as evidenced by variations in the frequency of use of key words in reviews. These findings suggest that online ratings and reviews may provide insight into unequal experiences of essential health care.