Do performance indicators predict regulator ratings of healthcare providers? Cross-sectional study of acute hospitals in England

[Reliability of peer review-like dialogue in the German statutory quality assurance program]

BACKGROUND

Quality measurement in the German statutory program for quality in health care follows a two-step process. For selected areas of health care, quality is measured via performance indicators (first step). Providers failing to achieve benchmarks in these indicators subsequently enter into a peer review process (second step) and are asked by the respective regional authority to provide a written statement regarding their indicator results. The statements are then evaluated by peers, with the goal to assess the provider's quality of care. In the past, similar peer review-based approaches to the measurement of health care quality in other countries have shown a tendency to lack reliability. So far, the reliability of this component of the German statutory program for quality in health care has not been investigated.

METHOD

Using logistic regression models, the influence of the respective regional authority on the peer review component of health care quality measurement in Germany was investigated using three exemplary indicators and data from 2016.

RESULTS

Both the probability that providers are asked to provide a statement as well as the results produced by the peer review process significantly depend on the regional authority in charge. This dependence cannot be fully explained by differences in the indicator results or by differences in case volume.

CONCLUSIONS

The present results are in accordance with earlier findings, which show low reliability for peer review-based approaches to quality measurement. Thus, different results produced by the peer review component of the quality measurement process may in part be due to differences in the way the review process is conducted. This heterogeneity among the regional authorities limits the reliability of this process. In order to increase reliability, the peer review process should be standardized to a higher degree, with clear review criteria, and the peers should undergo comprehensive training for the review process. Alternatively, the future peer review component could be adapted to focus rather on identification of improvement strategies than on reliable provider comparisons.

An overview and visual analysis of research on government regulation in healthcare

Objective

During the period of COVID-19, government regulation (GR) played an important role in healthcare. This study examines the current research situation of GR in healthcare, discusses the research hotspots, the most productive authors and countries, and the most common journals, and analyzes the changes in GR in healthcare before and after the outbreak of COVID-19.

Methods

This study followed PRISMA guidelines to collect literature on GR in healthcare. And the VOSviewer software was used to perform a quantitative analysis of these documents to obtain a visual map, including year, country, institution, journal, author, and research topic.

Results

A total of 1,830 papers that involved 976 academic journals, 3,178 institutions, and 133 countries were identified from 1985 to 2023. The United States was the country with the highest production (n = 613), followed by the United Kingdom (n = 289). The institution with the largest number of publications was the University of London in the UK (n = 103); In the author collaboration network, the biggest cluster is Bomhoff M, Bouwman R, Friele R, et al. The top five journals in terms of the number of articles were BMC Health Services Research (n = 70), Plos One (n = 35), Health Policy (n = 33), Social Science & Medicine (n = 29), Health Policy and Planning (n = 29), and Frontiers in Public Health (n = 27). The existing literature mainly focused on "health policy," "public health," "China," "mental health," "India," "qualitative research," "legislation," and "governance," et al. Since 2020, research on "COVID-19" has also become a priority in the domain of healthcare.

Conclusion

This study reveals the overall performance of the literature on GR published in healthcare. Healthcare needs GR, especially in response to the COVID-19 epidemic, which has played an irreplaceable role. The outbreak of COVID-19 not only tested the health systems of various countries, but also changed GR in healthcare. With the end of COVID-19, whether these changes will end remains to be further studied.

Analysis of Medico-Legal Complaint Data: A Retrospective Study of Three Large Italian University Hospitals

(1) Background: Identifying hospital-related critical, and excellent, areas represents the main goal of this paper, in both a national and local setting. Information was collected and organized for an internal company's reports, regarding civil litigation that has been affecting the hospital, to relate the obtained results with the phenomenon of medical malpractice on a national scale. This is for the development of targeted improvement strategies, and for investing available resources in a proficient way. (2) Methods: In the present study, data from claims management in Umberto I General Hospital, Agostino Gemelli University Hospital Foundation and Campus Bio-Medico University Hospital Foundation, from 2013 to 2020 were collected. A total of 2098 files were examined, and a set of 13 outcome indicators in the assessment of "quality of care" was proposed. (3) Results: From the total number, only 779 records (37.1%) were attributable to the categories indexable for the present analysis. This data highlights how, following a correct and rigorous categorization of hospital events, it is possible to analyze these medico-legal aspects using a small number of indicators. Furthermore, it is important to consider how a consistent percentage of remaining events was difficult to index, and was also of poor scientific interest. (4) Conclusions: The proposed indicators do not require standards to be compared to, but provide a useful instrument for comparative purposes. In fact, in addition to comparative assessment between different business realities distributed throughout the territory, the use of outcome indicators allows for a longitudinal analysis evaluating the performance of an individual structure over time.

The measurement and improvement of maternity service performance through inspection and rating: An observational study of maternity services in acute hospitals in England

•

The performance of maternity departments varies and is a policy concern.

•

Inspection and ratings is a possible means to increase performance.

•

Routinely collected data may help measure the performance of maternity departments.

•

Using routine data, post-inspection performance was not associated with inspection scores.

•

Inspection and rating do not stimulate improvement on the performance measures studied.

Objectives

To determine whether the prior performance of maternity services, as measured by Royal College of Obstetricians and Gynaecologists performance indicators, is associated with ratings by the Care Quality Commission at subsequent inspection, and whether performance changes occur after inspection.

Methods

We used hospital activity data from 176 maternity sites inspected between October 2013 and March 2016 to generate a set of performance indicators developed by the Royal College of Obstetricians and Gynaecologists. We linked these data to Care Quality Commission data on inspection dates and rating scores and used regression models, controlling for site level effects, to estimate the relationships between inspection ratings and performance indicators before and after inspections.

Results

Coefficients measuring the relationship between indicator performance and subsequent inspection rating score had wide confidence intervals which crossed zero suggesting no statistically significant relationship prior to inspection. The same absence of statistical significance was observed for changes in indicator performance after inspection.

Conclusions

The use of routine data for performance monitoring is becoming increasingly important as regular inspection is costly and regulators require accurate and timely intelligence. However, we found no statistically significant relationships between inspection ratings and performance indicators before or after inspections in maternity services. This calls into question the validity and reliability of the performance indicators, the inspection process and ratings, or both, as measures of performance.

Using quality indicators to predict inspection ratings: cross-sectional study of general practices in England

Background

The Care Quality Commission regulates, inspects, and rates general practice providers in England. Inspections are costly and infrequent, and are supplemented by a system of routine quality indicators, measuring patient satisfaction and the management of chronic conditions. These indicators can be used to prioritise or target inspections.

Aim

To determine whether this set of indicators can be used to predict the ratings awarded in subsequent inspections.

Design and setting

This cross-sectional study was conducted using a dataset of 6860 general practice providers in England.

Method

The indicators and first-inspection ratings were used to build ordered logistic regression models to predict inspection outcomes on the four-level rating system (‘outstanding’, ‘good’, ‘requires improvement’, and ‘inadequate’) for domain ratings and the ‘overall’ rating. Predictive accuracy was assessed using the percentage of correct predictions and a measure of agreement (weighted κ).

Results

The model correctly predicted 79.7% of the ‘overall’ practice ratings. However, 78.8% of all practices were rated ‘good’ on ‘overall’, and the weighted κ measure of agreement was very low (0.097); as such, predictions were little more than chance. This lack of predictive power was also found for each of the individual domain ratings.

Conclusion

The poor power of performance of these indicators to predict subsequent inspection ratings may call into question the validity and reliability of the indicators, inspection ratings, or both. A number of changes to the way data relating to performance indicators are collected and used are suggested to improve the predictive value of indicators. It is also recommended that assessments of predictive power be undertaken prospectively when sets of indicators are being designed and selected by regulators.