Perceived barriers to effective implementation of public reporting of hospital performance data in Australia: a qualitative study

The impact of transparency constraints on the efficiency of the Russian healthcare system: systematic literature review

There is an ongoing debate among researchers and policy-makers on how to make transparency a powerful tool of healthcare systems. This study addresses how the availability and accessibility of information about medical services to the general population affects healthcare outcomes in Russia. A systematic review was conducted and reported according to the Preferred Reporting Items for Systematic Reviewing and Meta-Analysis (PRISMA) guidelines. Transparency indicators of health facilities used in the world's most efficient healthcare systems are also reviewed. Although the increase of transparency in the Russian healthcare system is considered as a tool for improving its efficiency, very little has been done to improve the actual level of transparency. The existing institutional specifics of the Russian healthcare system impose serious restrictions on acceptable levels of transparency. In the reviewed empirical Russian studies, transparency is often viewed simplistically as either information available on the websites of medical organizations or issues related to the amount of accessible indicators of compulsory medical statistical reporting. The novelty of this study consists in (a) reviewing the most recent studies on the topic and (b) including studies in Russian in the analysis. We elaborate on general and specific policy implications for improving transparency-driven outcomes in the Russian healthcare system.

Challenges in Implementing the WHO Hospital Readiness Checklist for the COVID-19 Pandemic in Indonesian Hospitals: A Qualitative Study

Purpose

To assist hospitals in preparing for a surge of patients during the COVID-19 pandemic, the World Health Organization (WHO) published an assessment tool called the rapid hospital readiness checklist. This checklist has been used by numerous countries, including Indonesia. However, several technical issues were discovered, primarily as a result of the manual recording of the checklist on a spreadsheet. This study aimed to identify challenges related to the hospital readiness checklist that was implemented in Indonesia.

Materials and Methods

This qualitative study used focus group discussions to collect data. The study was conducted in East Java, Indonesia, in October 2021, with the participation of nine organizations from the provinces of East Java and Bali. Data were thematically analyzed, and the findings were presented in a narrative format.

Results

Hospital participants had experience in filling out a hospital readiness checklist every 3 months. Some challenges faced by the hospital was the need to manually enter data into the checklist, lack of coordination and communication, there were various perceptions in hospitals as there was no technical guide in completing the checklist, absence of feedback, and the data returning empty due to filling errors. Additionally, City X health office also identified challenges in monitoring and evaluating the hospital checklists.

Conclusion

This study, which included both hospitals and the health office, described the challenges encountered in the assessment of the hospital readiness checklist. Both hospitals and DHO experienced some challenges with the current system. The checklist has the potential to evolve into a public reporting to improve efficiency and faster decision-making.

Public disclosure of hospital clinicians' performance data: insights from medical directors

Objective This study gathered information from public hospital chief medical officers to better understand underlying mechanisms through which public reporting affects institutional behavioural change and decision making towards quality improvement. Methods This qualitative study used thematic analysis of 17 semistructured, in-depth interviews among a peak group of medical directors representing 26 health services in Victoria, Australia. Results The medical directors indicated a high level of in-principle support for public reporting of identifiable, individual clinician-level data. However, they also described varying conceptual understanding of what public reporting of performance data is. Overall, they considered public reporting of individual clinicians' performance data a means to improve health care quality, increase transparency and inform consumer healthcare decision making. Most identified caveats that would need to be met before such data should be publicly released, in particular the need to resolve issues around data quality and timeliness, context and interpretation and ethics. Acknowledgement of the public's right to access individual clinician-level data was at odds with some medical directors' belief that such reporting may diminish trust between clinicians and their employers, thus eroding rather than motivating quality improvement. Conclusions Public reporting of identifiable individual healthcare clinicians' performance data is an issue that merits robust research and debate given the effects such reporting may have on doctors and on hospital quality and safety. What is known about the topic? The public reporting of individual clinician-level data is a mechanism used in some countries, but not in Australia, for increasing health care transparency and quality. Clinician-level public reporting of doctors' performance attracts contention and debate in Australia. What does this paper add? This paper informs debate around the public reporting of individual clinician-level performance data. Among a discrete cohort of senior hospital administrators in Victoria, Australia, there was strong in-principle support for such public reporting as a means to improve hospital quality and safety. What are the implications for practitioners? Before public reporting of individual clinician performance data could occur in Australia, resolution of issues would be required relating to legality and ethics, data context and interpretation, data quality and timeliness.

Can publicly reported data be used to understand performance in an Australian rural hospital?

BACKGROUND

Despite agreement among policymakers, funders, consumers and researchers about the value of public reporting of health information, limited attention has been paid to how it can be used to understand the performance of rural hospitals.

OBJECTIVE

To determine whether publicly available information can be used to measure health service performance in a rural hospital.

METHOD

The study used performance data routinely reported for public consumption in Australia. Data across four domains, multiple measures and time periods were collected to examine access and equity; efficiency and sustainability; quality, safety and patient orientation; and employee engagement. Performance of the rural hospital was examined using a visualisation tool.

RESULTS

Visualisation of multiple measures of performance over time was achievable but required a high degree of health information management skills.

CONCLUSION AND IMPLICATIONS

Publicly reported data can be used to represent performance for a rural hospital. Timeliness, level of detail available and peer groupings of data limits optimal utility. Consumers, clinicians and health service managers wanting to understand the performance of rural hospitals will need to use significant health information management skills to gain a picture of performance. Further research in the applied use of publicly available performance data and relevant dashboards for rural hospitals is suggested.

Priority measures for publicly reporting primary care performance: Results of public engagement through deliberative dialogues in 3 Canadian provinces

Objective

While public reporting of hospital‐based performance measurement is commonplace, it has lagged in the primary care sector, especially in Canada. Despite the increasing recognition of patients as active partners in the health‐care system, little is known about what information about primary care performance is relevant to the Canadian public. We explored patient perspectives and priorities for the public reporting of primary care performance measures.

Methods

We conducted six deliberative dialogue sessions across three Canadian provinces (British Columbia, Ontario, Nova Scotia). Participants were asked to rank and discuss the importance of collecting and reporting on specific dimensions and indicators of primary care performance. We conducted a thematic analysis of the data.

Results

Fifty‐six patients participated in the dialogue sessions. Measures of access to primary care providers, communication with providers and continuity of information across all providers involved in a patient's care were identified as the highest priority indicators of primary care performance from a patient perspective. Several common measures of quality of care, such as rates of cancer screening, were viewed as too patient dependent to be used to evaluate the health system or primary care provider's performance.

Conclusions

Our findings suggest that public reporting aimed at patient audiences should focus on a nuanced measure of access, incorporation of context reported alongside measurement that is for public audiences, clear reporting on provider communication and a measure of information continuity. Participants highlighted the importance the public places on their providers staying up to date with advances in care.

A systematic review and meta-synthesis of policy intervention characteristics that influence the implementation of government-directed policy in the hospital setting: implications for infection prevention and control

Background

Government-directed policy plays an important role in the regulation and supervision of healthcare quality. Effective implementation of these policies has the potential to significantly improve clinical practice and patient outcomes, including the prevention of healthcare-associated infections. A systematic review of research describing the implementation of government-directed policy in the hospital setting was performed with the aim to identify policy intervention characteristics that influence implementation.

Methods

A systematic search of four electronic databases was undertaken to identify eligible articles published between 2007 and 2017. Studies were included if published in the English language and described the implementation of government-directed policy in a high-income country hospital setting. Data on policy and implementation were extracted for each article and interpretive syntheses performed.

Results

A total of 925 articles were retrieved and titles and abstracts reviewed, with 69 articles included after review of abstract and full text. Qualitative synthesis of implementation data showed three overarching themes related to intervention characteristics associated with implementation: clarity; infrastructure; and alignment.

Conclusion

Better understanding and consideration of policy intervention characteristics during development and planning will facilitate more effective implementation although research describing implementation of government-directed policy in the hospital setting is limited and of variable quality. The findings of this study provide guidance to staff tasked with the development or implementation of government-directed policy in the hospital setting, infection prevention and control professionals seeking to maximise the impact of policy on practice and improve patient outcomes.

Public needs for information disclosure on healthcare performance: Different determinants between Japan and the Netherlands

The accumulated healthcare performance data related to unwarranted practice variations are not necessarily disseminated to patients and citizens. To clarify the needs for public disclosure, we explored Japanese and Dutch citizens' preferences and values towards information disclosure and healthcare disparity.Online opt-in survey was conducted and we asked citizens their preference to know about the healthcare performance indicators of regions and hospitals, and their attitudes towards healthcare equity. After a descriptive statistical analysis, Chi-squared automatic interaction detection tree analysis was performed to explore the socio-demographic determinants which were associated with positive value for information disclosure and healthcare equity. Then, we compared the combination of attributes of the highest and the lowest subgroups of each country and compared within and between countries. Last, logistic regression analysis was performed to further evaluate the impact of each determinant.Significant differences were observed between the 2 countries (Japan [JPN] 1038; Netherlands [NL] 1040). The crucial attributes identified were age, sex, educational background, and living area (JPN), along with age and sex (NL). Japanese comprised multiple subgroups with heterogeneous values, showed relatively low interest in knowing the information, and seemed to accept healthcare inequality, especially among urban males aged 20 to 59 years. Contrarily, Dutch people mostly showed high interest in both items. Female and older respondents valued information disclosure highly across countries.To share healthcare performance knowledge and empowering the public, historical, cultural, and socio-demographic context including health literacy of citizens' subgroups should be considered in making comprehensive public reports.

Public reporting of hospital performance data: views of senior medical directors in Victoria, Australia

Objective The aim of the present study was to better understand senior medical directors' perceptions of public reporting of hospital performance data, how public reporting affects institutional behavioural change towards quality improvement and how it could be improved. Methods Interviews were undertaken with 17 medical directors representing 26 metropolitan and regional public hospitals in Victoria, Australia, between June and August 2016. Data were analysed thematically. Results Medical directors are well placed to comment on clinical and administrative aspects of quality, safety and performance monitoring in public hospitals. Their responses largely suggested that public reporting of hospital performance data in Australia is immature and not fulfilling its potential. There was little consensus among informants around what public reporting is, who it is for or its purpose. Although public reporting was considered to have important functions for hospitals and consumers, it was generally considered to lack robustness and have underutilised potential to inform consumers, build trust and drive quality and performance improvements within hospitals. Conclusions The next steps needed to advance public reporting of hospital performance data in Australia include engaging clinicians and patients in selection and development of metrics, improving timeliness of reporting, and improving communication of information so that it is accessible and meaningful for different audiences. What is known about the topic? Public reporting of hospital performance data is a mechanism increasingly used in the Australian health system, but it has attracted little research. What does this paper add? This paper reveals a lack of shared understanding among medical directors in Victoria, Australia, on what public reporting of hospital performance data is, who it is for and its purpose. The paper highlights the potential importance of public reporting of hospital performance data for rural and regional healthcare consumers and how it may be strengthened. What are the implications for practitioners? Stronger systems of public reporting of hospital performance data have the potential to increase consumer engagement and improve hospital performance, quality and safety. Awareness of the discourse around public reporting of hospital performance data can increase practitioners' engagement in debate and development of reporting systems.

Hospitals accreditation status in Indonesia: associated with hospital characteristics, market competition intensity, and hospital performance?

Background

Hospital accreditation is widely adopted as a visible measure of an organisation’s quality and safety management standards compliance. There is still inconsistent evidence regarding the influence of hospital accreditation on hospital performance, with limited studies in developing countries. This study aims to explore the association of hospital characteristics and market competition with hospital accreditation status and to investigate whether accreditation status differentiate hospital performance.

Methods

East Java Province, with a total 346 hospitals was selected for this study. Hospital characteristics (size, specialty, ownership) and performance indicator (bed occupancy rate, turnover interval, average length of stay, gross mortality rate, and net mortality rate) were retrieved from national hospital database while hospital accreditation status were recorded based on hospital accreditation report. Market density, Herfindahl-Hirschman index (HHI), and hospitals relative size as competition indicators were calculated based on the provincial statistical report data. Logistic regression, Mann-Whitney U-test, and one sample t-test were used to analyse the data.

Results

A total of 217 (62.7%) hospitals were accredited. Hospital size and ownership were significantly associated with of accreditation status. When compared to government-owned, hospital managed by ministry of defense (B = 1.705, p = 0.012) has higher probability to be accredited. Though not statistically significant, accredited hospitals had higher utility and efficiency indicators, as well as higher mortality.

Conclusions

Hospital with higher size and managed by government have higher probability to be accredited independent to its specialty and the intensity of market competition. Higher utility and mortality in accredited hospitals needs further investigation.

Electronic supplementary material

The online version of this article (10.1186/s12913-019-4187-x) contains supplementary material, which is available to authorized users.

Citizen perspectives on the use of publicly reported primary care performance information: Results from citizen-patient dialogues in three Canadian provinces

OBJECTIVE

Performance measurement and reporting is proliferating in all sectors of the healthcare system, including primary care, despite a dearth of evidence on how the public uses reports on primary care performance. We explored how the public might use this information, to guide the development of effective reporting systems for primary care.

METHODS

We conducted six full-day deliberative dialogue sessions with a purposive sample of 56 citizen-patients across three Canadian provinces (British Columbia, Ontario and Nova Scotia). Participants identified how they would use publicly reported performance data. We conducted a thematic analysis of the data by region.

RESULTS

Common uses for primary care performance information emerged across all sessions. Participants most often discussed the utility of this information for community advocacy and participation in health system decision making. Similar barriers for using performance information to choose a primary care provider were identified in each region including the perceived lack of choice of providers and the high value placed on relationships with current providers. Finally, the value of public performance reporting in enhancing trust that people would receive good care was also a common theme.

CONCLUSIONS

Citizen-patient perspectives highlight that public reporting on primary care performance could promote the health system's responsiveness by enabling public engagement in decision making at the community level. The role of public reporting in promoting trust rather than empowering patient choice may reflect unique elements of the Canadian health system's context.

The impact of Australian healthcare reforms on emergency department time-based process outcomes: An interrupted time series study

Background

In 2011, the Australian government introduced national healthcare reforms aimed at increasing the timeliness and quality of hospital care. The healthcare reforms included, but were not limited to, emergency department (ED) time-based targets, financial incentives, and public performance reporting of hospital data. We sought to evaluate the impact of the national healthcare reforms on ED time-based process outcomes.

Methods

A quasi-experimental study of ED presentations from 2006 to 2016 in the state of Victoria, Australia. Uncontrolled, interrupted time-series analyses were used to evaluate, by hospital peer groups, the effect of national healthcare reforms on: patient wait times for treatment; treatment within recommended time; and patient departure within four hours of arrival in ED.

Results

There were small improvements in ED time-based process outcomes following the introduction of the national healthcare reforms. These occurred in most hospital peer groups immediately and over the longer term, across the various triage categories. The largest improvements occurred in small hospitals and smallest improvements in medium sized hospitals. ED time-based targets, now abolished by the Australian government, were not achieved in any hospital peer groups.

Conclusions

Our findings suggest that national healthcare reforms had the potential to prompt fundamental changes in ED processes leading to significant improvements in ED performances across most hospital peer groups but were generally unable to reach the ED targets imposed nationally. ED performances also varied by hospital peer groups. Attention to ED time-based process outcomes within hospital peer groups may provide insights into hospital practices that could improve the quality and efficiency of ED care.

Public reporting of hospital quality data: What do referring physicians want to know?

OBJECTIVE

To identify ambulatory care physicians' priorities for hospital quality criteria to support them in counselling patients what hospital to choose.

METHODS

Three hundred non-hospital-based stratified randomly sampled physicians, representing the five main referring specialties in Germany participated in a cross-sectional survey. Physicians rated the importance of 80 hospital quality criteria to be used in their counselling of patients in need of hospital care. Criteria selection was based on a literature analysis and the content of Germany's mandatory hospital quality reports. We calculated the most important criteria and performed an ordinal regression analysis to examine whether the physicians' characteristics 'age', 'sex', 'specialty', 'practice type' and 'region' affected physicians' importance ratings.

RESULTS

To counsel patients in need of a hospital referral, physicians preferred hospital quality criteria that reflect their own and their patients' experiences with a hospital. Additionally, hospitals' expertise and results of treatment were rated highly important. In contrast, hospitals' structural characteristics and compliance with external requirements were rated less important. Physicians' characteristics affected importance ratings only negligibly.

CONCLUSIONS

To support referring physicians' counselling of patients regarding what hospital to choose in order to achieve optimal patient outcomes eventually, hospital report cards must be enriched by information on physicians' and their patients' experiences with hospitals. Hospitals' structural characteristics play a minor role in counselling of patients needing hospital care.

Impact of public release of performance data on the behaviour of healthcare consumers and providers

BACKGROUND

It is becoming increasingly common to publish information about the quality and performance of healthcare organisations and individual professionals. However, we do not know how this information is used, or the extent to which such reporting leads to quality improvement by changing the behaviour of healthcare consumers, providers, and purchasers.

OBJECTIVES

To estimate the effects of public release of performance data, from any source, on changing the healthcare utilisation behaviour of healthcare consumers, providers (professionals and organisations), and purchasers of care. In addition, we sought to estimate the effects on healthcare provider performance, patient outcomes, and staff morale.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, and two trials registers on 26 June 2017. We checked reference lists of all included studies to identify additional studies.

SELECTION CRITERIA

We searched for randomised or non-randomised trials, interrupted time series, and controlled before-after studies of the effects of publicly releasing data regarding any aspect of the performance of healthcare organisations or professionals. Each study had to report at least one main outcome related to selecting or changing care.

DATA COLLECTION AND ANALYSIS

Two review authors independently screened studies for eligibility and extracted data. For each study, we extracted data about the target groups (healthcare consumers, healthcare providers, and healthcare purchasers), performance data, main outcomes (choice of healthcare provider, and improvement by means of changes in care), and other outcomes (awareness, attitude, knowledge of performance data, and costs). Given the substantial degree of clinical and methodological heterogeneity between the studies, we presented the findings for each policy in a structured format, but did not undertake a meta-analysis.

MAIN RESULTS

We included 12 studies that analysed data from more than 7570 providers (e.g. professionals and organisations), and a further 3,333,386 clinical encounters (e.g. patient referrals, prescriptions). We included four cluster-randomised trials, one cluster-non-randomised trial, six interrupted time series studies, and one controlled before-after study. Eight studies were undertaken in the USA, and one each in Canada, Korea, China, and The Netherlands. Four studies examined the effect of public release of performance data on consumer healthcare choices, and four on improving quality.There was low-certainty evidence that public release of performance data may make little or no difference to long-term healthcare utilisation by healthcare consumers (3 studies; 18,294 insurance plan beneficiaries), or providers (4 studies; 3,000,000 births, and 67 healthcare providers), or to provider performance (1 study; 82 providers). However, there was also low-certainty evidence to suggest that public release of performance data may slightly improve some patient outcomes (5 studies, 315,092 hospitalisations, and 7502 providers). There was low-certainty evidence from a single study to suggest that public release of performance data may have differential effects on disadvantaged populations. There was no evidence about effects on healthcare utilisation decisions by purchasers, or adverse effects.

AUTHORS' CONCLUSIONS

The existing evidence base is inadequate to directly inform policy and practice. Further studies should consider whether public release of performance data can improve patient outcomes, as well as healthcare processes.

Public performance reporting and hospital choice: a cross-sectional study of patients undergoing cancer surgery in the Australian private healthcare sector

OBJECTIVES

National mandatory public performance reporting (PPR) for Australian public hospitals, including measures of cancer surgery waiting times, was introduced in 2011. PPR is voluntary for private hospitals. The aims of this study were to assess whether PPR of hospital data is used by patients with breast, bowel or lung cancer when selecting a hospital for elective surgery and how PPR could be improved to meet their information needs.

DESIGN

A national cross-sectional postal questionnaire.

SETTING

Australian private healthcare sector.

PARTICIPANTS

Private patients with breast, bowel or lung cancer who attended a public or private hospital for elective surgery (n=243) in 2016.

OUTCOME MEASURES

Patients' choice of hospital, use of PPR information and preferred areas of PPR information. Descriptive and conventional qualitative content analyses were conducted.

RESULTS

Two hundred and twenty-eight respondents (94%) attended a private hospital. Almost half could choose a hospital. Choice of hospital was not influenced by PPR data (92% unaware) but by their specialist (90%). Respondents considered PPR to be important (71%) but they did not want to see the information, preferring their general practitioners (GPs) to tell them about it (40%). Respondents considered surgery costs (59%), complications (58%) and recovery success rates (57%) to be important areas of information that should be publicly reported. Almost half suggested that quality indicators should be reported at the individual clinician level. Analysis of the open-ended questions identified four themes: (1) decision-making factors; (2) data credibility; (3) unmet information needs and (4) unintended consequences.

CONCLUSIONS

PPR of hospital data had no substantial impact on patients' choice of hospital. Nonetheless, many respondents expressed interest in using it in future. To increase PPR awareness and usability, personalised and integrated information on cost and quality of hospitals is required. Dissemination of PPR information via specialists and GPs could assist patients to interpret the data and support decision-making.

The use of public performance reporting by general practitioners: a study of perceptions and referral behaviours

Background

Public performance reporting (PPR) of hospital data aims to improve quality of care in hospitals and to inform consumer choice. In Australia, general practitioners (GPs) are gatekeepers to secondary care with patients requiring their referral for non-emergency access. Despite their intermediary role, GPs have been generally overlooked as potential users of PPR of hospital data, with the majority of the PPR research focussing on consumers, surgeons and hospitals.

Methods

We examined the use of PPR of hospital data by GPs when referring patients to hospitals. Semi-structured interviews were conducted with 40 GPs, recruited via the Victorian Primary Care Practice-Based Research Network and GP teaching practices in Victoria, Australia. The interviews were recorded, transcribed and analysed thematically.

Results

We found that the majority of GPs did not use PPR when referring patients to hospitals. Instead, they relied mostly on informal sources of information such as their own or patients’ previous experiences. Barriers that prevented GPs’ use of PPR in their decision making included: lack of awareness and accessibility; perceived lack of data credibility; restrictive geographical catchments for certain hospitals; limited choices of public hospitals in regional and rural areas; and no mandatory PPR for private hospitals.

Conclusions

Our findings suggest that lack of PPR awareness prevented GPs from using it in their referral practice. As gatekeepers to secondary care, GPs are in a position to guide patients in their treatment decisions and referrals using available PPR data. We suggest that there needs to be greater involvement by GPs in the development of hospital performance and quality indicators in Australia if GPs are to make greater use of them. The indicators require further development before GPs perceive them as valid, credible, and of use for informing their referral practices.

Electronic supplementary material

The online version of this article (10.1186/s12875-018-0719-4) contains supplementary material, which is available to authorized users.

Medical directors’ perspectives on strengthening hospital quality and safety

Purpose

The purpose of this paper is to understand the concerns and factors that impact on hospital quality and safety, particularly related to use of performance data, within a setting of devolved governance.

Design/methodology/approach

This qualitative study used thematic analysis of interviews with public hospital medical directors. For additional context, findings were framed by themes from a review of hospital safety and quality in the same jurisdiction.

Findings

Varying approaches and levels of complexity were described about what and how performance data are reviewed, prioritised, and quality improvements implemented. Although no consistent narrative emerged, facilitators of improvement were suggested relating to organisational culture, governance, resources, education, and technologies. These hospital-level perspectives articulate with and expand on the system-level themes in a state-wide review of hospital safety and quality.

Research limitations/implications

The findings are not generalisable, but point to an underlying absence of system-wide agreement on how to perceive, retrieve, analyse, prioritise and action hospital performance data.

Practical implications

Lack of electronic medical records and an inefficient incident reporting system limits the extent to which performance and incident data can be analysed, linked and shared, thus limiting hospital performance improvement, oversight and learning.

Social implications

Variable approaches to quality and safety, standards of care, and hospital record keeping and reporting, mean that healthcare consumers might expect inconsistency across Victorian hospitals.

Originality/value

The views of medical directors have been little researched. This work uses their voice to better understand contextual factors that situate and impact on hospital quality and safety towards understanding the mixed effectiveness of hospital quality improvement strategies.