Health system performance comparison: new directions in research and policy

Health system description and assessment: a scoping review of templates for systematic analyses

BACKGROUND

Understanding and comparing health systems is key for cross-country learning and health system strengthening. Templates help to develop standardised and coherent descriptions and assessments of health systems, which then allow meaningful analyses and comparisons. Our scoping review aims to provide an overview of existing templates, their content and the way data is presented.

MAIN BODY

Based on the WHO building blocks framework, we defined templates as having (1) an overall framework, (2) a list of indicators or topics, and (3) instructions for authors, while covering (4) the design of the health system, (5) an assessment of health system performance, and (6) should cover the entire health system. We conducted a scoping review of grey literature published between 2000 and 2023 to identify templates. The content of the identified templates was screened, analyzed and compared. We found 12 documents that met our inclusion criteria. The building block `health financing´ is covered in all 12 templates; and many templates cover ´service delivery´ and ´health workforce'. Health system performance is frequently assessed with regard to 'access and coverage', 'quality and safety', and 'financial protection'. Most templates do not cover 'responsiveness' and 'efficiency'. Seven templates combine quantitative and qualitative data, three are mostly quantitative, and two are primarily qualitative. Templates cover data and information that is mostly relevant for specific groups of countries, e.g. a particular geographical region, or for high or for low and middle-income countries (LMICs). Templates for LMICs rely more on survey-based indicators than administrative data.

CONCLUSIONS

This is the first scoping review of templates for standardized descriptions of health systems and assessments of their performance. The implications are that (1) templates can help analyze health systems across countries while accounting for context; (2) template-guided analyses of health systems could underpin national health policies, strategies, and plans; (3) organizations developing templates could learn from approaches of other templates; and (4) more research is needed on how to improve templates to better achieve their goals. Our findings provide an overview and help identify the most important aspects and topics to look at when comparing and analyzing health systems, and how data are commonly presented. The templates were created by organizations with different agendas and target audiences, and with different end products in mind. Comprehensive health systems analyses and comparisons require production of quantitative indicators and complementing them with qualitative information to build a holistic picture.

CLINICAL TRIAL REGISTRATION

  Not applicable.

Data for health system comparison and assessment in the African Region: A review of 63 indicators available in international databases

Background

Achieving universal health coverage in the African region requires health systems strengthening. Assessing and comparing health systems contributes to this process, but requires internationally comparable data. The European Observatory on Health Systems and Policies has produced Health Systems in Transition (HiT) reviews in Europe, Asia, North America and the Caribbean with a standardised template. This study explores data availability in international databases for the quantitative health and health system indicators in the HiT template for the WHO African region.

Methods

We identified ten databases which contained data for 40 of the 80 original HiT indicators and an additional 23 proxy indicators to fill some gaps. We then assessed data availability for the resulting 63 indicators by country and time, i.e. first/last year of data, years of data available overall and since 2000, and we explored for each indicator (1) against the country with the greatest availability overall and (2) against annual availability for all years since 2000.

Results

Overall data availability was greatest in South Africa (93.0% of possible total points) and least in South Sudan (59.5%). Since 2000, Uganda (60.4%) has had the highest data availability and South Sudan (37.2%) the lowest. By topic, data availability was the highest for health financing (91.4%; median start/end date 2000/2019) and background characteristics (88.5%; 1990/2020) and was considerably lower for health system performance (54.5%; 2000/2018) and physical and human resources (44.8%; 2004/2013). Data are available for different years in different countries, and at irregular intervals, complicating time series analysis. No data are available for service provision indicators.

Conclusions

Gaps in data in international databases across time, countries, and topics undermine systematic health systems comparisons and assessments, regional health systems strengthening, and efforts to achieve universal health coverage. More efforts are needed to strengthen national data collection and management and integrate national data into international databases to support cross-country assessments, peer learning, and planning. In tandem, more research is needed to understand the specific historical, cultural, administrative, and technological determinants influencing country data availability, as well as the facilitators and barriers of data sharing between countries and international databases, and the potential of new technologies to increase timeliness of data.

Territorial gaps on quality of causes of death statistics over the last forty years in Spain

BACKGROUND

The quality of the statistics on causes of death (CoD) does not present consolidated indicators in literature further than the coding group of ill-defined conditions of the International Classification of Diseases. Our objective was to assess the territorial quality of CoD by reliability of the official mortality statistics in Spain over the years 1980-2019.

METHODS

A descriptive epidemiological design of four decades (1980-, 1990-, 2000-, and 2010-2019) by region (18) and sex was implemented. The CoD cases, age-adjusted rates and ratios (to all-cause) were assigned by reliability to unspecific and ill-defined quality categories. The regional mortality rates were contrasted to the Spanish median by decade and sex by the Comparative Mortality Ratio (CMR) in a Bayesian perspective. Statistical significance was considered when the CMR did not contain the value 1 in the 95% credible intervals.

RESULTS

Unspecific, ill-defined, and all-cause rates by region and sex decreased over 1980-2019, although they scored higher in men than in women. The ratio of ill-defined CoD decreased in both sexes over these decades, but was still prominent in 4 regions. CMR of ill-defined CoD in both sexes exceeded the Spanish median in 3 regions in all decades. In the last decade, women's CMR significantly exceeded in 5 regions for ill-defined and in 6 regions for unspecific CoD, while men's CMR exceeded in 4 and 2 of the 18 regions, respectively on quality categories.

CONCLUSIONS

The quality of mortality statistics of causes of death has increased over the 40 years in Spain in both sexes. Quality gaps still remain mostly in Southern regions. Authorities involved might consider to take action and upgrading regional and national death statistics, and developing a systematic medical post-grade training on death certification.

(I can't get no) satisfaction: A comparative study of healthcare recommodification in Europe, 2010-18

European health reforms during the last decades have strengthened patient rights and introduced choice, competition and financial incentives in a sector that has typically been state-directed and centrally controlled. The marketisation of health care has also drawn out profit and introduced private provision. The main argument behind this trend is that market competition will improve service quality and deliver health services more efficiently. Such reforms often fall under the umbrella of New Public Management (NPM), and there is a lack of empirical research on their effects. The purpose of this paper is to investigate the association between healthcare marketisation and health system outcomes across European nations. In order to measure a country's degree of healthcare marketisation we employed indicators of healthcare decommodification. The concept refers to the extent to which an individual's access to healthcare is dependent upon their market position and the extent to which a country's provision of health is independent from the market. These indicators are three measures that assess the financing, provision and coverage of the private sector, and thus reflects the varied role of the market in a health care system: private health care expenditure as amount of GDP, private hospital beds as amount of total hospital bed stock, and public healthcare coverage. As indicator of health system outcome, we employed a measure that has not previously been investigated in the context of healthcare marketisation: satisfaction with health care system. We used multilevel analyses on five waves (2009-2017) of the biannual European Social Survey (ESS), with our final models including more than 120,000 individuals from 21 countries. Our methodological approach allowed us to study both cross-sectional and longitudinal relationships. The strongest substantial associations were between coverage and satisfaction, with high public healthcare coverage being associated with higher satisfaction.

Governing Personalized Health: A Scoping Review

Genetic research is advancing rapidly. One important area for the application of the results from this work is personalized health. These are treatments and preventive interventions tailored to the genetic profile of specific groups or individuals. The inclusion of personalized health in existing health systems is a challenge for policymakers. In this article, we present the results of a thematic scoping review of the literature dealing with governance and policy of personalized health. Our analysis points to four governance challenges that decisionmakers face against the background of personalized health. First, researchers have highlighted the need to further extend and harmonize existing research infrastructures in order to combine different types of genetic data. Second, decisionmakers face the challenge to create trust in personalized health applications, such as genetic tests. Third, scholars have pointed to the importance of the regulation of data production and sharing to avoid discrimination of disadvantaged groups and to facilitate collaboration. Fourth, researchers have discussed the challenge to integrate personalized health into regulatory-, financing-, and service provision structures of existing health systems. Our findings summarize existing research and help to guide further policymaking and research in the field of personalized health governance.

Access and effectiveness inpatient care indicators and economic crisis: analysis based on the Brazilian Unified Health System data, Brazil and Southern Brazil states, 2009-2018

In the context of crisis and resource constraints, it is reasonable to assume the deteriorated weaknesses of the Unified Health System (SUS), such as regional inequalities, underfinancing, and care quality issues. This study explored the application of easily comprehensible and calculated access and effectiveness indicators that could reflect the hospital network crisis. Five indicators extracted from the Hospital Information System, related to Brazil and states of the Southeastern region, were analyzed in the 2009-2018 period: hospitalizations resulting in death; surgical hospitalizations resulting in death; elective surgeries in the total of surgical hospitalizations; hip prostheses in the senior population; and angioplasties in the population aged 20 years and over. Statistical control charts were used to compare indicators between states, before and from 2014. In Brazil, overall hospital deaths had a slight increase while surgical deaths declined; elective surgeries and hipprosthesis also decreased. In Southeastern Brazil, Rio de Janeiro was the worst performer, especially the decrease of the elective surgeries. The results illustrate the potential of indicators to monitor crisis effects on hospital care.

Worlds of Healthcare: A Healthcare System Typology of OECD Countries

In this paper, we present an extended typology of OECD healthcare systems. Our theoretical framework integrates the comparative-institutional perspective of existing classifications with current ideas from the international health policy research debate. We argue that combining these two perspectives provides a more comprehensive picture of modern healthcare systems and takes the past decade's dynamic of reforms into account. Moreover, this approach makes the typology more beneficial in terms of understanding and explaining cross-national variation in population health and health inequalities. Empirically, we combine indicators on supply, public-private mix, and institutional access regulations from earlier typologies with information on primary care orientation and performance management in prevention and quality of care. The results from a series of cluster analyses indicate that at least five distinct types of healthcare systems can be identified. Moreover, we provide quantitative information on the consistency of cluster membership for individual countries via system types.

Comparison of Methods for Measuring Radiotherapy Utilisation

AIMS

Evidence-based estimates of appropriate rates of radiotherapy utilisation are usually stated as the proportion of cancer patients who should receive radiotherapy at least once in their lifetime. However, the prolonged follow-up required to measure the lifetime radiotherapy rate limits its value in monitoring access to radiotherapy in routine practice. The objectives of this study were to evaluate shorter-term methods for measuring radiotherapy utilisation and to determine how well they predict the lifetime radiotherapy rate.

MATERIALS AND METHODS

The Ontario Cancer Registry provided records of all cases of cancer diagnosed in Ontario between 1984 and 2015. Records of all radiotherapy delivered by Ontario cancer centres were linked to individual cases in the Ontario Cancer Registry. Patients were followed forward for 20 years to determine the relationship between short-term and long-term rates of use of radiotherapy. Radiotherapy utilisation was also estimated by comparing total radiotherapy workload with cancer incidence; these measures were compared with observed long-term radiotherapy rates.

RESULTS

The rate of use of radiotherapy within 1 year of diagnosis (RT_1y) was strongly predictive of the rate of use of radiotherapy after 20 years (RT_20y); for each annual cohort of cases between 1984 and 1995, RT_20y was approximately equal to 1.3 × RT_1y. The number of cases treated for the first time with radiotherapy in a specified period, divided by the number of new cases diagnosed in the same period, was about equal to the proportion of cases treated with radiotherapy within 20 years of diagnosis (RT_20y).

CONCLUSIONS

The lifetime rate of use of radiotherapy may be predicted quite accurately from the rate observed within 1 year of diagnosis, or from the ratio of new cases treated to cancer incidence in a specified period. Either of these measures may therefore be used to audit radiotherapy utilisation against the existing evidence-based targets.

Toward Common Data Elements for International Research in Long-term Care Homes: Advancing Person-Centered Care

To support person-centered, residential long-term care internationally, a consortium of researchers in medicine, nursing, behavioral, and social sciences from 21 geographically and economically diverse countries have launched the WE-THRIVE consortium to develop a common data infrastructure. WE-THRIVE aims to identify measurement domains that are internationally relevant, including in low-, middle-, and high-income countries, prioritize concepts to operationalize domains, and specify a set of data elements to measure concepts that can be used across studies for data sharing and comparisons. This article reports findings from consortium meetings at the 2016 meeting of the Gerontological Society of America and the 2017 meeting of the International Association of Gerontology and Geriatrics, to identify domains and prioritize concepts, following best practices to identify common data elements (CDEs) that were developed through the US National Institutes of Health/National Institute of Nursing Research's CDEs initiative. Four domains were identified, including organizational context, workforce and staffing, person-centered care, and care outcomes. Using a nominal group process, WE-THRIVE prioritized 21 concepts across the 4 domains. Several concepts showed similarity to existing measurement structures, whereas others differed. Conceptual similarity (convergence; eg, concepts in the care outcomes domain of functional level and harm-free care) provides further support of the critical foundational work in LTC measurement endorsed and implemented by regulatory bodies. Different concepts (divergence; eg, concepts in the person-centered care domain of knowing the person and what matters most to the person) highlights current gaps in measurement efforts and is consistent with WE-THRIVE's focus on supporting resilience and thriving for residents, family, and staff. In alignment with the World Health Organization's call for comparative measurement work for health systems change, WE-THRIVE's work to date highlights the benefits of engaging with diverse LTC researchers, including those in low-, middle-, and high-income countries, to develop a measurement infrastructure that integrates the aspirations of person-centered LTC.

Comparative health system performance in six middle-income countries: cross-sectional analysis using World Health Organization study of global ageing and health

Objective To assess and compare health system performance across six middle-income countries that are strengthening their health systems in pursuit of universal health coverage. Design Cross-sectional analysis from the World Health Organization Study on global AGEing and adult health, collected between 2007 and 2010. Setting Six middle-income countries: China, Ghana, India, Mexico, Russia and South Africa. Participants Nationally representative sample of adults aged 50 years and older. Main outcome measures We present achievement against key indicators of health system performance across effectiveness, cost, access, patient-centredness and equity domains. Results We found areas of poor performance in prevention and management of chronic conditions, such as hypertension control and cancer screening coverage. We also found that cost remains a barrier to healthcare access in spite of insurance schemes. Finally, we found evidence of disparities across many indicators, particularly in the effectiveness and patient centredness domains. Conclusions These findings identify important focus areas for action and shared learning as these countries move towards achieving universal health coverage.

Ranked Performance of Canada's Health System on the International Stage: A Scoping Review

INTRODUCTION

Since the release of the World Health Report in 2000, health system performance ranking studies have garnered significant health policy attention. However, this literature has produced variable results. The objective of this study was to synthesize the research and analyze the ranked performance of Canada's health system on the international stage.

METHOD

We conducted a scoping review exploring Canada's place in ranked health system performance among its peer Organisation for Economic Co-operation and Development countries. Arksey and O'Malley's five-stage scoping review framework was adopted, yielding 48 academic and grey literature articles. A literature extraction tool was developed to gather information on themes that emerged from the literature.

SYNTHESIS

Although various methodologies were used to rank health system performance internationally, results generally suggested that Canada has been a middle-of-the-pack performer in overall health system performance for the last 15 years. Canada's overall rankings were 7/191, 11/24, 10/11, 10/17, "Promising" and "B" grade across different studies. According to past literature, Canada performed well in areas of efficiency, productivity, attaining health system goals, years of life lived with disability and stroke mortality. By contrast, Canada performed poorly in areas related to disability-adjusted life expectancy, potential years of life lost, obesity in adults and children, diabetes, female lung cancer and infant mortality.

CONCLUSION

As countries introduce health system reforms aimed at improving the health of populations, international comparisons are useful to inform cross-country learning in health and social policy. While ranking systems do have shortcomings, they can serve to shine a spotlight on Canada's health system strengths and weaknesses to better inform the health policy agenda.

Geriatric Patient Safety Indicators Based on Linked Administrative Health Data to Assess Anticoagulant-Related Thromboembolic and Hemorrhagic Adverse Events in Older Inpatients: A Study Proposal

Background

Frail older people with multiple interacting conditions, polypharmacy, and complex care needs are particularly exposed to health care-related adverse events. Among these, anticoagulant-related thromboembolic and hemorrhagic events are particularly frequent and serious in older inpatients. The growing use of anticoagulants in this population and their substantial risk of toxicity and inefficacy have therefore become an important patient safety and public health concern worldwide. Anticoagulant-related adverse events and the quality of anticoagulation management should thus be routinely assessed to improve patient safety in vulnerable older inpatients.

Objective

This project aims to develop and validate a set of outcome and process indicators based on linked administrative health data (ie, insurance claims data linked to hospital discharge data) assessing older inpatient safety related to anticoagulation in both Switzerland and France, and enabling comparisons across time and among hospitals, health territories, and countries. Geriatric patient safety indicators (GPSIs) will assess anticoagulant-related adverse events. Geriatric quality indicators (GQIs) will evaluate the management of anticoagulants for the prevention and treatment of arterial or venous thromboembolism in older inpatients.

Methods

GPSIs will measure cumulative incidences of thromboembolic and bleeding adverse events based on hospital discharge data linked to insurance claims data. Using linked administrative health data will improve GPSI risk adjustment on patients’ conditions that are present at admission and will capture in-hospital and postdischarge adverse events. GQIs will estimate the proportion of index hospital stays resulting in recommended anticoagulation at discharge and up to various time frames based on the same electronic health data. The GPSI and GQI development and validation process will comprise 6 stages: (1) selection and specification of candidate indicators, (2) definition of administrative data-based algorithms, (3) empirical measurement of indicators using linked administrative health data, (4) validation of indicators, (5) analyses of geographic and temporal variations for reliable and valid indicators, and (6) data visualization.

Results

Study populations will consist of 166,670 Swiss and 5,902,037 French residents aged 65 years and older admitted to an acute care hospital at least once during the 2012-2014 period and insured for at least 1 year before admission and 1 year after discharge. We will extract Swiss data from the Helsana Group data warehouse and French data from the national health insurance information system (SNIIR-AM). The study has been approved by Swiss and French ethics committees and regulatory organizations for data protection.

Conclusions

Validated GPSIs and GQIs should help support and drive quality and safety improvement in older inpatients, inform health care stakeholders, and enable international comparisons. We discuss several limitations relating to the representativeness of study populations, accuracy of administrative health data, methods used for GPSI criterion validity assessment, and potential confounding bias in comparisons based on GQIs, and we address these limitations to strengthen study feasibility and validity.

Health system frameworks and performance indicators in eight countries: A comparative international analysis

OBJECTIVES

Performance indicators are a popular mechanism for measuring the quality of healthcare to facilitate both quality improvement and systems management. Few studies make comparative assessments of different countries' performance indicator frameworks. This study identifies and compares frameworks and performance indicators used in selected Organisation for Economic Co-operation and Development health systems to measure and report on the performance of healthcare organisations and local health systems. Countries involved are Australia, Canada, Denmark, England, the Netherlands, New Zealand, Scotland and the United States.

METHODS

Identification of comparable international indicators and analyses of their characteristics and of their broader national frameworks and contexts were undertaken. Two dimensions of indicators - that they are nationally consistent (used across the country rather than just regionally) and locally relevant (measured and reported publicly at a local level, for example, a health service) - were deemed important.

RESULTS

The most commonly used domains in performance frameworks were safety, effectiveness and access. The search found 401 indicators that fulfilled the 'nationally consistent and locally relevant' criteria. Of these, 45 indicators are reported in more than one country. Cardiovascular, surgery and mental health were the most frequently reported disease groups.

CONCLUSION

These comparative data inform researchers and policymakers internationally when designing health performance frameworks and indicator sets.

Defining a typology of primary care practices: a novel approach

Objective

To define a typology of primary care (PC) practices based on a mixed inductive/deductive approach that uses a large number of variables describing organizational and demographic characteristics of practices and a priori hierarchical structuring of the data.

Design

Secondary analysis of the Swiss part of the QUALICOPC study using a multiple factor analysis approach incorporating 74 variables hierarchically structured and including information on infrastructures, clinical care, workforces, accessibility and geographic location of PC practices.

Setting

Switzerland.

Participants

Two hundred randomly selected PC practices.

Main Outcome Measures

Typology of PC practices based on axes identified through the multiple factorial approach.

Results

The factorial analysis extracted two uncorrelated axes summarizing 17% of the global variance. The first axis is mainly associated with two dimensions related to the comprehensiveness of services, namely 'clinical care provided' (Pearson's r = 0.73) and 'available infrastructures' (r = 0.78). The second axis is mainly associated with the workforce in the practice such as the number of general practitioners or other health workers (r = 0.69). Swiss PC practices were mapped using these two axes.

Conclusions

This innovative approach allows defining a global typology of PC practices. Based upon Swiss data, two axes were identified to globally describe PC organization: comprehensiveness of services and workforces development. This exploratory study demonstrates a promising way, first to characterize globally one or several PC models that emerge from complex features, second to compare more accurately PC organization between countries and finally to assess how these models might be associated with patients' outcomes.

The Challenge of Cardiovascular Diseases and Diabetes to Public Health: A Study Based on Qualitative Systemic Approach

The most common modeling approaches to understanding incidence, prevalence and control of chronic diseases in populations, such as statistical regression models, are limited when it comes to dealing with the complexity of those problems. Those complex adaptive systems have characteristics such as emerging properties, self-organization and feedbacks, which structure the system stability and resistance to changes. Recently, system science approaches have been proposed to deal with the range, complexity, and multifactor nature of those public health problems. In this paper we applied a multilevel systemic approach to create an integrated, coherent, and increasingly precise conceptual framework, capable of aggregating different partial or specialized studies, based on the challenges of the Longitudinal Study of Adult Health - ELSA-Brasil. The failure to control blood pressure found in several of the study's subjects was discussed, based on the proposed model, analyzing different loops, time lags, and feedback that influence this outcome in a population with high educational level, with reasonably good health services access. We were able to identify the internal circularities and cycles that generate the system's resistance to change. We believe that this study can contribute to propose some new possibilities of the research agenda and to the discussion of integrated actions in the field of public health.