Assessing health system performance in developing countries: a review of the literature

Achieving universal health coverage in small island states: could importing health services provide a solution?

Background

Universal health coverage (UHC) is difficult to achieve in settings short of medicines, health workers and health facilities. These characteristics define the majority of the small island developing states (SIDS), where population size negates the benefits of economies of scale. One option to alleviate this constraint is to import health services, rather than focus on domestic production. This paper provides empirical analysis of the potential impact of this option.

Methods

Analysis was based on publicly accessible data for 14 SIDS, covering health-related travel and health indicators for the period 2003–2013, together with in-depth review of medical travel schemes for the two highest importing SIDS—the Maldives and Tuvalu.

Findings

Medical travel from SIDS is accelerating. The SIDS studied generally lacked health infrastructure and technologies, and the majority of them had lower than the recommended number of physicians in a country, which limits their capacity for achieving UHC. Tuvalu and the Maldives were the highest importers of healthcare and notably have public schemes that facilitate medical travel and help lower the out-of-pocket expenditure on medical travel. Although different in approach, design and performance, the medical travel schemes in Tuvalu and the Maldives are both examples of measures used to increase access to health services that cannot feasibly be provided in SIDS.

Interpretation

Our findings suggest that importing health services (through schemes to facilitate medical travel) is a potential mechanism to help achieve universal healthcare for SIDS but requires due diligence over cost, equity and quality control.

Health-related quality of life in elderly diabetic outpatients in Vietnam

BACKGROUND

Health-related quality of life (HRQoL) is an important indicator for designing care and treatment services for patients with diabetes. This is especially true given its rapid increase among the elderly population in Vietnam. HRQoL data in elderly diabetic Vietnamese are currently limited. This study aimed to 1) measure the HRQoL of elderly patients with type 2 diabetes (T2DM) in Vietnam and 2) identify related factors and their relationship with HRQoL.

PATIENTS AND METHODS

A cross-sectional study was conducted. We recruited 171 patients aged ≥60 years with T2DM at the Outpatient Department, National Geriatric Hospital from June to November 2015. Patients were asked to evaluate their health status using the EuroQol Five Dimensions Three Levels (EQ-5D-3L) and the Visual Analog Scale (VAS). Sociodemographic, diabetic treatment, and management characteristics were collected. Multivariate Tobit regression was used to determine which factors were associated with HRQoL, and the strength of this relationship.

RESULTS

Patients reported some problems in all areas of the EQ-5D: pain/discomfort (50.9%), mobility (33.3%), anxiety/depression (24.0%), usual activities (21.1%), and self-care (10.5%). The mean EQ-5D index score was 0.80 (SD=0.20), and the mean EQ-VAS was 57.5 (SD=14.4). Patients who were male, lived in an urban area, could afford treatment, were taking fewer medications, and monitored blood pressure often (1-4 times a week) had a higher EQ-5D index when compared to other groups. Meanwhile, a longer duration of diabetes and older age were negatively associated with the EQ-5D index. Patients with any comorbidity had lower VAS scores than their counterparts.

CONCLUSION

The presence of diabetes and comorbidity were responsible for a significant decrease in HRQoL. Screening and identifying health problems, providing prompt treatment, and facilitating self-management among patients have the potential to increase diabetic patients' HRQoL.

Measuring health systems strength and its impact: experiences from the African Health Initiative

Background

Health systems are essential platforms for accessible, quality health services, and population health improvements. Global health initiatives have dramatically increased health resources; however, funding to strengthen health systems has not increased commensurately, partially due to concerns about health system complexity and evidence gaps demonstrating health outcome improvements. In 2009, the African Health Initiative of the Doris Duke Charitable Foundation began supporting Population Health Implementation and Training Partnership projects in five sub-Saharan African countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia) to catalyze significant advances in strengthening health systems. This manuscript reflects on the experience of establishing an evaluation framework to measure health systems strength, and associate measures with health outcomes, as part of this Initiative.

Methods

Using the World Health Organization’s health systems building block framework, the Partnerships present novel approaches to measure health systems building blocks and summarize data across and within building blocks to facilitate analytic procedures. Three Partnerships developed summary measures spanning the building blocks using principal component analysis (Ghana and Tanzania) or the balanced scorecard (Zambia). Other Partnerships developed summary measures to simplify multiple indicators within individual building blocks, including health information systems (Mozambique), and service delivery (Rwanda). At the end of the project intervention period, one to two key informants from each Partnership’s leadership team were asked to list – in rank order – the importance of the six building blocks in relation to their intervention.

Results

Though there were differences across Partnerships, service delivery and information systems were reported to be the most common focus of interventions, followed by health workforce and leadership and governance. Medical products, vaccines and technologies, and health financing, were the building blocks reported to be of lower focus.

Conclusion

The African Health Initiative experience furthers the science of evaluation for health systems strengthening, highlighting areas for further methodological development – including the development of valid, feasible measures sensitive to interventions in multiple contexts (particularly in leadership and governance) and describing interactions across building blocks; in developing summary statistics to facilitate testing intervention effects on health systems and associations with health status; and designing appropriate analytic models for complex, multi-level open health systems.

Socioeconomic inequalities of outpatient and inpatient service utilization in China: personal and regional perspectives

BACKGROUND

China's health system has shown remarkable progress in health provision and health outcomes in recent decades, however inequality in health care utilization persists and poses a serious social problem. While government pro-poor health policies addressed affordability as the major obstacle to equality in health care access, this policy direction deserves further examination. Our study examines the issue of health care inequalities in China, analyzing both regional and individual socioeconomic factors associated with the inequality, and provides evidence to improve governmental health policies.

METHODS

The China Health and Nutrition Survey (CHNS) 1991-2011 data were used to analyze the inequality of health care utilization. The random effects logistic regression technique was used to model health care utilization as the dependent variable, and income and regional location as the independent variables, controlling for individuals' age, gender, marital status, education, health insurance, body mass index (BMI), and period variations. The dynamic trend of 1991-2011 regional disparities was estimated using an interaction term between the regional group dummy and the wave dummy.

RESULTS

The probability of using outpatient service and inpatient services during the previous 4 weeks was 8.6 and 1.1% respectively. Compared to urban residents, suburban (OR: 0.802, 95% CI: 0.720-0.893), town (OR: 0.722, 95% CI: 0.648-0.804), rich (OR: 0.728, 95% CI: 0.656-0.807) and poor village (OR: 0.778, 95% CI: 0.698-0.868) residents were less likely to use outpatient service; and rich (OR: 0.609, 95% CI: 0.472-0.785) and poor village (OR: 0.752, 95% CI: 0. 576-0.983) residents were less likely to use inpatient health care. But the differences between income groups were not significant, except the differences between top and bottom income group in outpatient service use.

CONCLUSION

Regional location was a more important factor than individual characteristics in determining access to health care. Besides demand-side subsidies, Chinese policy makers should pay enhanced attention to health care resource allocation to address inequity in health care access.

INTEGRATE-HTA: A LOW- AND MIDDLE-INCOME COUNTRY PERSPECTIVE

OBJECTIVES

The INTEGRATE-HTA project recommends that complexity be taken into account when conducting health technology assessments (HTAs) and suggests a five-step process for doing that. This study examines whether the approach suggested by INTEGRATE-HTA could be useful, appropriate, and feasible in the context of low- and middle-income countries (LMIC) given some of the typical challenges that healthcare systems face in those countries.

METHODS

A nonexhaustive literature review was performed on the implementation in low and middle income countries of the five aspects recommended by the INTEGRATE-HTA project, using the following search terms: national health planning, health sector strategy, health sector performance, assessment criteria, health (management) information, complexity, context, stakeholder consultation.

RESULTS

HTA is being practiced in LMIC in various ways and through different mechanisms, for example in health sector reviews, even though it is usually not referred to as HTA. It does not necessarily follow the five steps distinguished in the INTEGRATE-HTA model (scoping; defining the initial logic model; providing concepts and methods to identify, collect, and synthesize evidence in relation to various dimensions; extracting and presenting evidence in respect of agreed assessment criteria; providing guidance to draw conclusions and formulate recommendations).

CONCLUSIONS

The conditions for functional HTA are not always fulfilled in LMICs. At least four aspects would require special attention: (a) the scope and quality of routine health information that can support and be fed into health technology assessments and strategic planning; (b) consensus on health system performance assessment frameworks and their main criteria, in particular the inclusion of social disparities/equity and sustainability;

Quality of life profile of general Vietnamese population using EQ-5D-5L

BACKGROUND

Health-related quality of life (HRQOL) is a vital benchmark to assess the effects of health interventions and policies. Measuring HRQOL of the general population is essential to establish a reference for health outcomes evaluations. However, evidence on HRQOL of general populations in low and middle income countries is very limited. This study aimed to measure HRQOL of the Vietnamese population by using the EuroQol-5 dimensions-5 levels (EQ-5D-5L) instrument and determine its associated factors.

METHODS

A cross-sectional study was performed in Hanoi with 1571 residences in Hanoi, the capital city of Vietnam. EQ-5D-5L and EQ- visual analogue scale (EQ-VAS) were used to assess HRQOL. Potential covariates included socio-demographic characteristics, having acute symptoms in the last four weeks, chronic diseases in the last three months, having multiple health issues, and health service utilisation in the last twelve months. A generalized linear model was employed to identify the association between HRQOL and covariates.

RESULTS

Overall, the mean EQ-5D utility index was 0.91 (SD = 0.15), and the mean EQ-VAS score was 87.4 (SD = 14.3). The highest proportion of respondents reporting any problems was in Usual activities (24.3%), followed by Anxiety/Depression (15.2%) and Pain/Discomfort (10.0%), while the lowest percentage was in Self-care (2.5%). Lower HRQOL composite scores were related to unemployment, lower income, higher education, living in urban areas, having chronic diseases, having multiple health issues and using health service. For any health problem self-reported by respondents, the health utility reduced by 0.02 (respiratory diseases) to 0.15 (musculoskeletal diseases).

CONCLUSIONS

Health utility of the general population and reductions for self-reported health problems in this study are useful for future population health evaluations and comparisons. It also informs the development of interventions to reduce health problems of the general population.

Quality of life profile of general Vietnamese population using EQ-5D-5L

BACKGROUND

Health-related quality of life (HRQOL) is a vital benchmark to assess the effects of health interventions and policies. Measuring HRQOL of the general population is essential to establish a reference for health outcomes evaluations. However, evidence on HRQOL of general populations in low and middle income countries is very limited. This study aimed to measure HRQOL of the Vietnamese population by using the EuroQol-5 dimensions-5 levels (EQ-5D-5L) instrument and determine its associated factors.

METHODS

A cross-sectional study was performed in Hanoi with 1571 residences in Hanoi, the capital city of Vietnam. EQ-5D-5L and EQ- visual analogue scale (EQ-VAS) were used to assess HRQOL. Potential covariates included socio-demographic characteristics, having acute symptoms in the last four weeks, chronic diseases in the last three months, having multiple health issues, and health service utilisation in the last twelve months. A generalized linear model was employed to identify the association between HRQOL and covariates.

RESULTS

Overall, the mean EQ-5D utility index was 0.91 (SD = 0.15), and the mean EQ-VAS score was 87.4 (SD = 14.3). The highest proportion of respondents reporting any problems was in Usual activities (24.3%), followed by Anxiety/Depression (15.2%) and Pain/Discomfort (10.0%), while the lowest percentage was in Self-care (2.5%). Lower HRQOL composite scores were related to unemployment, lower income, higher education, living in urban areas, having chronic diseases, having multiple health issues and using health service. For any health problem self-reported by respondents, the health utility reduced by 0.02 (respiratory diseases) to 0.15 (musculoskeletal diseases).

CONCLUSIONS

Health utility of the general population and reductions for self-reported health problems in this study are useful for future population health evaluations and comparisons. It also informs the development of interventions to reduce health problems of the general population.

Introducing leprosy post-exposure prophylaxis into the health systems of India, Nepal and Indonesia: a case study

Background

Leprosy has a wide range of clinical and socio-economic consequences. India, Indonesia and Nepal contribute significantly to the global leprosy burden. After integration, the health systems are pivotal in leprosy service delivery. The Leprosy Post Exposure Prophylaxis (LPEP) program is ongoing to investigate the feasibility of providing single dose rifampicin (SDR) as post-exposure prophylaxis (PEP) to the contacts of leprosy cases in various health systems. We aim to compare national leprosy control programs, and adapted LPEP strategies in India, Nepal and Indonesia. The purpose is to establish a baseline of the health system’s situation and document the subsequent adjustment of LPEP, which will provide the context for interpreting the LPEP results in future.

Methods

The study followed the multiple-case study design with single units of analysis. The data collection methods were direct observation, in-depth interviews and desk review. The study was divided into two phases, i.e. review of national leprosy programs and description of the LPEP program. The comparative analysis was performed using the WHO health system frameworks (2007).

Results

In all countries leprosy services including contact tracing is integrated into the health systems. The LPEP program is fully integrated into the established national leprosy programs, with SDR and increased documentation, which need major additions to standard procedures. PEP administration was widely perceived as well manageable, but the additional LPEP data collection was reported to increase workload in the first year.

Conclusions

The findings of our study led to the recommendation that field-based leprosy research programs should keep health systems in focus. The national leprosy programs are diverse in terms of organizational hierarchy, human resource quantity and capacity. We conclude that PEP can be integrated into different health systems without major structural and personal changes, but provisions are necessary for the additional monitoring requirements.

Electronic supplementary material

The online version of this article (10.1186/s12913-017-2611-7) contains supplementary material, which is available to authorized users.

Organizational level indicators to address health equity work in local public health agencies: A scoping review

OBJECTIVE

To determine what organizational level indicators exist that could be used by local Ontario public health agencies to monitor and guide their progress in addressing health equity.

METHOD

This scoping review employed Arksey and O'Malley's (2005) six-stage framework. Multiple online databases and grey literature sources were searched using a comprehensive strategy. Studies were included if they described or used indicators to assess an organization's health equity activity. Abstracted indicator descriptions were classified using the roles for public health action identified by the Canadian National Collaborating Centre for Determinants of Health (NCCDH). Health equity experts participated in a consultation phase to examine items extracted from the literature.

SYNTHESIS

Eighteen peer-reviewed studies and 30 grey literature reports were included. Abstracted indicators were considered for 1) relevance for organizational assessment, 2) ability to highlight equity-seeking populations, and 3) potential feasibility for application. Twenty-eight items formed the basis for consultation with 13 selected health equity experts. Items considered for retention were all noted to require significant clarification, definition and development. Those eliminated were often redundant or not an organizational level indicator.

CONCLUSION

Few evidence-based, validated indicators to monitor and guide progress to address health inequities at the level of the local public health organization were identified. There is a need for continued development of identified indicator items, including careful operationalization of concepts and establishing clear definitions for key terms.

Modeling the Impact of Short-Term and Long-Term Determinants of European Health Systems’ Performance: A Panel Data Approach

The purpose of this paper is to provide empirical evidence to help health systems’ policy makers within European countries in making the most appropriate decisions to maintain and improve the health of the people they serve. For this purpose, we have analyzed secondary data provided by the annual reports of the Euro Health Consumer Index, which ranks European countries in terms of their health systems’ ability to serve the needs of healthcare consumers. We consider both the short-run and long-run approach in determining the influence of Patient rights and information, Accessibility, Prevention, Range and reach of services, and Pharmaceuticals on Outcomes. By means of a panel data analysis, we capture the influences not only in a single point of time, but in a time span of five years (2012–2016). The main findings suggest that pharmaceuticals and range and reach of services are positively associated with improving immediate outcomes, while for sustainable results, efforts should be directed to prevention programs and means of accessibility improvement.

Effects of a social accountability approach, CARE's Community Score Card, on reproductive health-related outcomes in Malawi: A cluster-randomized controlled evaluation

BACKGROUND

Social accountability approaches, which emphasize mutual responsibility and accountability by community members, health care workers, and local health officials for improving health outcomes in the community, are increasingly being employed in low-resource settings. We evaluated the effects of a social accountability approach, CARE's Community Score Card (CSC), on reproductive health outcomes in Ntcheu district, Malawi using a cluster-randomized control design.

METHODS

We matched 10 pairs of communities, randomly assigning one from each pair to intervention and control arms. We conducted two independent cross-sectional surveys of women who had given birth in the last 12 months, at baseline and at two years post-baseline. Using difference-in-difference (DiD) and local average treatment effect (LATE) estimates, we evaluated the effects on outcomes including modern contraceptive use, antenatal and postnatal care service utilization, and service satisfaction. We also evaluated changes in indicators developed by community members and service providers in the intervention areas.

RESULTS

DiD analyses showed significantly greater improvements in the proportion of women receiving a home visit during pregnancy (B = 0.20, P < .01), receiving a postnatal visit (B = 0.06, P = .01), and overall service satisfaction (B = 0.16, P < .001) in intervention compared to control areas. LATE analyses estimated significant effects of the CSC intervention on home visits by health workers (114% higher in intervention compared to control) (B = 1.14, P < .001) and current use of modern contraceptives (57% higher) (B = 0.57, P < .01). All 13 community- and provider-developed indicators improved, with 6 of them showing significant improvements.

CONCLUSIONS

By facilitating the relationship between community members, health service providers, and local government officials, the CSC contributed to important improvements in reproductive health-related outcomes. Further, the CSC builds mutual accountability, and ensures that solutions to problems are locally-relevant, locally-supported and feasible to implement.

Seasonal variation in geographical access to maternal health services in regions of southern Mozambique

BACKGROUND

Geographic proximity to health facilities is a known determinant of access to maternal care. Methods of quantifying geographical access to care have largely ignored the impact of precipitation and flooding. Further, travel has largely been imagined as unimodal where one transport mode is used for entire journeys to seek care. This study proposes a new approach for modeling potential spatio-temporal access by evaluating the impact of precipitation and floods on access to maternal health services using multiple transport modes, in southern Mozambique.

METHODS

A facility assessment was used to classify 56 health centres. GPS coordinates of the health facilities were acquired from the Ministry of Health while roads were digitized and classified from high-resolution satellite images. Data on the geographic distribution of populations of women of reproductive age, pregnancies and births within the preceding 12 months, and transport options available to pregnant women were collected from a household census. Daily precipitation and flood data were used to model the impact of severe weather on access for a 17-month timeline. Travel times to the nearest health facilities were calculated using the closest facility tool in ArcGIS software.

RESULTS

Forty-six and 87 percent of pregnant women lived within a 1-h of the nearest primary care centre using walking or public transport modes respectively. The populations within these catchments dropped by 9 and 5% respectively at the peak of the wet season. For journeys that would have commenced with walking to primary facilities, 64% of women lived within 2 h of life-saving care, while for those that began journeys with public transport, the same 2-hour catchment would have contained 95% of the women population. The population of women within two hours of life-saving care dropped by 9% for secondary facilities and 18% for tertiary facilities during the wet season.

CONCLUSIONS

Seasonal variation in access to maternal care should not be imagined through a dichotomous and static lens of wet and dry seasons, as access continually fluctuates in both. This new approach for modelling spatio-temporal access allows for the GIS output to be utilized not only for health services planning, but also to aid near real time community-level delivery of maternal health services.

What can we learn on public accountability from non-health disciplines: a meta-narrative review

OBJECTIVE

In health, accountability has since long been acknowledged as a central issue, but it remains an elusive concept. The literature on accountability spans various disciplines and research traditions, with differing interpretations. There has been little transfer of ideas and concepts from other disciplines to public health and global health. In the frame of a study of accountability of (international) non-governmental organisations in local health systems, we carried out a meta-narrative review to address this gap. Our research questions were: (1) What are the main approaches to accountability in the selected research traditions? (2) How is accountability defined? (3) Which current accountability approaches are relevant for the organisation and regulation of local health systems and its multiple actors?

SETTING

The search covered peer-reviewed journals, monographs and readers published between 1992 and 2012 from political science, public administration, organisational sociology, ethics and development studies. 34 papers were selected and analysed.

RESULTS

Our review confirms the wide range of approaches to the conceptualisation of accountability. The definition of accountability used by the authors allows the categorisation of these approaches into four groups: the institutionalist, rights-based, individual choice and collective action group. These four approaches can be considered to be complementary.

CONCLUSIONS

We argue that in order to effectively achieve public accountability, accountability strategies are to be complementary and synergistic.

Health status and health service utilization in remote and mountainous areas in Vietnam

BACKGROUND

Self-rated health status and healthcare services utilization are important indicators to evaluate the performance of health system. In disadvantaged areas, however, little is known about the access and outcomes of health care services. This study aimed to assess health-related quality of life (HRQOL), health status and healthcare access and utilization of residents in mountainous and remote areas in Vietnam.

METHODS

A cross-sectional study was conducted in a convenient sample of residents in two provinces of Vietnam. Information about socio-economic, health status, HRQOL, healthcare seeking and services utilization were interviewed. EuroQol - 5 Dimensions - 5 Levels (EQ-5D-5 L) was used to measure HRQOL.

RESULTS

Of 200 respondents, mean age was 44.9 (SD = 13.9), 38.0 % were male. One third reported having any problem in Mobility, Usual activities, Pain or Discomfort, Anxiety or Depression. Women tended to suffer more problems in Pain/Discomfort and Anxiety/Depression and lower overall HRQOL than men. Over 90 % of respondents reported at least one health problem. Flu, cold and headache were the most commonly reported symptoms (41.5 %). Most of people preferred community health center when they had illness (96.0 %). Only 18.5 % people used traditional healers with the average of 5.8 times per year. Ethnicity, households' expenditure, illness and morbidity status, difficulty in accessing health care services were related to HRQOL.; Meanwhile, socioeconomic status, health problems, quality of services, and distances were associated with access to healthcare and traditional medicine services.

CONCLUSIONS

Residents in difficult-to-reach areas had high prevalence of health problems and experienced social and structural barriers of healthcare services access. It is necessary to improve the availability and quality of healthcare and traditional medicine services to improve the health status of disadvantaged people.

A comparison of hierarchical cluster analysis and league table rankings as methods for analysis and presentation of district health system performance data in Uganda

In 2003, the Uganda Ministry of Health introduced the district league table for district health system performance assessment. The league table presents district performance against a number of input, process and output indicators and a composite index to rank districts. This study explores the use of hierarchical cluster analysis for analysing and presenting district health systems performance data and compares this approach with the use of the league table in Uganda. Ministry of Health and district plans and reports, and published documents were used to provide information on the development and utilization of the Uganda district league table. Quantitative data were accessed from the Ministry of Health databases. Statistical analysis using SPSS version 20 and hierarchical cluster analysis, utilizing Wards' method was used. The hierarchical cluster analysis was conducted on the basis of seven clusters determined for each year from 2003 to 2010, ranging from a cluster of good through moderate-to-poor performers. The characteristics and membership of clusters varied from year to year and were determined by the identity and magnitude of performance of the individual variables. Criticisms of the league table include: perceived unfairness, as it did not take into consideration district peculiarities; and being oversummarized and not adequately informative. Clustering organizes the many data points into clusters of similar entities according to an agreed set of indicators and can provide the beginning point for identifying factors behind the observed performance of districts. Although league table ranking emphasize summation and external control, clustering has the potential to encourage a formative, learning approach. More research is required to shed more light on factors behind observed performance of the different clusters. Other countries especially low-income countries that share many similarities with Uganda can learn from these experiences.

Use of Balanced Scorecard Methodology for Performance Measurement of the Health Extension Program in Ethiopia

In 2004, Ethiopia introduced a community-based Health Extension Program to deliver basic and essential health services. We developed a comprehensive performance scoring methodology to assess the performance of the program. A balanced scorecard with six domains and 32 indicators was developed. Data collected from 1,014 service providers, 433 health facilities, and 10,068 community members sampled from 298 villages were used to generate weighted national, regional, and agroecological zone scores for each indicator. The national median indicator scores ranged from 37% to 98% with poor performance in commodity availability, workforce motivation, referral linkage, infection prevention, and quality of care. Indicator scores showed significant difference by region (P < 0.001). Regional performance varied across indicators suggesting that each region had specific areas of strength and deficiency, with Tigray and the Southern Nations, Nationalities and Peoples Region being the best performers while the mainly pastoral regions of Gambela, Afar, and Benishangul-Gumuz were the worst. The findings of this study suggest the need for strategies aimed at improving specific elements of the program and its performance in specific regions to achieve quality and equitable health services.

Development and use of a research productivity assessment tool for clinicians in low-resource settings in the Pacific Islands: a Delphi study

Background

Research performance assessments have proliferated, but research indicators for use amongst clinicians in poorly resourced countries have been ill-defined. The aims of the present paper were to determine a set of indicators as determined by clinician participants from the Pacific Islands and a panel of research experts for use in the performance assessment of clinicians.

Methods

Two focus group discussions, one for nurses and one for doctors, were used to obtain the views of 28 Pacific Island clinicians of the BRRACAP Study about what the research indicators should be. A modified Delphi survey was used to obtain a consensus amongst 19 research experts, with Pacific Island research experience, as to what the indicators should be and then to rank these in terms of importance. A survey of the participants obtained data on the research tasks/actions performed 20 months after the initial research workshop. A resultant tool comprising of 21 indicators was used to assess the performance of 18 Pacific participants.

Results

The Pacific Island clinicians determined that research was important and that performance should be measured. They identified research indicators that could be used in their settings and ranked their importance using a points system. The panel of experts identified implementation of research findings, collaborations and actual change in practice as more important, with bibliometric measurements low down in the scale. Although only 64 % of the 28 BRRACAP Study participants returned the questionnaire, 39 % of those performed more than half of the 21 indicators used. Of the 18 Pacific clinicians assessed, 7 (39 %) performed 10 or more tasks.

Conclusions

A research performance assessment tool was developed using process and output indicators identified by Pacific clinicians and a panel of research experts. The tool, which placed emphasis on process and outputs that were not bibliometric based, proved useful in assessing the performance of Pacific clinicians working in a low-resource setting.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-016-0077-4) contains supplementary material, which is available to authorized users.

Satisfaction and associated factors of outpatient psychiatric service consumers in Ethiopia

PURPOSE

The purpose of this study was to assess the level of patient satisfaction and associated factors with psychiatric outpatient services in Ethiopia.

PATIENTS AND METHODS

A cross-sectional study was performed from May 2015 to June 2015. A total of 454 participants selected by systematic random sampling were included in this study. Pretested and interviewer-administered questionnaire was used to collect the data. Patient satisfaction was measured using Charleston Psychiatric Outpatient Satisfaction Scale, and other validated tools were used to assess the associated variables. Multivariate logistic regressions with 95% confidence interval (CI) were used to assess the strength, and P-value <0.05 was used to indicate significance of association.

RESULTS

A total of 441 respondents were enrolled, with a response rate of 97.1% and magnitude of satisfaction of 61.2%. Being male (adjusted odds ratio [AOR] =0.612, 95% CI: 0.39, 0.94), being widowed (AOR =0.13, 95% CI: 0.05, 0.36), urban residence (AOR =0.49, 95% CI: 0.31, 0.78), diagnosed with schizophrenia (AOR =0.48, 95% CI: 0.28, 0.81), unfavorable attitude (AOR =0.49, 95% CI: 0.28, 0.86), and poor social functioning (AOR =0.52, 95% CI: 0.34, 0.80) were significantly associated with satisfaction.

CONCLUSION

More than one-third of psychiatric service consumers were dissatisfied with the service they received. Integrating patients to their own treatment plan and regular service evaluation are important to improve satisfaction.

Emergency department quality and safety indicators in resource-limited settings: an environmental survey

Background

As global emergency care grows, practical and effective performance measures are needed to ensure high quality care. Our objective was to systematically catalog and classify metrics that have been used to measure the quality of emergency care in resource-limited settings.

Methods

We searched MEDLINE, Embase, CINAHL, and the gray literature using standardized terms. The references of included articles were also reviewed. Two researchers screened titles and abstracts for relevance; full text was then reviewed by three researchers. A structured data extraction tool was used to identify and classify metrics into one of six Institute of Medicine (IOM) quality domains (safe, timely, efficient, effective, equitable, patient-centered) and one of three of Donabedian’s structure/process/outcome categories. A fourth expert reviewer blinded to the initial classifications re-classified all indicators, with a weighted kappa of 0.89.

Results

A total of 1705 articles were screened, 95 received full text review, and 34 met inclusion criteria. One hundred eighty unique metrics were identified, predominantly process (57 %) and structure measures (27 %); 16 % of metrics were related to outcomes. Most metrics evaluated the effectiveness (52 %) and timeliness (28 %) of care, with few addressing the patient centeredness (11 %), safety (4 %), resource-efficiency (3 %), or equitability (1 %) of care.

Conclusions

The published quality metrics in emergency care in resource-limited settings primarily focus on the effectiveness and timeliness of care. As global emergency care is built and strengthened, outcome-based measures and those focused on the safety, efficiency, and equitability of care need to be developed and studied to improve quality of care and resource utilization.

Electronic supplementary material

The online version of this article (doi:10.1186/s12245-015-0088-x) contains supplementary material, which is available to authorized users.

The Astronomy of Africa's Health Systems Literature During the MDG Era: Where Are the Systems Clusters?

Growing international concern about the need for improved health systems in Africa has catalyzed an expansion of the health systems literature. This review applies a bibliometric procedure to analyze the acceleration of scientific writing on this theme. We focus on research published during the Millennium Development Goal (MDG) era between 1990 and 2014, reporting findings from a systematic review of a database comprised of 17,655 articles about health systems themes from sub-Saharan African countries or subregions. Using bibliometric tools for co-word textual analysis, we analyzed the incidence and associations of keywords and phrases to generate and visualize topical foci on health systems as clusters of themes, much in the manner that astronomers represent groupings of stars as galaxies of celestial entities. The association of keywords defines their relative position, with the size of images weighted by the relative frequency of terms. Sets of associated keywords are arrayed as stars that cluster as "galaxies" of concepts in the knowledge universe represented by health systems research from sub-Saharan Africa. Results show that health systems research is dominated by literature on diseases and categorical systems research topics, rather than on systems science that cuts across diseases or specific systemic themes. Systems research is highly developed in South Africa but relatively uncommon elsewhere in the region. "Black holes" are identified by searching for terms in our keyword library related to terms in widely cited reviews of health systems. Results identify several themes that are unexpectedly uncommon in the country-specific health systems literature. This includes research on the processes of achieving systems change, the health impact of systems strengthening, processes that explain the systems determinants of health outcomes, or systematic study of organizational dysfunction and ways to improve system performance. Research quantifying the relationship of governance indicators to health systems strengthening is nearly absent from the literature. Long-term experimental studies and statistically rigorous research on cross-cutting themes of health systems strengthening are rare. Studies of organizational malaise or corruption are virtually absent. Trend analysis shows the emergence of organizational research on specific priority diseases, such as on HIV/AIDS, malaria, and tuberculosis, but portrays a lack of focus on integrated systems research on the general burden of disease. If health systems in Africa are to be strengthened, then organizational change research must be a more concerted focus in the future than has been the case in the past.

A national system for monitoring the performance of hospitals in Ethiopia

Many countries struggle to develop and implement strategies to monitor hospitals nationally. The challenge is particularly acute in low-income countries where resources for measurement and reporting are scarce. We examined the experience of developing and implementing a national system for monitoring the performance of 130 government hospitals in Ethiopia. Using participatory observation, we found that the monitoring system resulted in more consistent hospital reporting of performance data to regional health bureaus and the federal government, increased transparency about hospital performance and the development of multiple quality-improvement projects. The development and implementation of the system, which required technical and political investment and support, would not have been possible without strong hospital-level management capacity. Thorough assessment of the health sector’s readiness to change and desire to prioritize hospital quality can be helpful in the early stages of design and implementation. This assessment may include interviews with key informants, collection of data about health facilities and human resources and discussion with academic partners. Aligning partners and donors with the government’s vision for quality improvement can enhance acceptability and political support. Such alignment can enable resources to be focused strategically towards one national effort – rather than be diluted across dozens of potentially competing projects. Initial stages benefit from having modest goals and the flexibility for continuous modification and improvement, through active engagement with all stakeholders.

Linking Health System Responsiveness to Political Rights and Civil Liberties

Responsiveness is a dimension of health system functioning and might be dependent upon contextual factors related to politics. Given this, we performed cross-national comparisons with the aim of investigating: 1) the associations of political factors with patients’ reports of health system responsiveness and 2) the extent to which health input and output might explain these associations. World Health Survey data were analyzed for 44 countries (n = 103 541). Main outcomes included, respectively, 8 and 7 responsiveness domains for inpatient and outpatient care. Linear multilevel regressions were used to assess the associations of politics (namely, civil liberties and political rights), socioeconomic development, health system input, and health system output (measured by maternal mortality) with responsiveness domains, adjusted for demographic factors. Political rights showed positive associations with dignity (regression coefficient = 0.086 [standard error = 0.039]), quality (0.092 [0.049]), and support (0.113 [0.048]) for inpatient care and with dignity (0.075 [0.040]), confidentiality (0.089 [0.043]), and quality (0.124 [0.053]) for outpatient care. Positive associations were observed for civil liberties as well. Health system input and output reduced observed associations. Results tentatively suggest that strengthening political rights and, to a certain extent, civil liberties might improve health system responsiveness, in part through their effect on health system input and output.

A conceptual framework for evaluation of public health and primary care system performance in iran

INTRODUCTION

The main objective of this study was to design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial public health and primary care performance and to assess their share in the overall health impacts of the community.

METHODS

We used several tools and techniques, including system thinking, literature review to identify relevant attributes of health system performance framework and interview with the key stakeholders. The PubMed, Scopus, web of science, Google Scholar and two specialized databases of Persian language literature (IranMedex and SID) were searched using main terms and keywords. Following decision-making and collective agreement among the different stakeholders, 51 core indicators were chosen from among 602 obtained indicators in a four stage process, for monitoring and evaluation of Health Deputies.

RESULTS

We proposed a conceptual framework by identifying the performance area for Health Deputies between other determinants of health, as well as introducing a chain of results, for performance, consisting of Input, Process, Output and Outcome indicators. We also proposed 5 dimensions for measuring the performance of Health Deputies, consisting of efficiency, effectiveness, equity, access and improvement of health status.

CONCLUSION

The proposed Conceptual Framework illustrates clearly the Health Deputies success in achieving best results and consequences of health in the country. Having the relative commitment of the ministry of health and Health Deputies at the University of Medical Sciences is essential for full implementation of this framework and providing the annual performance report.

Patients satisfaction in an academic walk-in centre: a new model of residents training achieved by family doctors

Background

Walk-in centres may improve access to healthcare for some patients, due to their convenient location and extensive opening hours, with no need for an appointment. Herein, we describe and assess a new model of walk-in centre, characterised by care provided by residents and supervision achieved by experienced family doctors. The main aim of the study was to assess patients’ satisfaction about the care they received from residents and their supervision by family doctors. The secondary aim was to describe walk-in patients’ demographic characteristics and to identify potential associations with satisfaction.

Methods

The study was conducted in the walk-in centre of Lausanne. Patients who consulted between 11th and 31st April were automatically included and received a questionnaire in French. We used a five-point Likert scale, ranging from “not at all satisfied” to “very satisfied”, converted from values of 1 to 5. We focused on the satisfaction regarding residents’ care and supervision by a family doctor. The former was divided in three categories: “Skills”, “Treatment” and “Behaviour”. A mean satisfaction score was calculated for each category and a multivariable logistic model was applied in order to identify associations with patients’ demographics.

Results

The overall response rate was 47% [184/395]. Walk-in patients were more likely to be women (62%), young (median age 31), with a high education level (40% of University degree or equivalent). Patients were “very satisfied” with residents’ care, with a median satisfaction score between 4.5 and 5, for each category. Over 90% of patients were “satisfied” or “very satisfied” that a family doctor was involved in the consultation. Age showed the greatest association with satisfaction.

Conclusion

Patients were highly satisfied with care provided by residents and with the involvement of a family doctor in the consultation. Older age showed the greatest positive association with satisfaction with a positive impact. The high level satisfaction reported by walk-in patients supports this new model of walk-in centre.

Electronic supplementary material

The online version of this article (doi:10.1186/1756-0500-7-874) contains supplementary material, which is available to authorized users.

Evaluating the use of locally-based health facility assessments in Afghanistan: a pilot study of a novel research method

BACKGROUND

Through the Balanced Scorecard program there have been independent, annual and nationwide assessments of the Afghan health system from 2004 to 2013. During this period, Afghanistan remained in a dynamic state of conflict, requiring innovative approaches to health service evaluation in insecure areas. The primary objective of this pilot study was to evaluate the reliability of health facility assessments conducted by a novel, locally-based data collection method compared to a standard survey team.

METHODS

In this cross-sectional study, one standard survey team of clinicians and multiple rapidly trained locally-based survey teams of teachers conducted health facility assessments in Badghis province, Afghanistan from March - August, 2010. Outpatient facilities covered under the country's Basic Package of Health Services were eligible for inclusion. Both approaches attempted to survey as many health facilities as safely possible, up to 25 total facilities per method. Each facility assessed was scored on 23 health services indicators used to evaluate performance in the annual Balanced Scorecard national assessment. For facilities assessed by both survey methods, the indicator scores produced by each method were compared using Spearman's correlation coefficients and linear regression analysis with generalized estimating equations.

RESULTS

The standard survey team was able to assess 11 facilities; the locally-based approach was able to assess these 11 facilities, as well as 13 additional facilities in areas of greater insecurity. Among the 11 facilities assessed by both approaches, 19 of 23 indicators were statistically similar by survey method (p < .05). Spearman's coefficients varied widely from (-0.39) to (0.71). The differences were greatest for items requiring specialized data collector knowledge on reviewing patient records, patient examination and counseling, and health worker reported satisfaction.

CONCLUSIONS

This pilot study of a novel method of data collection in health facility assessments showed that an approach using locally-based survey teams provided markedly increased access to areas of insecurity. Though analysis was limited by small sample size, indicator scores used for facility evaluation were relatively comparable overall, but less reliable for items requiring clinical knowledge or when asking health worker opinions, suggesting that alternative approaches may be needed to assess these parameters in insecure environments.

Examining the need assessment process by identifying the need profiles of elderly care recipients in the Ten-year Long-Term Care Project (TLTCP) of Taiwan

OBJECTIVE

To deal with the increasing long-term care (LTC) needs of elderly people in Taiwan, the government launched the Ten-year Long-term Care Project (TLTCP) in 2007, and through the care management system, care plans for those in need were distributed and implemented by care managers according to the single assessment process. Based on the emphasis of linking the right need assessment to the care plan, this study aimed to explore the need profiles of LTC recipients with regard to their health indicators to serve as a validity check on the identified dependency levels and care plans in the current care management system.

DESIGN

A model based on latent class analysis (LCA) was used for dealing with the issue of health heterogeneity. LCA provides an empirical method that examines the interrelationships among health indicators and characterizes the underlying set of mutually exclusive latent classes that account for the observed indicators. The analysis included a total of 2901 elderly care recipients in the LTC dataset from a southern city, 1 of the 5 major metropolitan areas in Taiwan. The identified dependency levels of the samples and their care plans in need assessment were compared and discussed.

RESULTS

Four need profiles were explored in the LTC dataset. Apart from the low (LD) (32.95%) and moderate dependent groups (MD) (17.48%), there were 2 groups identified among the high-dependency levels, including the severe physical and psychological dependency (SPP) (26.37%) and the comorbidities and severe dependency (CSD) groups (23.20%), which in sum were approximately identified as high dependency (HD) by care managers in the LTC dataset. In addition, the CSD group currently costs more for their care plans on average in LTC services (NT. 277,081.15, approximately 9200 USD) than the SPP group (NT. 244,084.21) and the other groups.

CONCLUSION

Need assessment is a key to success in care management in LTC. The results of this study showed the importance of focusing on multifacet indicators, especially the mental and social health indicators in need assessments by improving the unified assessment process to sensitively detect those with various needs and then link them to the right care plan.

Patients' level of satisfaction on quality of health care at Mwananyamala hospital in Dar es Salaam, Tanzania

BACKGROUND

Enhancing quality of health care delivered in public health facilities in developing countries is a key prerequisite to increase utilization and sustainability of health care services in the population. The aim of the study was to determine patients' level of satisfaction on the quality of health care delivered at the out-patient department (OPD) in Mwananyamala hospital in Dar es Salaam, Tanzania.

METHODS

A cross-sectional study design was conducted from April to May, 2012. A systematic sampling method was employed to select 422 study subjects. A pre-tested SERVQUAL questionnaire was used to collect data and one-sample t-test was employed to identify patients' level of satisfaction and principal component analysis to identify key items that measure quality of care.

RESULTS

Patients' level of satisfaction mean gap score was (-2.88 ± 3.1) indicating overall dissatisfaction with the quality of care. The level of dissatisfaction in the five service dimensions were as follows: assurance (-0.47), reliability (-0.49), tangible (-0.52), empathy (-0.55), and responsiveness (-0.72).

CONCLUSION

Patients attending OPD at Mwananyamala hospital demonstrates an overall dissatisfaction on quality of care. Hospital management should focus on: improvement on communication skills among OPD staff in showing compassion, politeness and active listening, ensure availability of essential drugs, and improvement on clinicians' prescription skills.

Association between coverage of maternal and child health interventions, and under-5 mortality: a repeated cross-sectional analysis of 35 sub-Saharan African countries

BACKGROUND

Infant and child mortality rates are among the most important indicators of child health, nutrition, implementation of key survival interventions, and the overall social and economic development of a population. In this paper, we investigate the role of coverage of maternal and child health (MNCH) interventions in contributing to declines in child mortality in sub-Saharan Africa.

DESIGN

Data are from 81 Demographic and Health Surveys from 35 sub-Saharan African countries. Using ecological time-series and child-level regression models, we estimated the effect of MNCH interventions (summarized by the percent composite coverage index, or CCI) on child mortality with in the first 5 years of life net of temporal trends and covariates at the household, maternal, and child levels.

RESULTS

At the ecologic level, a unit increase in standardized CCI was associated with a reduction in under-5 child mortality rate (U5MR) of 29.0 per 1,000 (95% CI: -43.2, -14.7) after adjustment for survey period effects and country-level per capita gross domestic product (pcGDP). At the child level, a unit increase in standardized CCI was associated with an odds ratio of 0.86 for child mortality (95% CI: 0.82-0.90) after adjustment for survey period effect, country-level pcGDP, and a set of household-, maternal-, and child-level covariates.

CONCLUSIONS

MNCH interventions are important in reducing U5MR, while the effects of economic growth in sub-Saharan Africa remain weak and inconsistent. Improved coverage of proven life-saving interventions will likely contribute to further reductions in U5MR in sub-Saharan Africa.

Building on the EGIPPS performance assessment: the multipolar framework as a heuristic to tackle the complexity of performance of public service oriented health care organisations

Background

Performance of health care systems is a key concern of policy makers and health service managers all over the world. It is also a major challenge, given its multidimensional nature that easily leads to conceptual and methodological confusion. This is reflected by a scarcity of models that comprehensively analyse health system performance.

Discussion

In health, one of the most comprehensive performance frameworks was developed by the team of Leggat and Sicotte. Their framework integrates 4 key organisational functions (goal attainment, production, adaptation to the environment, and values and culture) and the tensions between these functions.

We modified this framework to better fit the assessment of the performance of health organisations in the public service domain and propose an analytical strategy that takes it into the social complexity of health organisations. The resulting multipolar performance framework (MPF) is a meta-framework that facilitates the analysis of the relations and interactions between the multiple actors that influence the performance of health organisations.

Summary

Using the MPF in a dynamic reiterative mode not only helps managers to identify the bottlenecks that hamper performance, but also the unintended effects and feedback loops that emerge. Similarly, it helps policymakers and programme managers at central level to better anticipate the potential results and side effects of and required conditions for health policies and programmes and to steer their implementation accordingly.

Prioritizing the Determinants of Social-health Inequality in Iran: A Multiple Attribute Decision Making Application

Background:

One of the main challenges of healthcare systems of developing countries is health inequality. Health inequality means inequality in individuals’ ability and proper functioning, resulting in inequality in social status and living conditions, which thwarts social interventions implemented by the government.

Objectives:

This study aimed to determine and prioritize the social determinants of health inequality in Iran.

Materials and Methods:

This was a mixed method study with two phases of qualitative and quantitative research. The study population consisted of experts dealing with social determinants of health. A purposive, stratified and non-random sampling method was used. Semi-structured interviews were conducted to collect qualitative data along with a multiple attribute decision making method for the quantitative phase of the research in which the TOPSIS technique was employed for prioritization. The qualitative findings were entered into NVivo for analysis, as were the quantitative data entered into MATLAB software.

Results:

The results approved the suitability of the conceptual framework of social determinants of health suggested by the WHO (world health organization) for studying social determinants of health inequality; however, this framework general and theoretical rather than a guideline for practice. Thus, in this study, 15 themes and 31 sub-themes were determined as social determinants of social health inequality in Iran. Based on the findings of the quantitative phase of our research, socioeconomic status, living facilities such as housing, and social integrity had the greatest effect on decreasing health inequality.

Conclusions:

A major part of the inequality in health distribution is avoidable because they are mostly caused by adjustable factors like economic conditions, educational conditions, employment, living facilities, etc. As in the majority of developing countries the living and health conditions are the same as Iran, the findings of this study may be applicable for other developing countries.

A system dynamics model for national drug policy

BACKGROUND

Data modeling techniques can create a virtual world to analyze decision systems. National drug authorities can use such techniques to take care of their deficiencies in decision making processes. This study was designed to build a system dynamics model to simulate the effects of market mix variables (5 P's) on the national drug policy (NDP) indicators including availability, affordability, quality, and rationality. This was aimed to investigate how to increase the rationality of decision making, evaluate different alternatives, reduce the costs and identify the system obstacles.System dynamics is a computer-based approach for analyzing and designing complex systems over time. In this study the cognitive casualty map was developed to make a concept about the system then the stock-flow model was set up based on the market demand and supply concept.

RESULTS

The model demonstrates the interdependencies between the NDP variables through four cognitive maps. Some issues in availability, willingness to pay, rational use and quality of medicines are pointed in the model. The stock-flow diagram shows how the demand for a medicine is formed and how it is responded through NDP objectives. The effects of changing variables on the other NDP variables can be studied after running the stock-flow model.

CONCLUSION

The model can initiate a fundamental structure for analyzing NDP. The conceptual model made a cognitive map to show many causes' and effects' trees and reveals some relations between NDP variables that are usually forgotten in the medicines affairs. The model also provides an opportunity to be expanded with more details on a specific disease for better policy making about medication.

Health systems performance assessment in low-income countries: learning from international experiences

BACKGROUND

The study aimed at developing a set of attributes for a 'good' health system performance assessment (HSPA) framework from literature and experiences in different contexts and using the attributes for a structured approach to lesson learning for low-income countries (LICs).

METHODS

Literature review to identify relevant attributes for a HSPA framework; attribute validation for LICs in general, and for Uganda in particular, via a high-level Ugandan expert group; and, finally, review of a selection of existing HSPA frameworks using these attributes.

RESULTS

Literature review yielded six key attributes for a HSPA framework: an inclusive development process; its embedding in the health system's conceptual model; its relation to the prevailing policy and organizational set-up and societal context; the presence of a concrete purpose, constitutive dimensions and indicators; an adequate institutional set-up; and, its capacity to provide mechanisms for eliciting change in the health system. The expert group contextualized these attributes and added one on the adaptability of the framework.Lessons learnt from the review of a selection of HSPA frameworks using the attributes include: it is possible and beneficial to involve a range of stakeholders during the process of development of a framework; it is important to make HSPA frameworks explicit; policy context can be effectively reflected in the framework; there are marked differences between the structure and content of frameworks in high-income countries, and low- and middle-income countries; champions can contribute to put HSPA high on the agenda; and mechanisms for eliciting change in the health system should be developed alongside the framework.

CONCLUSION

It is possible for LICs to learn from literature and the experience of HSPA in other contexts, including HICs. In this study a structured approach to lesson learning included the development of a list of attributes for a 'good' HSPA framework. The attributes thus derived can be utilized by LICs like Uganda seeking to develop/adjust their HSPA frameworks as guidelines or a check list, while taking due consideration of the specific context. The review of frameworks from varied contexts, highlighted varied experiences which provide lessons for LICs.

Strengthening primary health care in low- and middle-income countries: generating evidence through evaluation

Since the publication of the World Health Report 2008, there has been renewed interest in the potential of primary health care (PHC) to deliver global health policy agendas. The WHO Western Pacific Regional Strategy 2010 states that health systems in low- and middle-income countries (LMICs) can be strengthened using PHC values as core principles. This review article explores the development of an evidence-based approach for assessing the effectiveness of PHC programs and interventions in LMICs. A realist review method was used to investigate whether there is any internationally consistent approach to evaluating PHC. Studies from LMICs using an explicit methodology or framework for measuring PHC effectiveness were collated. Databases of published articles were searched, and a review of gray literature was undertaken to identify relevant reports. The review found no consistent approach for assessing the effectiveness of PHC interventions in LMICs. An innovative approach used in China, which developed a set of core community health facility indicators based on stakeholder input, does show some potential for use in other LMIC contexts.

Assessing health system performance: a model-based approach

It is difficult to assess countries' relative success in addressing issues of public health because countries are subject to very different background conditions. To address this problem we offer a model-based approach for assessing health system performance. Specifically, an index of public health is regressed against a vector of variables intended to capture economic, educational, cultural, geographic, and epidemiological endowments. The residual from this model is regarded as a plausible measure of public health performance at the national level. We argue that a model-based approach to performance is informative for policymakers and academics as it focuses attention on those aspects of a country's health profile that are not constrained by structural factors. This sharpens comparisons across countries and through time, and also allows one to evaluate the degree to which health systems have lived up to their potential.

Interventions for hiring, retaining and training district health systems managers in low- and middle-income countries

BACKGROUND

District managers are playing an increasingly important role in determining the performance of health systems in low- and middle-income countries as a result of decentralization.

OBJECTIVES

To assess the effectiveness of interventions to hire, retain and train district health systems managers in low- and middle-income countries.

SEARCH METHODS

We searched a wide range of international databases, including the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE and EMBASE. We also searched online resources of international agencies, including the World Bank, to find relevant grey literature. Searches were conducted in December 2011.

SELECTION CRITERIA

District health systems managers are those persons who are responsible for overseeing the operations of the health system within a defined, subnational geographical area that is designated as a district. Hiring and retention interventions include those that aim to increase the attractiveness of district management positions, as well as those related to hiring and retention processes, such as private contracting. Training interventions include education programs to develop future managers and on-the-job training programs for current managers. To be included, studies needed to use one of the following study designs: randomized controlled trial, nonrandomized controlled trial, controlled before-and-after study, and interrupted time series analysis.

DATA COLLECTION AND ANALYSIS

We report measures of effect in the same way that the primary study authors have reported them. Due to the varied nature of interventions included in this review we could not pool data across studies.

MAIN RESULTS

Two studies met our inclusion criteria. The findings of one study conducted in Cambodia provide low quality evidence that private contracts with international nongovernmental organizations (NGOs) for district health systems management ('contracting-in') may improve health care access and utilization. Contracting-in increased use of antenatal care by 28% and use of public facilities by 14%. However, contracting-in was not found to have an effect on population health outcomes. The findings of the other study provide low quality evidence that intermittent training courses over 18 months may improve district health system managers' performance. In three countries in Latin America, managers who did not receive the intermittent training courses had between 2.4 and 8.3 times more management deficiencies than managers who received the training courses. No studies that aimed to investigate interventions for retaining district health systems managers met our study selection criteria for inclusion in this review.

AUTHORS' CONCLUSIONS

There is low quality evidence that contracting-in may improve health care accessibility and utilization and that intermittent training courses may improve district health systems managers' performance. More evidence is required before firm conclusions can be drawn regarding the effectiveness of these interventions in diverse settings. Other interventions that might be promising candidates for hiring and retaining (e.g., government regulations, professional support programs) as well as training district health systems managers (e.g., in-service workshops with on-site support) have not been adequately investigated.

The impact of clinical social franchising on health services in low- and middle-income countries: a systematic review

BACKGROUND

The private sector plays a large role in health services delivery in low- and middle-income countries; yet significant gaps remain in the quality and accessibility of private sector services. Clinical social franchising, which applies the commercial franchising model to achieve social goals and improve health care, is increasingly used in developing countries to respond to these limitations. Despite the growth of this approach, limited evidence documents the effect of social franchising on improving health care quality and access.

OBJECTIVES AND METHODS

We examined peer-reviewed and grey literature to evaluate the effect of social franchising on health care quality, equity, cost-effectiveness, and health outcomes. We included all studies of clinical social franchise programs located in low- and middle-income countries. We assessed study bias using the WHO-Johns Hopkins Rigour Scale and used narrative synthesis to evaluate the findings.

RESULTS

Of 113 identified articles, 23 were included in this review; these evaluated a small sample of franchises globally and focused on reproductive health franchises. Results varied widely across outcomes and programs. Social franchising was positively associated with increased client volume and client satisfaction. The findings on health care utilization and health impact were mixed; some studies find that franchises significantly outperform other models of health care, while others show franchises are equivalent to or worse than other private or public clinics. In two areas, cost-effectiveness and equity, social franchises were generally found to have poorer outcomes.

CONCLUSIONS

Our review indicates that social franchising may strengthen some elements of private sector health care. However, gaps in the evidence remain. Additional research should include: further documentation of the effect of social franchising, evaluating the equity and cost-effectiveness of this intervention, and assessing the role of franchising within the context of the greater healthcare delivery system.

Towards a performance measurement system for health equity in a local health integration network

While there is a growing literature on building performance measurement systems for health equities, this literature for the most part has not dealt with the challenges of coordinating the various parts of the system, the heterogeneous nature of such systems, or how evaluations and measurement can themselves improve performance. This paper describes the initial steps taken to build a performance measurement system to coordinate health equity across 18 hospitals led by the Toronto Central Local Health Integration Network, which is a regional health authority serving a population of more than 2.5 million residents (near in population to Chicago and Rome) and the most socially diverse urban network in Ontario, Canada. This paper also describes some principles that can help inform a performance measurement system. The innovative aspect of this paper is that these principles were developed through feedback by the hospitals.

Health systems frameworks in their political context: framing divergent agendas

BACKGROUND

Despite the mounting attention for health systems and health systems theories, there is a persisting lack of consensus on their conceptualisation and strengthening. This paper contributes to structuring the debate, presenting landmarks in the development of health systems thinking against the backdrop of the policy context and its dominant actors. We argue that frameworks on health systems are products of their time, emerging from specific discourses. They are purposive, not neutrally descriptive, and are shaped by the agendas of their authors.

DISCUSSION

The evolution of thinking over time does not reflect a progressive accumulation of insights. Instead, theories and frameworks seem to develop in reaction to one another, partly in line with prevailing paradigms and partly as a response to the very different needs of their developers. The reform perspective considering health systems as projects to be engineered is fundamentally different from the organic view that considers a health system as a mirror of society. The co-existence of health systems and disease-focused approaches indicates that different frameworks are complementary but not synthetic. The contestation of theories and methods for health systems relates almost exclusively to low income countries. At the global level, health system strengthening is largely narrowed down to its instrumental dimension, whereby well-targeted and specific interventions are supposed to strengthen health services and systems or, more selectively, specific core functions essential to programmes. This is in contrast to a broader conceptualization of health systems as social institutions.

SUMMARY

Health systems theories and frameworks frame health, health systems and policies in particular political and public health paradigms. While there is a clear trend to try to understand the complexity of and dynamic relationships between elements of health systems, there is also a demand to provide frameworks that distinguish between health system interventions, and that allow mapping with a view of analysing their returns. The choice for a particular health system model to guide discussions and work should fit the purpose. The understanding of the underlying rationale of a chosen model facilitates an open dialogue about purpose and strategy.

Health systems and health equity: the challenge of the decade

For decades, concerns regarding health systems have been prominent in the global health discourse, leading to numerous publications and laudable declarations, as well as compacts and consensus statements intended to guide policy and practice. This discussion paper is intended to neither summarise nor systematically review this vast field of interest. Instead, the paper reflects on some challenges for attaining health systems equity and raises questions related to the contributions of both national and the global health systems to this mission.

The emergence of global attention to health systems strengthening

After a period of proliferation of disease-specific initiatives, over the past decade and especially since 2005 many organizations involved in global health have come to direct attention and resources to the issue of health systems strengthening. We explore how and why such attention emerged. A qualitative methodology, process-tracing, was used to construct a case history and analyse the factors shaping and inhibiting global political attention for health systems strengthening. We find that the critical factors behind the recent burst of attention include fears among global health actors that health systems problems threaten the achievement of the health-related Millennium Development Goals, concern about the adverse effects of global health initiatives on national health systems, and the realization among global health initiatives that weak health systems present bottlenecks to the achievement of their organizational objectives. While a variety of actors now embrace health systems strengthening, they do not constitute a cohesive policy community. Moreover, the concept of health systems strengthening remains vague and there is a weak evidence base for informing policies and programmes for strengthening health systems. There are several reasons to question the sustainability of the agenda. Among these are the global financial crisis, the history of pendulum swings in global health and the instrumental embrace of the issue by some actors.

Changes in clients' care ratings after HIV prevention training of hospital workers in Malawi

OBJECTIVE

To examine the changes in clients' health-care ratings before and after hospital workers received an HIV prevention intervention in Malawi, which increased the workers' personal and work-related HIV prevention knowledge, attitudes and preventive behaviors.

DESIGN

Pre- and post-intervention client surveys.

SETTING

A large urban referral hospital in Malawi.

PARTICIPANTS

Clients at purposefully selected inpatient and outpatient units on designated days (baseline, n = 310 clients; final, n = 683).

INTERVENTION

Ten-session peer-group intervention for health workers focused on HIV transmission, personal and work-related prevention, treating clients and families respectfully and incorporating HIV-related teaching.

MAIN OUTCOME MEASURES

Brief face-to-face clients' interview obtaining ratings of confidentiality of HIV, whether HIV-related teaching occurred and ratings of service quality.

RESULTS

Compared with baseline, at the final survey, clients reported higher confidence about confidentiality of clients' HIV status (83 vs. 75%, P < 0.01) and more clients reported that a health worker talked to them about HIV and AIDS (37 versus 28%, P < 0.01). More clients rated overall health services as 'very good' (five-item mean rating, 68 versus 59%, P < 0.01) and this was true for both inpatients and outpatients examined separately. However, there was no improvement in ratings of the courtesy of laboratory or pharmacy workers or of the adequacy of treatment instructions in the pharmacy.

CONCLUSIONS

HIV prevention training for health workers can have positive effects on clients' ratings of services, including HIV-related confidentiality and teaching, and should be scaled-up throughout Malawi and in other similar countries. Hospitals need to improve laboratory and pharmacy services.

Efficiency as a domain of health care systems: a phenomenographic approach

PURPOSE

This paper focuses on efficiency as a central theme of the Italian health care reforms, combining macrolevel policies with microlevel (i.e., operating room) perceptions of the concept.

DESIGN/METHODOLOGY/APPROACH

According to the phenomenographic approach, this analysis investigates how the components of a surgical team (22 semistructured interviews) experience efficiency in their daily workflows.

FINDINGS

The main findings show that the concept of efficiency is multidimensional. According to participants' perspective, several categories of efficiency collected in an outcome space emphasize an holistic view of efficiency driving health policies and strategies.

SOCIAL IMPLICATIONS

The suggestion of further relationships between perspectives and other constructs (i.e., quality, safety, patient focus, process) at micro and macro level could enhance the impact of health reforms.

ORIGINALITY/VALUE

A qualitative approach conducted at microlevel help to recognize the phenomenon (of efficiency), engaging the individual conception that practitioners have of the health efficiency.

Does HIV services decentralization protect against the risk of catastrophic health expenditures?: some lessons from Cameroon

OBJECTIVE

Scaling up antiretroviral treatment (ART) through decentralization of HIV care is increasingly recommended as a strategy toward ensuring equitable access to treatment. However, there have been hitherto few attempts to empirically examine the performance of this policy, and particularly its role in protecting against the risk of catastrophic health expenditures (CHE). This article therefore seeks to assess whether HIV care decentralization has a protective effect against the risk of CHE associated with HIV infection. DATA SOURCE AND STUDY DESIGN: We use primary data from the cross-sectional EVAL-ANRS 12-116 survey, conducted in 2006-2007 among a random sample of 3,151 HIV-infected outpatients followed up in 27 hospitals in Cameroon. DATA COLLECTION AND METHODS: Data collected contain sociodemographic, economic, and clinical information on patients as well as health care supply-related characteristics. We assess the determinants of CHE among the ART-treated patients using a hierarchical logistic model (n = 2,412), designed to adequately investigate the separate effects of patients and supply-related characteristics.

PRINCIPAL FINDINGS

Expenditures for HIV care exceed 17 percent of household income for 50 percent of the study population. After adjusting for individual characteristics and technological level, decentralization of HIV services emerges as the main health system factor explaining interclass variance, with a protective effect on the risk of CHE.

CONCLUSION

The findings suggest that HIV care decentralization is likely to enhance equity in access to ART. Decentralization appears, however, to be a necessary but insufficient condition to fully remove the risk of CHE, unless other innovative reforms in health financing are introduced.