Time to have a paradigm shift in health care quality measurement

Hospital performance evaluation indicators: a scoping review

BACKGROUND

Hospitals are the biggest consumers of health system budgets and hence measuring hospital performance by quantitative or qualitative accessible and reliable indicators is crucial. This review aimed to categorize and present a set of indicators for evaluating overall hospital performance.

METHODS

We conducted a literature search across three databases, i.e., PubMed, Scopus, and Web of Science, using possible keyword combinations. We included studies that explored hospital performance evaluation indicators from different dimensions.

RESULTS

We included 91 English language studies published in the past 10 years. In total, 1161 indicators were extracted from the included studies. We classified the extracted indicators into 3 categories, 14 subcategories, 21 performance dimensions, and 110 main indicators. Finally, we presented a comprehensive set of indicators with regard to different performance dimensions and classified them based on what they indicate in the production process, i.e., input, process, output, outcome and impact.

CONCLUSION

The findings provide a comprehensive set of indicators at different levels that can be used for hospital performance evaluation. Future studies can be conducted to validate and apply these indicators in different contexts. It seems that, depending on the specific conditions of each country, an appropriate set of indicators can be selected from this comprehensive list of indicators for use in the performance evaluation of hospitals in different settings.

How subjective well-being, patient-reported clinical improvement (PROMs) and experience of care (PREMs) relate in an acute psychiatric care setting?

BACKGROUND

Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are increasingly acknowledged as critical tools for enhancing patient-centred, value-based care. However, research is lacking on the impact of using standardized patient-reported indicators in acute psychiatric care. The aim of this study was to explore whether subjective well-being indicators (generic PROMs) are relevant for evaluating the quality of hospital care, distinct from measures of symptom improvement (disease-specific PROMs) and from PREMs.

METHODS

Two hundred and forty-eight inpatients admitted to a psychiatric university hospital were included in the study between January and June 2021. Subjective well-being was assessed using standardized generic PROMs on well-being, symptom improvement was assessed using standardized disease-specific PROMs, and experience of care using PREMs. PROMs were completed at admission and discharge, PREMs were completed at discharge. Clinicians rated their experience of providing treatment using adapted PREMs items.

RESULTS

Change in subjective well-being (PROMs) at discharge was significantly (p < 0.001), but moderately (r² = 28.5%), correlated to improvement in symptom outcomes, and weakly correlated to experience of care (PREMs) (r² = 11.0%), the latter being weakly explained by symptom changes (r² = 6.9%). Patients and clinicians assessed the experience of care differently.

CONCLUSIONS

This study supports the case for routinely measuring patients' subjective well-being to better capture the unmet needs of patients undergoing psychiatric hospital treatment, and the use of standardized patient-reported measures as key indicators of high quality of care across mental health services.

Pedagogical value of a hospitality awards programme

Objective

Assistance Publique-Hôpitaux de Paris (AP-HP), the leading university hospital in France, proposed to offer its services to candidate on a voluntary basis for a hospitality award, certifying compliance to a 240-item home-made questionnaire designed by healthcare providers and patients’ representatives. It combined an objective examination of the services and patients’ questionnaires, covering seven domains: reception and information from admission to discharge; cleanliness, comfort and environment; proposed services (eg, access to Wi-Fi); culture, relaxation and well-being; meals; linen and relationship quality with hospital staff. The procedure was completed in two steps: an initial self-evaluation to detect improvable deficiencies, followed by an awarding visit. A service received the hospitality award if at least 80% of the reference criteria were met during this second evaluation. Here, we describe the construction of this hospitality awards programme and present a comparison of the scores obtained during the two steps.

Design and methods

Retrospective comparison by usual statistical tests.

Setting

AP-HP, grouping 39 university hospitals (21 000 beds, 8 million annual patient visits).

Participants

The 211 services from 29 different hospitals engaged in the procedure (2017–2019).

Results

Only one service did not get the award (self-evaluation 83%, visit score 79%). The score was higher during the awarding visit (89.0%±5.6%) than during self-evaluation (85.5%±4.3%, n=211, p<0.00001), with increased scores for the following domains (p<0.005): patient reception and information; cleanliness, comfort and environment; proposed services; culture, relaxation and well-being.

Conclusion

(1) Internal self-evaluation is feasible. (2) By diffusing criteria of hospitality, the procedure had a pedagogical value leading to rapid and significant improvements. (3) This quality assessment procedure results in an award that can be posted in the departments. By appealing to pride, this procedure should promote hospitality in hospitals.

Patient's perspective on improving the quality of acute medical care: determining patient reported outcomes

Rationale

There is an increasing societal demand for quality assurance and transparency of medical care. The American National Academy of Medicine has determined patient centredness as a quality domain for improvement of healthcare. While many of the current quality indicators are disease specific, most emergency department (ED) patients present with undifferentiated complaints. Therefore, there is a need for generic outcome measures. Our objective was to determine relevant patient reported outcomes (PROs) for quality measurement of acute care.

Methods

We conducted semistructured interviews in patients ≥18 years presenting at the ED for internal medicine. Patients with a cognitive impairment or language barrier were excluded. Interviews were analysed using qualitative content analysis.

Results

Thirty patients were interviewed. Patients reported outcomes as relevant in five domains: relief of symptoms, understanding the diagnosis, presence and understanding of the diagnostic and/or therapeutic plan, reassurance and patient experiences. Experiences were often mentioned as relevant to the perceived quality of care and appeared to influence the domain reassurance.

Conclusion

We determined five domains of relevant PROs in acute care. These domains will be used for developing generic patient reported measures for acute care. The patients’ perspective will be incorporated in these measures with the ultimate aim of organising truly patient-centred care at the ED.

Validation of PROMIS Physical Function Instruments in Patients With an Orthopaedic Trauma to a Lower Extremity

OBJECTIVES

To evaluate the reliability, convergent validity, known-groups validity, and responsiveness of the Patient-Reported Outcomes Measurement Information System (PROMIS) Mobility Computer Adaptive Test (CAT) and PROMIS Physical Function 8a Short Form.

DESIGN

Prospective cohort study.

SETTING

Two Level-I trauma centers.

PATIENTS

Eligible adults with an isolated lower extremity trauma injury receiving treatment were approached consecutively (n = 402 consented at time 1, median = 80 days after treatment). After 6 months, 122 (30.3%) completed another assessment.

INTERVENTION

Cross-sectional and longitudinal monitoring of patients.

MAIN OUTCOME MEASUREMENTS

Floor and ceiling effects, reliability (marginal reliability and Cronbach's alpha), convergent validity, known-groups discriminant validity (weight-bearing status and fracture severity), and responsiveness (Cohen's d effect size) were evaluated for the PROMIS Mobility CAT, PROMIS Physical Function 8a Short Form, and 5 other measures of physical function.

RESULTS

PROMIS PFSF8a and Foot and Ankle Ability Measure Activities of Daily Living Index had ceiling effects. Both PROMIS measures demonstrated excellent internal consistency reliability (mean marginal reliability 0.94 and 0.96; Cronbach's alpha = 0.96). Convergent validity was supported by high correlations with other measures of physical function (r = 0.70-0.87). Known-groups validity by weight-bearing status and fracture severity was supported as was responsiveness (Mobility CAT effect size = 0.81; Physical Function Short Form 8a = 0.88).

CONCLUSIONS

The PROMIS Mobility CAT and Physical Function 8a Short Form demonstrated reliability, convergent and known-groups discriminant validity, and responsiveness in a sample of patients with a lower extremity orthopaedic trauma injury.

Framework of care: communicating the structure and processes of care

AIM

This article attempts to present a 'macro view' of the role and nature of an organization's Framework of Care (FrOC). This 'view' arises from a critical reflection on the available literature and the combined professional experience of the authors, who have worked in a variety of healthcare systems and settings in Australia, North America, United Kingdom, and the Middle East.

BACKGROUND

FrOC can be defined as the systems and processes within an organization that structure the delivery of care. These systems and processes are made evident in a series of documents, such as the Mission and Vision statement, Policies and Procedures, Standards of Care, Clinical Practice Guidelines, Clinical Pathways, and Protocols. These frameworks can provide structure for important organizational activities such as clinical audits, quality management and clinical information system (CIS) 'decision support', thereby supporting clinicians in their efforts to deliver high-quality, evidence-based care. How a healthcare organization structures its systems and processes of care directly impacts the patient and caregiver experience - made evident in patient and staff satisfaction with the services provided.

RECOMMENDATIONS

Mapping out and understanding an organization's FrOC is a critical first step for interprofessional teams attempting to implement evidence into practice and/or accreditation teams and expert consultants critiquing the performance of an organization.

Disability Policy Evaluation: Combining Logic Models and Systems Thinking

Policy evaluation focuses on the assessment of policy-related personal, family, and societal changes or benefits that follow as a result of the interventions, services, and supports provided to those persons to whom the policy is directed. This article describes a systematic approach to policy evaluation based on an evaluation framework and an evaluation process that combine the use of logic models and systems thinking. The article also includes an example of how the framework and process have recently been used in policy development and evaluation in Flanders (Belgium), as well as four policy evaluation guidelines based on relevant published literature.

What questionnaires to use when measuring quality of life in sacral tumor patients: the updated sacral tumor survey

BACKGROUND CONTEXT

Patient-reported outcomes are becoming increasingly important when investigating results of patient and disease management. In sacral tumor, the symptoms of patients can vary substantially; therefore, no single questionnaire can adequately account for the full spectrum of symptoms and disability.

PURPOSE

The purpose of this study is to analyze redundancy within the current sacral tumor survey and make a recommendation for an updated version based on the results and patient and expert opinions.

STUDY DESIGN/SETTING

A survey study from a tertiary care orthopedic oncology referral center was used.

PATIENT SAMPLE

The patient sample included 70 patients with sacral tumors (78% chordoma).

OUTCOME MEASURES

The following 10 questionnaires included in the current sacral tumor survey were evaluated: the Patient-Reported Outcomes Measurement Information System (PROMIS) Global Item short form, PROMIS Pain Intensity short form, PROMIS Pain Interference short form, PROMIS Neuro-QOL v1.0 Lower Extremity Function short form, PROMIS v1.0 Anxiety short form, the PROMIS v1.0 Depression short form, the International Continence Society Male short form, the Modified Obstruction-Defecation Syndrome questionnaire, the PROMIS Sexual Function Profile v1.0, and the Stoma Quality of Life tool.

METHODS

We performed an exploratory factor analysis to calculate the possible underlying latent traits. Spearman rank correlation coefficients were used to measure to what extent the questionnaires converged. We hypothesized the existence of six domains based on current literature: mental health, physical health, pain, gastrointestinal symptoms, sexual function, and urinary incontinence. To assess content validity, we surveyed 32 patients, 9 orthopedic oncologists, 1 medical oncologist, 1 radiation oncologist, and 1 orthopedic oncology nurse practitioner with experience in treating sacral tumor patients on the relevance of the domains.

RESULTS

Reliability as measured by Cronbach alpha ranged from 0.65 to 0.96. Coverage measured by floor and ceiling effects ranged from 0% to 52% and from 0% to 30%, respectively. Explanatory factor analysis identified three traits to which the questionnaires that were expected to measure a similar construct correlated the most: mental health, physical function, and pain. Content validity index demonstrated low disagreement among patients (range: 0.10-0.18) and high agreement among physicians (range: 0.91-1.0) on the relevance of the proposed domains. Social health was identified by 50% of the commenting patients as an important yet missing domain.

CONCLUSIONS

The current sacral tumor survey is incomplete and time-consuming, and not all surveys are appropriate for the sacral tumor population. Our recommended survey contains less than half the questions and includes the newly recognized social health domain.

Can Hospital Competition Really Affect Hospital Behavior or Not? An Empirical Study of Different Competition Measures Comparison in Taiwan

Different approaches to measure the hospital competition index might lead to inconsistent results of the effects of hospital competition on innovation adoption. The purpose of this study is to adopt a different approach to define market area and measure the level of competition to examine whether hospital competition has a positive effect on hospital behavior, taking quality indicator projects participation as an example. A total of 238 hospitals located in Taipei, Taichung, and Kaohsiung were recruited in this study. Competition index was used as the independent variable, and participation lists of Taiwan Clinical Performance Indicator and Taiwan Healthcare Indicator Series in 2012 were used as dependent variables. All data used in this study were retrieved from the 2012 national hospital profiles and the participation list of the 2 quality indicator projects in 2012; these profiles are issued by the Taiwan Ministry of Health and Welfare annually. Geopolitical boundaries and 4 kinds of fixed radiuses were used to define market area. Herfindahl-Hirschman Index and hospital density were used to measure the level of competition. A total of 12 competition indices were produced in this study by employing the geographic information system, while max-rescaled R² was used to evaluate and compare the models on goodness of fit. The results show that the effects of hospital competition on quality indicator projects participation were varied, which mean different indicators for market competition might reveal different conclusions. Furthermore, this study also found the Herfindahl-Hirschman Index at 5-km radius was the optimum competition index.