A systematic review: the dimensions to evaluate health care performance and an implication during the pandemic

Performance evaluation of the introduction of full sample traceability system within the specimen collection process

OBJECTIVES

To evaluate the efficacy, safety and efficiency performances related to the introduction of innovative traceability platforms and integrated blood collection systems, for the improvement of a total testing process, thus also assessing the economic and organizational sustainability of these innovative technologies.

METHODS

A mixed-method approach was utilized. A key-performance indicators dashboard was created based on a narrative literature review and expert consensus and was assessed through a real-life data collection from the University Hospital of Padova, Italy, comparing three scenarios over time (2013, 2016, 2019) with varying levels of technological integration. The economic and organizational sustainability was determined considering all the activities performed from the tube check-in to the validation of the results, with the integration of the management of the prevalent errors occurred during the process.

RESULTS

The introduction of integrated venous blood collection and full sample traceability systems resulted in significant improvements in laboratory performance. Errors in samples collected in inappropriate tubes decreased by 42 %, mislabelled samples by 47 %, and samples with irregularities by 100 %. Economic analysis revealed a cost saving of 12.7 % per tube, equating to a total saving of 447,263.80 € over a 12-month period. Organizational efficiency improved with a reduction of 13,061.95 h in time spent on sample management, allowing for increased laboratory capacity and throughput.

CONCLUSIONS

Results revealed the strategic relevance of introducing integrated venous blood collection and full sample traceability systems, within the Laboratory setting, with a real-life demonstration of TLA economic and organizational sustainability, generating an overall improvement of the process efficiency.

Performance measurement systems in primary health care: a systematic literature review

BACKGROUND

Performance measurement systems (PMS) are increasingly recognized as essential tools in healthcare services. However, there remains a significant gap in the literature regarding their development, implementation, and impact on primary health care (PHC). This study aims to systematically review peer-reviewed literature to identify and analyze existing constructs, methodologies, and challenges associated with PMS in primary care settings worldwide.

METHODS

This systematic review follows the PRISMA guidelines regarded as the gold standard for evidence synthesis in scientific and grey literature. The quality of the selected studies was assessed using the Rosalind Franklin Qualitative Research Appraisal Instrument (RF-QRA), focusing on transferability, reliability, credibility, and confirmability.

RESULTS

From an initial pool of 167 articles, 14 studies were selected for in-depth analysis. These studies highlighted several challenges, including difficulties in evaluating PMS post-implementation within primary care units, limited evidence on the longitudinal monitoring of performance indicators, and inconsistencies in methodological approaches. The findings also revealed that regional, operational, and cultural contexts influenced the most PMS adaptations.

CONCLUSIONS

This systematic review offers a comprehensive diagnosis of the best PMS models implemented globally over the past five years, emphasizing heterogeneity, diversification, and reliability. The findings underscore the potential for PMS to inform public policies to achieve high-performance primary healthcare systems and enhance decision-making processes at both operational and managerial levels.

Improvement of human performance in healthcare systems by taking into account physician behavior

PURPOSE

Efficiency standards, similar to industrial measures like overall equipment effectiveness (OEE), are being used in healthcare systems more and more. Performance indicator models applied to machines assume a constant completion time. However, for human resources, the completion time of a task may vary depending on the stress experienced. This study seeks to bridge this gap by integrating the human behavior of the physician into the performance evaluation.

DESIGN/METHODOLOGY/APPROACH

The paper presents a new algorithm called PerfoBalance that is intended to distribute waiting-patient values among doctors. By maximizing each physician's stress zones, this method helps to improve their performance as a whole. A thorough case study with medical professionals is carried out to confirm the effectiveness of the suggested methodology. The PerfoBalance algorithm is used in a variety of contexts to divide waiting-patient values among doctors and optimize stress zones.

FINDINGS

Experimental results demonstrate a significant improvement in physician efficiency when implementing the PerfoBalance algorithm. The algorithm strategically selects stress zones that contribute to higher performance rates for physicians by optimizing waiting-patient values.

ORIGINALITY/VALUE

By addressing the undervaluation of human performance difficulties in current efficiency models used in the healthcare industry, this research constitutes a significant contribution to the field. With its launch, the PerfoBalance algorithm offers a fresh viewpoint on waiting-patient value allocation and stress zone management in healthcare settings, hence representing a powerful method for increasing physician productivity.

Improving the quality of COVID-19 care in Sierra Leone: A modified Delphi process and serial nationwide assessments of quality of COVID-19 care in Sierra Leone

INTRODUCTION

Improving the quality of care that patients receive is paramount to improving patient outcomes and engendering trust during infectious disease outbreaks. Whilst Quality Improvement (QI) is well established to drive improvement in routine care and in health systems, there are fewer reports of its use during infectious disease outbreaks.

METHODS

A modified Delphi process was undertaken to create a standardized assessment tool for the quality of COVID-19 care in Sierra Leone. Four rounds of assessment were undertaken between July 2020 and July 2021. To assess change across the four assessment periods compared to baseline we used a mixed effects model and report coefficients and p values.

RESULTS

During the Delphi process, 12/14 participants selected the domains to be assessed within the tool. The final 50 questions included 13 outcome questions, 17 process questions and 20 input questions. A total of 94 assessments were undertaken over four assessment periods at 27 facilities. An increase of 8.75 (p = <0.01) in total score was seen in round 2, 10.67 (p = <0.01) in round 3 and 2.17 (p = 0.43) in round 4 compared to baseline. Mean cumulative scores for COVID-19 Treatment Centres were higher than Hospital Isolation Units (p<0.02) at all four timepoints. Significant improvements were reported in coordination, diagnostics, staffing, infection prevention and control (IPC), nutrition, and vulnerable populations domains, but not in the oxygen, care processes, infrastructure and drugs domains.

CONCLUSION

We demonstrate the feasibility of creating a quality of care assessment tool and conducting sequential nationwide assessments during an infectious disease outbreak. We report significant improvements in quality-of-care scores in round 2 and round 3 compared to baseline, however, these improvements were not sustained. We recommend the use of QI and the creation of standardised assessment tools to improve quality of care during outbreak responses.

Toward QoS Monitoring in IoT Edge Devices Driven Healthcare-A Systematic Literature Review

Smart healthcare is altering the delivery of healthcare by combining the benefits of IoT, mobile, and cloud computing. Cloud computing has tremendously helped the health industry connect healthcare facilities, caregivers, and patients for information sharing. The main drivers for implementing effective healthcare systems are low latency and faster response times. Thus, quick responses among healthcare organizations are important in general, but in an emergency, significant latency at different stakeholders might result in disastrous situations. Thus, cutting-edge approaches like edge computing and artificial intelligence (AI) can deal with such problems. A packet cannot be sent from one location to another unless the "quality of service" (QoS) specifications are met. The term QoS refers to how well a service works for users. QoS parameters like throughput, bandwidth, transmission delay, availability, jitter, latency, and packet loss are crucial in this regard. Our focus is on the individual devices present at different levels of the smart healthcare infrastructure and the QoS requirements of the healthcare system as a whole. The contribution of this paper is five-fold: first, a novel pre-SLR method for comprehensive keyword research on subject-related themes for mining pertinent research papers for quality SLR; second, SLR on QoS improvement in smart healthcare apps; third a review of several QoS techniques used in current smart healthcare apps; fourth, the examination of the most important QoS measures in contemporary smart healthcare apps; fifth, offering solutions to the problems encountered in delivering QoS in smart healthcare IoT applications to improve healthcare services.

Digital transformation of an academic hospital department: A case study on strategic planning using the balanced scorecard

Digital transformation has a significant impact on efficiency and quality in hospitals. New solutions can support the management of data overload and the shortage of qualified staff. However, the timely and effective integration of these new digital tools in the healthcare setting poses challenges and requires guidance. The balanced scorecard (BSC) is a managerial method used to translate new strategies into action and measure their impact in an institution, going beyond financial values. This framework enables quicker operational adjustments and enhances awareness of real-time performance from multiple perspectives, including customers, internal procedures, and the learning organization. The aim of this study was to adapt the BSC to the evolving digital healthcare environment, encompassing factors like the recent pandemic, new technologies such as artificial intelligence, legislation, and user preferences. A strategic mapping with identification of corresponding key performance indicators was performed. To achieve this, we employed a qualitative research approach involving retreats, interdisciplinary working groups, and semi-structured interviews with different stakeholders (administrative, clinical, computer scientists) in a rheumatology department. These inputs served as the basis for customizing the BSC according to upcoming or already implemented solutions and to define actionable, cross-level performance indicators for all perspectives. Our defined values include quality of care, patient empowerment, employee satisfaction, sustainability and innovation. We also identified substantial changes in our internal processes, with the electronic medical record (EMR) emerging as a central element for vertical and horizontal digitalization. This includes integrating patient-reported outcomes, disease-specific digital biomarker, prediction algorithms to increase the quality of care as well as advanced language models in order save resources. Gaps in communication and collaboration between medical departments have been identified as a main target for new digital solutions, especially in patients with more than one disorder. From a learning institution's perspective, digital literacy among patients and healthcare professionals emerges as a crucial lever for successful implementation of internal processes. In conclusion, the BSC is a helpful tool for guiding digitalization in hospitals as a horizontally and vertically connected process that affects all stakeholders. Future studies should include empirical analyses and explore correlations between variables and above all input and user experience from patients.

Evaluation of the implementation progress through key performance indicators in a new multimorbidity patient-centered care model in Chile

BACKGROUND

Complex health interventions involve deep organizational, structural, and cultural changes that challenge health teams and decision-makers. The explosion of chronic diseases has made the multimorbidity approach a global priority. The Centro de Innovación en Salud ANCORA UC implemented a Multimorbidity Patient-Centered Care Model in the Chilean public health system.

OBJECTIVE

This study aims to evaluate the progress of the implementation of the Multimorbidity Patient-Centered Care Model in seven primary care centers through key performance indicators.

METHODS

a set of indicators was designed to evaluate change management, operations, installation of new roles, and services and activities of the intervention strategy of the model. Key performance indicators were identified to monitor the implementation progress on minimal components for the model's sustainability. Each item was assigned against an expected minimum score of 67% of progress from the overall score. They were monitored twice in seven primary health centers in 2019 and 2020, which intervened 22,642 patients with the intervention.

RESULTS

The results showed that six of the seven primary care centers reached the minimum implementation threshold. The main advances were in operational conditions, and those with minor progress in implementation were the clinical services. Population size, organization, coordination of the health care teams, additional training, and decision-makers support were key factors that determined the degree of progress in a complex intervention.

CONCLUSION

It was possible to measure the progression of the implementation of a complex intervention through key performance indicators delivering relevant information for decision-makers that pursue a successful and faithful implementation. This study provides a valuable tool for the national scale-up of a similar model started in Chile by the Ministry of Health and other countries.

Engaging physicians and nurses in balanced scorecard evaluation-An implication at Palestinian hospitals and recommendations for policy makers

Introduction

Healthcare workers (HCWs) are seldom involved in balanced scorecard (BSC) deployments. This study aims to incorporate Palestinian HCWs in the BSC to create health policy recommendations and action plans using BSC-HCW1, a survey designed and validated based on BSC dimensions.

Methodology

In this cross-sectional study, the BSC-HCW1 survey was delivered to HCWs in 14 hospitals from January to October 2021 to get them involved in PE. The differences between physicians' and nurses' evaluations were assessed by the Mann-Whitney U-test. The causal relationships between factors were analyzed using multiple linear regression. The multicollinearity of the model was checked. Path analysis was performed to understand the BSC strategic maps based on the Palestinian HCWs' evaluations.

Results

Out of 800 surveys, 454 (57%) were retrieved. No evaluation differences between physicians and nurses were found. The BSC-HCW1 model explains 22-35% of HCW loyalty attitudes, managerial trust, and perceived patient trust and respect. HCWs' workload time-life balance, quality and development initiatives, and managerial performance evaluation have a direct effect on improving HCWs' loyalty attitudes (β = 0.272, P < 0.001; β = 0.231, P < 0.001; β = 0.199, P < 0.001, respectively). HCWs' engagement, managerial performance evaluation, and loyalty attitudes have a direct effect on enhancing HCWs' respect toward managers (β = 0.260, P < 0.001; β = 0.191, P = 0.001; β = 0.135, P = 0.010, respectively). Quality and development initiatives, HCWs' loyalty attitudes, and workload time-life balance had a direct effect on improving perceived patient respect toward HCWs (β = 254, P < 0.001; β = 0.137, P = 0.006, β = 0.137, P = 0.006, respectively).

Conclusion

This research shows that it is important to improve low-performing indicators, such as the duration of time HCWs spend with patients, their knowledge of medications and diseases, the quality of hospital equipment and maintenance, and the inclusion of strengths and weaknesses in HCWs' evaluations, so that HCWs are more loyal and less likely to want to leave. For Palestinian hospital managers to be respected more, they must include HCWs in their action plans and explain their evaluation criteria. Patients will respect Palestinian HCWs more if they prioritize their education and work quality, spend more time with patients, and reflect more loyalty. The results can be generalized since it encompassed 30% of Palestinian hospitals from all categories.

Engaging patients in balanced scorecard evaluation - An implication at Palestinian hospitals and recommendations for policy makers

Introduction

A balanced scorecard (BSC) is a comprehensive performance evaluation (PE) tool. A recent review summarized that a balanced consideration of PE from six perspectives in hospitals must be considered: financial, customer, internal, external, knowledge and growth, and managerial. However, patients were rarely engaged in BSC implementations. This research aims to engage Palestinian patients in BSC implementation to develop recommendations for policy makers.

Methodology

In this cross-sectional study, the BSC-PATIENT survey was distributed between January and October 2021. We evaluated patients' experiences and their attitudes toward BSC dimensions (BSCP ATT). The differences in evaluations based on admission status were analyzed using the Mann-Whitney U test. Causal relationships between patients' experiences and attitudes were analyzed using multiple linear regression. We tested the multicollinearity of the model. Path analysis was performed to understand the BSC strategic maps based on the Palestinian patients' evaluations.

Results

Out of 1,000 surveys, 740 were retrieved. The mean scores for Palestinian patient experiences evaluation showed that the services experience factor had the highest score (87.7 ± 17.7), and the patient care experience factor had the lowest score (57 ± 34.5). Patient experiences collectively predicted 56.4% of the variance in the BSCP ATT. The experience factors of information (β = 0.400, t = 13.543, P < 0.001), patient care (β = 0.241, t = 8.061, P < 0.001), services (β = 0.176, t = 6.497, P < 0.001), and building (β = 0.177, t = 6.308, P < 0.001) had the highest impact on BSCP ATT. The price had only a weak negative influence (β = -0.051, t = -2.040, P = 0.042). Accessibility to hospitals did not have any impact on BSCP ATT. Significant differences between inpatient and outpatients' evaluations in regard to experiences related to patient care (P = 0.042), services (P < 0.001), accessibility (P < 0.001), and BSCP ATT (P = 0.003) were found.

Conclusion

BSC-PATIENT successfully engaged patients in BSC PE at Palestinian hospitals. This research provides strong evidence for the impact of patients' information experience on their attitudes. Palestinian health policy makers must prioritize the design and delivery of patient education programs into their action plans and encourage a two-way information communication with patients. Strong evidence for patient care, services, and building experiences role in improving patients' attitudes was found. Managers should enhance patients' feedback and engagement culture in Palestinian hospitals.

How to Engage Health Care Workers in the Evaluation of Hospitals: Development and Validation of BSC-HCW1-A Cross-Sectional Study

Organizations worldwide utilize the balanced scorecard (BSC) for their performance evaluation (PE). This research aims to provide a tool that engages health care workers (HCWs) in BSC implementation (BSC-HCW1). Additionally, it seeks to translate and validate it at Palestinian hospitals. In a cross-sectional study, 454 questionnaires were retrieved from 14 hospitals. The composite reliability (CR), interitem correlation (IIC), and corrected item total correlation (CITC) were evaluated. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were used. In both EFA and CFA, the scale demonstrated a good level of model fit. All the items had loadings greater than 0.50. All factors passed the discriminant validity. Although certain factors' convergent validity was less than 0.50, their CR, IIC, and CITC were adequate. The final best fit model had nine factors and 28 items in CFA. The BSC-HCW1 is the first self-administered questionnaire to engage HCWs in assessing the BSC dimensions following all applicable rules and regulations. The findings revealed that this instrument's psychometric characteristics were adequate. Therefore, the BSC-HCW1 can be utilized to evaluate BSC perspectives and dimensions. It will help managers highlight which BSC dimension predicts HCW satisfaction and loyalty and examine differences depending on HCWs' and hospital characteristics.

Assessing Patient Experience and Attitude: BSC-PATIENT Development, Translation, and Psychometric Evaluation-A Cross-Sectional Study

Health care organizations (HCO) did not consider engaging patients in balanced scorecard (BSC) implementations to evaluate their performance. This paper aims to develop an instrument to engage patients in assessing BSC perspectives (BSC-PATIENT) and customize it for Palestinian hospitals. Two panels of experts participated in the item generation of BSC-PATIENT. Translation was performed based on guidelines. Pretesting was performed for 30 patients at one hospital. Then, 1000 patients were recruited at 14 hospitals between January and October 2021. Construct validity was tested through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Additionally, the composite reliability (CR), interitem correlation (IIC), and corrected item total correlation (CITC) were assessed to find redundant and low correlated items. As a result, the scales had a highly adequate model fit in the EFA and CFA. The final best fit model in CFA comprised ten constructs with 36 items. In conclusion, BSC-PATIENT is the first self-administered questionnaire specifically developed to engage patients in BSC and will allow future researchers to evaluate the impact of patient experience on attitudes toward BSC perspectives, as well as to compare the differences based on patient and hospital characteristics.

A differential process mining analysis of COVID-19 management for cancer patients

During the acute phase of the COVID-19 pandemic, hospitals faced a challenge to manage patients, especially those with other comorbidities and medical needs, such as cancer patients. Here, we use Process Mining to analyze real-world therapeutic pathways in a cohort of 1182 cancer patients of the Lausanne University Hospital following COVID-19 infection. The algorithm builds trees representing sequences of coarse-grained events such as Home, Hospitalization, Intensive Care and Death. The same trees can also show probability of death or time-to-event statistics in each node. We introduce a new tool, called Differential Process Mining, which enables comparison of two patient strata in each node of the tree, in terms of hits and death rate, together with a statistical significance test. We thus compare management of COVID-19 patients with an active cancer in the first vs. second COVID-19 waves to quantify hospital adaptation to the pandemic. We also compare patients having undergone systemic therapy within 1 year to the rest of the cohort to understand the impact of an active cancer and/or its treatment on COVID-19 outcome. This study demonstrates the value of Process Mining to analyze complex event-based real-world data and generate hypotheses on hospital resource management or on clinical patient care.