Quality indicators and quality assessment in child health

[Development of an indicator set for the evaluation of the quality of routine ambulatory health care for common disorders in children and adolescents]

BACKGROUND

In Germany, no consented quality indicator set (QI set) exists to date that can be used to assess the quality of pediatric care. Therefore, the aim of the project "Assessment of the quality of routine ambulatory health care for common disorders in children and adolescents" (QualiPäd) funded by the Innovation Committee of the Federal Joint Committee (grant no.: 01VSF19035) was to develop a QI set for the diseases asthma, atopic eczema, otitis media, tonsillitis, attention-deficit hyperactivity disorder (ADHD), depression and conduct disorder.

METHODS

For the observation period 2018/2019, quality indicators (QIs) were searched in indicator databases, guidelines and literature databases and complemented in part by newly formulated QIs (e.g., derived from guideline recommendations). The QIs were then assigned to content categories and dimensions according to Donabedian and OECD and reduced by removing duplicates. Finally, a panel of experts consulted the QIs using the modified RAND-UCLA Appropriateness Method (RAM).

RESULTS

The search resulted in a preliminary QI set of 2324 QIs. After the reduction steps and the evaluation of the experts, 282 QIs were included in the QI set (asthma: 72 QIs, atopic eczema: 25 QIs, otitis media: 31 QIs, tonsillitis: 12 QIs, ADHD: 53 QIs, depression: 43 QIs, conduct disorder: 46 QIs). The QIs are distributed among the following different categories: Therapy (138 QIs), Diagnostics (95 QIs), Patient-reported outcome measures/Patient-reported experience measures (PROM/PREM) (45 QIs), Practice management (31 QIs), and Health reporting (4 QIs). In the Donabedian model, 89% of the QIs capture process quality, 9% outcome quality, and 2% structural quality; according to the OECD classification, 61% measure effectiveness, 23% patient-centeredness, and 16% safety of care.

CONCLUSION

The consented QI set is currently being tested and can subsequently be used (possibly modified) to measure the quality of routine outpatient care for children and adolescents in Germany, in order to indicate the status quo and potential areas for improvement in outpatient care.

Quality indicators for collaborative care networks in persistent somatic symptoms and functional disorders: a modified delphi study

BACKGROUND

Care for persistent somatic symptoms and functional disorders (PSS/FD) is often fragmented. Collaborative care networks (CCNs) may improve care quality for PSS/FD. Effectiveness likely depends on their functioning, but we lack a straightforward quality evaluation system. We therefore aimed to develop quality indicators to evaluate CCNs for PSS/FD.

METHOD

Using an online three-round modified Delphi process, an expert panel provided, selected and ranked quality indicators for CCNs in PSS/FD. Recruited experts were diverse healthcare professionals with relevant experience in PSS/FD care in the Netherlands.

RESULTS

The expert panel consisted of 86 professionals representing 15 disciplines, most commonly physiotherapists, psychologists and medical specialists. 58% had more than 10 years experience in PSS/FD care. Round one resulted in 994 quotations, which resulted in 46 unique quality indicators. These were prioritised in round two and ranked in round three by the panel, resulting in a final top ten. The top three indicators were: "shared vision of care for PSS/FD", "pathways tailored to the individual patient", and "sufficiently-experienced caregivers for PSS/FD".

CONCLUSIONS

The identified quality indicators to evaluate CCNs in the field of PSS/FD can be implemented in clinical practice and may be useful in improving services and when assessing effectiveness.

Multisite Quality Improvement Program Within the Project ECHO Diabetes Remote Network

BACKGROUND

The telementoring Project ECHO (Extension for Community Healthcare Outcomes) model has been shown to improve disease management in diabetes in many underserved communities. The authors aim to evaluate if ECHO could also be an effective tool for quality improvement (QI) of diabetes care in these communities.

METHODS

Thirteen clinics in underserved communities in California and Florida participating in Project ECHO Diabetes were recruited for a 12-month QI program. The program provided weekly tele-education sessions, including a didactic presentation and case-based discussion. In addition, clinics chose their own set of quality measures to improve and met remotely to discuss their efforts, successes, and setbacks every quarter with mentorship from QI experts.

RESULTS

Of the 31 QI initiatives attempted by different clinics, all had either made improvements (25 initiatives, 80.6%) or were in the process of making improvements (6 initiatives, 19.4%) in structural, process, and outcome measures. Examples of these measures include whether clinics have protocols to identify high-risk patients (structure), numbers of continuous glucose monitor prescriptions submitted by the clinics (process), and percentage of patients with diabetes whose most recent HbA1c are > 9% (outcome). For one measure, 40.0% of the clinics had achieved a higher percentage of cumulative HbA1c measurement in the third quarter of the year, compared to the fourth quarter in the previous year. The cost of QI implementation varied widely due to different number of personnel involved across sites.

CONCLUSION

A QI program delivered via Project ECHO Diabetes can facilitate quality improvements in underserved communities.

Process over outcome quality in paediatrics? An analysis of outpatient healthcare quality indicators for seven common diseases

PURPOSE

The purpose of this study was to examine the scope, quality dimensions and treatment aspects covered by existing quality indicators (QIs) for the somatic diseases bronchial asthma, atopic eczema, otitis media and tonsillitis as well as the psychiatric disorders attention deficit hyperactivity disorder (ADHD), depression and conduct disorder in paediatrics.

METHODS

QIs were identified through an analysis of the guidelines and a systematic search of literature and indicator databases. Subsequently, two researchers independently assigned the QIs to the quality dimensions according to Donabedian and Organisation for Economic Cooperation and Development (OECD) and to the content categories covering the treatment process.

RESULTS

We found 1268 QIs for bronchial asthma, 335 QIs for depression, 199 QIs for ADHD, 115 QIs for otitis media, 72 QIs for conduct disorder, 52 QIs for tonsillitis and 50 QIs for atopic eczema. Of these, 78% focused on process quality, 20% on outcome quality and 2% on structural quality. Using OECD criteria, 72% of the QIs were assigned to effectiveness, 17% to patient-centredness, 11% to patient safety and 1% to efficiency. The QIs covered the following categories: diagnostics (30%), therapy (38%), patient-reported outcome measures/ observer-reported outcome measures/patient-reported experience measures (in sum 11%), health monitoring (11%) and office management (11%).

CONCLUSION

Most QIs focused on the dimensions of effectiveness and process quality, and on the categories of diagnostics and therapy, with outcome-focused and patient-focused QIs being under-represented. Possible reasons for this striking imbalance could be the easier measurability and clearer assignment of accountability in comparison to the QIs of outcome quality, patient-centredness and patient safety. To produce a more balanced picture of the quality of healthcare, the future development of QIs should prioritise the currently under-represented dimensions.

Pediatric nurse‐sensitive outcomes: A systematic review of international literature

BACKGROUND

Nurse-sensitive outcomes are measures for improvement and evaluation of the quality of nursing care delivered. The specific outcomes that need to be measured will be determined by the patient population, as well as the field and scope of practice, in which nursing care is being delivered. Currently, there is no internationally agreed upon set of nurse-sensitive outcomes for pediatric nursing, which provides specialist care to infants, children, and young people.

AIM

To identify and evaluate nurse-sensitive outcomes for pediatric nursing.

METHODS

A systematic review was conducted. Five electronic databases (British Nursing Index, CINAHL, EMBASE, MEDLINE, and EMCARE) were searched in the period up to February 2022. Studies were selected for inclusion using title and abstract screening using predetermined criteria. The Critical Appraisal Skills Programme tool was used for quality assessment. A narrative synthesis of the results was performed.

RESULTS

A total of 633 studies were identified from online searches, with 14 studies meeting the inclusion criteria. All studies had moderate to high methodological strength. A total of 57 nurse-sensitive outcomes were identified from all included studies. Using the nurse-sensitive outcome conceptual analysis framework, 25 (45%) of the items were classified as outcome attributes, 20 (35%) as process attributes, and 13 (23%) as structure attributes. The most frequently reported nurse-sensitive outcomes included pressure ulcers, nosocomial infections, hospital-acquired infections, peripheral intravenous infiltration, failure to rescue, and staffing levels.

CONCLUSIONS

This review provides an up-to-date and comprehensive list of nurse-sensitive outcomes for use in pediatric nursing and describes their frequency of use. However, further work is required to achieve consensus for an international core nurse-sensitive outcome set for pediatric nursing with policy recommendations to ensure agreed-upon minimum standards.

IMPLICATIONS FOR NURSING AND HEALTH POLICY

Policy initiatives and guideline recommendations on nurse-sensitive outcome frameworks as part of patient safety should be a part of key priorities for policy makers. The commonly reported nurse-sensitive outcomes should be incorporated into daily bedside pediatric clinical nursing practice as a mechanism to evaluate and improve the quality of care, enhancement of patient safety, and better outcomes.

Quality of child healthcare in European countries: common measures across international databases and national agencies

BACKGROUND

The evaluation of child healthcare is not yet widely explored, especially from a cross-country comparison perspective. The routine adoption of measures by national assessment agencies is under-investigated. Though the guiding principles developed at international level call for a child-centric multi-dimensional evaluation of child care, its feasibility is hampered by the availability of robust and harmonized data.

METHODS

To explore the data availability, international databases (IDBs) were scrutinized and measures dealing with child health-related issues were collated. In parallel, an ad hoc questionnaire was administrated to 30 Country Agents (CAs) to gather measures routinely adopted at local level. To facilitate the comparison of measures, a three-level conceptual map was developed.

RESULTS

The IDBs yielded at 207 measures that pertained mainly to non-health determinants of health, whereas the 352 measures obtained from CAs focused on process and outcome. A set of 33 common measures that related to immunization, morbidity and mortality were identified.

CONCLUSIONS

A limited set of measures used both in IDBs and at national level identify common areas of concerns that certainly capture crucial issues with child prevention and health outcomes. However, they are far from satisfying a child-centric multi-dimensional approach to the evaluation of child well-being and well-becoming. There is room for improvement at both international and national levels. IDBs should include and harmonize measures that concern the provision of child-centric services and encompass physical, social and mental development. At the national level, efforts towards the inclusion of measures that concern non-health determinants of health should be pursued.

The Complex Association of Race/Ethnicity With Pain Treatment Quality in an Urban Medical Center With 2 Pediatric Emergency Departments

OBJECTIVE

This study aimed to explore racial differences in analgesia quality.

METHODS

A retrospective cross-sectional study of 24,733 visits by individuals 21 years or younger with pain scores of 4 to 10 was performed using electronic medical records. We compared 2 process metrics, treatment with any analgesics within 60 minutes and treatment with opioids within 60 minutes, and one outcome metric, a reduction in pain score by 2 or more points within 90 minutes. Multivariable logistic regression adjusted for the effects of patient characteristics and health status. We also determined variations in analgesia quality among those with severe pain.

RESULTS

When compared with white children, black children were more likely to receive any analgesia (adjusted odds ratio [aOR], 1.94; 95% confidence interval, 1.71-2.21), but both blacks (aOR, 0.66; 0.51-0.85) and Hispanics (aOR, 0.56; 0.39-0.80) were less likely to receive opioids. Blacks were more likely to reduce their pain score (aOR, 1.50; 1.28-1.76).Among children with severe pain, both blacks and Hispanics were more likely to receive any analgesia (black: aOR, 2.05 [1.71-2.46]; Hispanic: aOR, 1.29 [1.05-1.59]), and Hispanic children were less likely to receive opioids (aOR, 0.58; 0.37-0.91). Again, black children were more likely to reduce their pain score (aOR, 1.42; 1.13-1.79).

CONCLUSIONS

The relationship between race/ethnicity and analgesia is complex. Although minority children were less likely to receive opioids, black children had better treatment outcomes. Future studies should explore clinical response to analgesia in addition to process measures to better understand if differential treatment may be justified to achieve equitable care outcomes.

Primary Care Quality Improvement Metrics and National Committee on Quality Assurance Medical Home Recognition for Children With Medical Complexity

The Complex Care Center at Cincinnati Children’s Hospital Medical Center developed and implemented a set of evidence-based clinical process measures of immunization delivery, preventive and chronic condition laboratory screening, and behavioral health medication surveillance for use in the primary care setting.

Developing key performance indicators for a tertiary children's hospital network

Objective The aim of this study is to describe the experience of developing key performance indicators (KPIs) for Sydney Children's Hospital Network (SCHN), the largest paediatric healthcare entity in Australia. Methods Beginning with a published methodology, the process of developing KPIs involved five phases: (1) identification of potential KPIs referencing the organisational strategic plan and pre-existing internal and external documents; (2) consolidation into a pragmatic set; (3) analysis of potential KPIs against selection criteria; (4) mapping these back against the strategic plan and management structure; and (5) presentation to key stakeholders to ensure suitability and traction. Consistent with the strategic plan, a subset of indicators was selected to address quality of care for children from priority populations. Results A pragmatic list of 60 mandated and 50 potential KPIs was created from the 328 new and 397 existing potentially relevant KPIs generated by the executive team. Of these, 20 KPIs were selected as the most important; 65% were process measures. The majority of mandated KPIs were process measures. Of the KPIs selected to highlight inequities, there were proportionately more outcome measures (44% outcome, 27% process). Less than one-third could currently be measured by the organisation and were thus aspirational. Conclusion Developing a KPI suite requires substantial time, effort and organisational courage. A structured approach to performance measurement and improvement is needed to ensure a balanced suite of KPIs that can be expected to drive an organisation to improve child health outcomes. Future directions for SCHN include a systematic approach to implementation beyond the mandated KPIs, including KPIs that reflect equity and improved outcomes for priority populations, development of meaningful measures for the aspirational KPIs, adding structure KPIs and measurement of changes in child health outcomes related to the development of this KPI process. What is known about the topic? Health services are increasingly required to demonstrate accountability through KPIs. There is a body of literature on both theoretical frameworks for measuring performance and a long list of possible measures, however developing a meaningful suite of KPIs remains a significant challenge for individual organisations. What does this paper add? This paper describes lessons learned from the practical, pragmatic application of a published methodology to develop a suite of KPIs for the largest paediatric healthcare entity in Australia. It provides a select list of the highest-level KPIs selected by the organisation to stimulate further discussion among similar organisations in relation to KPI selection and implementation. What are the implications for practitioners? Developing and implementing a suite of meaningful KPIs for a large organisation requires courage, an understanding of health informatics, stakeholder engagement, stamina and pragmatism. The process we describe can be replicated and/or modified as needed, with discussion of key lessons learned to help practitioners plan ahead.

Is It Feasible to Use Electronic Health Records for Quality Measurement of Adolescent Care?

OBJECTIVE

To determine the extent to which it is feasible to implement quality measures on electronic health records (EHRs) as currently implemented in pediatric health centers.

METHODS

A survey of information technology professionals at 10 institutions that provide primary care services to adolescents. The survey asked whether data about care was being captured electronically across the nine domains relevant to adolescent well care: Screening, Health Risks, Sexual Health, Diagnosis and History, Laboratory Results, Prescriptions, Referrals, Forms Management, and Patient Demographics. For each domain, we developed a scale of the extent to which the EHR makes quality measurement feasible.

RESULTS

Overall feasibility scores varied across centers from 34% to 85% and from 53% to 80% across care domains. One centre reported 100% feasibility for 8 of 10 care domains.

CONCLUSIONS

Electronic health records can facilitate quality improvement, but the feasibility of such use depends on the presence, validity, and accessibility of the quality data in the EHR. Even among the largest and most sophisticated pediatric EHR systems, quality of care measurement is not possible yet for all aspects of adolescent well care without manual effort to review and code data. Nevertheless, almost all quality measures were reported to be feasible in some systems.

Assessing nursing quality in paediatric intensive care units: a cross-sectional study in China

BACKGROUND

Nursing-sensitive indicators are considered effective tools for improving the quality of care in hospitals. However, these have not been used in paediatric intensive care units (PICUs) in China.

AIM

To develop nursing-sensitive indicators for PICUs and to assess the quality of nursing in PICUs in China based on the nursing-sensitive indicators.

DESIGN

Multi-centre, cross-sectional study.

METHODS

Structure, process and outcome indicators were developed and measured from 1 January to 31 March 2014 in seven PICUs in China.

RESULTS

The structure indicators showed that one nurse cared for an average of 2·8 patients in a PICU, and 44% of nurses had a bachelor's degree. The process indicators revealed that hand-washing compliance varied across PICUs, whereas pain management and physical restraint have not been adequately addressed in China. The outcome indicators revealed that the incidence rates of ventilator-associated pneumonia and central-line-associated blood stream infections were 2·96 and 0·7, respectively, per 1000 device days. Patients were intubated for a total of 4392 mechanical ventilator days, and 32 patients (7·29‰) had an unplanned extubation. Nurses were moderately satisfied in their jobs (3·1 ± 0·3), and parents reported that nurses provide high quality of care.

CONCLUSIONS

This study developed and used nursing-sensitive indicators to assess the quality of nursing in PICUs in China, which provided a reference for national and international comparisons of nursing quality in PICUs. Nursing staffing levels and education should be improved. Pain management and physical restraints should be regulated in China's PICUs. Nurse managers need to explore staff attitudes towards implementation of family-centred care. The development of a national database of nursing quality indicators can contribute to quality and safety improvement.

RELEVANCE TO CLINICAL PRACTICE

This study developed a set of nursing-sensitive indicators, and these indicators were used to assess and improve the quality of nursing in PICUs.

Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees

BACKGROUND

Monitoring quality is an important way of understanding how the health care system is serving children and families. The Children's Health Insurance Program Reauthorization Act of 2009 (CHIPRA) Pediatric Quality Measures Program (PQMP) funded efforts to develop and enhance measures to assess care for children and adolescents. We describe the processes used by the PQMP grantees to develop measures to assess the health care of children and adolescents in Medicaid and the Children's Health Insurance Program.

METHODS

Key steps in the measures development process include identifying concepts, reviewing and synthesizing evidence, prioritizing concepts, defining how measures should be calculated, and measure testing. Stakeholder engagement throughout the process is critical. Case studies illustrate how PQMP grantees adapted the process to respond to the nature of measures they were charged to develop and overcome challenges encountered.

RESULTS

PQMP grantees used varied approaches to measures development but faced common challenges, some specific to the field of pediatrics and some general to all quality measures. Major challenges included the limited evidence base, data systems difficult or unsuited for measures reporting, and conflicting stakeholder priorities.

CONCLUSIONS

As part of the PQMP, grantees were able to explore innovative methods to overcome measurement challenges, including new approaches to building the evidence base and stakeholder consensus, integration of alternative data sources, and implementation of new testing methods. As a result, the PQMP has developed new quality measures for pediatric care while also building an infrastructure, expertise, and enhanced methods for measures development that promise to provide more relevant and meaningful tools for improving the quality of children's health care.

Medical complexity and pediatric emergency department and inpatient utilization

OBJECTIVES

To characterize the use of and disposition from a tertiary pediatric emergency department (PED) by children with chronic conditions with varying degrees of medical complexity.

METHODS

We conducted a retrospective cohort study using a dataset of all registered PED patient visits at Seattle Children's Hospital from January 1, 2008, through December 31, 2009. Children's medical complexity was classified by using a validated algorithm (Clinical Risk Group software) into nonchronic and chronic conditions: episodic chronic, lifelong chronic, progressive chronic, and malignancy. Outcomes included PED length of stay (LOS) and disposition. Logistic regression generated age-adjusted odds ratios (AOR) of admission with 95% confidence intervals (CIs).

RESULTS

PED visits totaled 77 748; 20% (15 433) of which were for children with chronic conditions. Compared with visits for children without chronic conditions, those for children with chronic conditions had increased PED LOS (on average, 79 minutes longer; 95% CI 77-81; P < .0001) and hospital (51% vs 10%) and PICU (3.2% vs 0.1%) admission rates (AOR 10.3, 95% CI 9.9-10.7 to hospital and AOR 25.0, 95% CI 17.0-36.0 to PICU). Admission rates and PED LOS increased with increasing medical complexity.

CONCLUSIONS

Children with chronic conditions comprise a significant portion of annual PED visits in a tertiary pediatric center; medical complexity is associated with increased PED LOS and hospital or PICU admission. Clinical Risk Group may have utility in identifying high utilizers of PED resources and help support the development of interventions to facilitate optimal PED management, such as pre-arrival identification and individual emergency care plans.

Care delivery and outcomes among Belgian children and adolescents with type 1 diabetes

We aimed to investigate care processes and outcomes among children and adolescents with type 1 diabetes treated in hospital-based multidisciplinary paediatric diabetes centres. Our retrospective cross-sectional study among 12 Belgian centres included data from 974 patients with type 1 diabetes, aged 0-18 years. Questionnaires were used to collect data on demographic and clinical characteristics, as well as process of care completion and outcomes of care in 2008. Most patients lived with both biological or adoption parents (77 %) and had at least one parent of Belgian origin (78 %). Nearly all patients (≥95 %) underwent determination of HbA(1c) and BMI. Screening for retinopathy (55 %) and microalbuminuria (73 %) was less frequent, but rates increased with age and diabetes duration. Median HbA(1c) was 61 mmol/mol (7.7 %) [interquartile range 54-68 mmol/mol (7.1-8.4 %)] and increased with age and insulin dose. HbA(1c) was higher among patients on insulin pump therapy. Median HbA(1c) significantly differed between centres [from 56 mmol/mol (7.3 %) to 66 mmol/mol (8.2 %)]. Incidence of severe hypoglycaemia was 30 per 100 patient-years. Admissions for ketoacidosis had a rate of 3.2 per 100 patient-years. Patients not living with both biological or adoption parents had higher HbA(1c) and more admissions for ketoacidosis. Parents' country of origin was not associated with processes and outcomes of care.

CONCLUSION

Outcomes of care ranked well compared to other European countries, while complication screening rates were intermediate. The observed centre variation in HbA(1c) remained unexplained. Outcomes were associated with family structure, highlighting the continuing need for strategies to cope with this emerging challenge.

What is quality improvement and why should child neurologists care?

In this article, the authors discuss the 6 domains of care identified by the Institute of Medicine report, Crossing the Chasm, with examples and questions that are especially relevant to physicians caring for children who have neurologic disorders and their families.