[Biometric properties of QUALIFY: a tool for assessing quality indicators]

INTRODUCTION

Accurate health care evaluation using quality indicators (QIs) is of vital clinical importance for a quality-oriented health care system. The QUALIFY tool is the current research standard for assessing QIs of health care; however, its biometric properties in psychiatry have not yet been evaluated empirically. Our aim was to evaluate the internal consistency and structure of QUALIFY.

METHODS

This study applies a literature-based post-hoc analytical design to a sample of 289 QIs of mental health care. First, the indicators were assessed on the basis of nineteen ordinal QUALIFY criteria as a single measuring tool. Second, using Cronbach's alpha the internal consistency of the measuring tool was evaluated and the structure of QUALIFY using an explorative principal component analysis was tested.

RESULTS AND DISCUSSION

The tool showed an acceptable internal consistency (Cronbach's α=0.75), with three criteria (consideration of potential risks/side effects when using the indicator, implementation barriers taken into account, and the ability to influence the indicator) being inconsistent with the full scale. If these three criteria were not taken into account, the tool had a good internal consistency (Cronbach's α=0.81). The QUALIFY structural matrix comprises three components, one of which reflected six from eight original quality criteria of the scientific category. The other two components represent the semiotic structure of the QIs.

CONCLUSION

QUALIFY is an internally inconsistent instrument, which may be useful to assess mental health care QIs. The information about the structure of QUALIFY can be applied for the purposes of research planning as well as the interpretation and development of QIs.

[Development of an indicator set for the evaluation of the population-based integrated healthcare model 'Gesundes Kinzigtal' (Healthy Kinzigtal)]

INTRODUCTION

The project "INTEGRAL-10-year evaluation of the population-based integrated health care model 'Gesundes Kinzigtal' (Healthy Kinzigtal)" (ICM-GK) is funded by the Innovation Committee of the Federal Joint Committee (G-BA) (grant no. 01VSF16002). The evaluation is to be based on a set of indicators that can be captured in routine data. On the one hand, they can be used to assess ICM-GK programs that are program-specific and geared towards prevention and disease management. On the other hand, possible negative side effects of the ICM-GK, which is designed as a "shared savings contract", are to be examined by also observing care needs not covered by the ICM-GK contract. Since an indicator set for the evaluation of regional integrated care (IC) programs in Germany is not yet available, a suitable indicator set should be developed.

METHODS

RESULTS: The methodological framework links the OECD concept for quality assessment of health systems with Kessner's tracer methodology. Disease groups with a high prevalence ("common diseases"), prevention potential and potential for improvement through IC were selected as tracers. The literature search resulted in 239 QIs and the QI database search in 293 QIs, which were supplemented by 21 QIs from the focus groups. Out of a total of 553 QIs, 251 QIs remained after removal of duplicates and comparison with the data basis. This preliminary QI set was reduced to 101 QIs by consensus. In addition, 48 health reporting indicators were supplemented which serve to classify regional quality results. The final QI set maps the following 19 disease categories/tracers: heart failure (16 QIs), myocardial infarction (4 QIs), CHD (10 QIs), stroke (6 QIs), metabolic syndrome (7 QIs of which 5 were diabetes-related), COPD (6 QIs), asthma (3 QIs), chronic pain (5 QIs), back pain (3 QIs), geriatrics (7 QIs), dementia (8 QIs), osteoporosis (3 QIs), rheumatism (3 QIs), multiple sclerosis (2 QIs), depression (4 QIs), antibiotic therapy (3 QIs), drug safety (1 QI), child care (5 QIs), early detection/prevention (5 QIs). 33 of these QIs are dedicated to five tracers that are not explicitly ICM-GK programs. Most QIs assess aspects of the effectiveness of care for the chronically ill and measure process quality.

DISCUSSION

The set of indicators initially enables the quality assessment of regional, cross-indication care quality in the population-based integrated health care model 'Gesundes Kinzigtal' on the basis of routine data. Although the QI set focuses on effectiveness and process quality, it also includes QIs for preventive and acute care, coordination of care, patient orientation and safety, and outcomes. In contrast to other QI sets, both primary care and specialist health care and integrated, cross-sectoral and cross-professional care aspects have been considered. The benefits of the QI set for comparisons of regional quality and the evaluation of different IC programs remain to be tested.

CONCLUSION

On the basis of a broadly based research and participatory development process, a set of indicators has been developed that enables comprehensive evaluation of the regional quality of care of cross-indication, integrated care models focusing on common diseases. In order to be able to increasingly evaluate aspects of care coordination and patient orientation, health promotion as well as nursing, palliative and emergency care in the future, it would be helpful if routine data were collected or made accessible in these areas as well.

["Transferability": Is GRADE the solution?]

Biological and social factors, such as age, comorbidities and the care system, may - as well-established effect-modifiers - limit the transferability of study results to populations with dissimilar characteristics. In order to enable transparent and evidence-based decisions in systematic reviews and guidelines targeting subpopulations that are little or not represented in the study landscape, the GRADE approach is a valid tool to assess the certainty of the evidence. GRADE provides a structured methodology that covers all steps, from developing a precise question, prioritizing patient-relevant outcomes and assessing the available evidence to derive recommendations for practice, among other things. Evaluating confidence in a body of evidence comprises judgments on risk of bias, study heterogeneity, directness, including comparability between study population and target population, precision of effect estimates and publication bias. Because GRADE demands transparent decisions about the applicability of study results from the study population to the target population, gaps in the evidence landscape can be uncovered. Overall, the approach cannot solve the problem of the transferability of study results. It does, however, support the explicit handling of applicability and can give impetus to targeted research gaps.

[Quality appraisal in systematic reviews of normative literature. A problem analysis]

Systematic reviews aim at searching, selecting, analyzing and synthesizing scientific literature in a transparent and systematic way in order to inform decision-making in the health care system on the basis of the best available evidence. In recent years, such reviews have also gained importance also in bio-, public health- and research ethics, as well as in health technology assessment. Such reviews do not only analyze ethically relevant empirical literature (e.g. on risk and benefit), but normative literature as well, i.e. literature consisting of ethical arguments. As the appraisal of the literature that should be included is paramount for a systematic review, the problem of how to appraise the quality of normative literature arises. This problem has not yet been solved satisfactorily. After developing a pragmatic definition for "normative literature", a typology of different types of systematic reviews of normative literature is presented. Based on existing approaches for quality appraisal, this paper identifies three possible strategies for solving the problem of quality appraisal of normative literature, and discusses their respective strength and weaknesses relative to the different types of systematic reviews. It becomes apparent that none of the existing approaches is able to solve the problem of quality appraisal in a general and convincing way. The paper concludes with stating minimal conditions regarding the elaboration of future strategies, and outlines a promising strategy that is theoretically acceptable and practically feasible.