Identifying the Need for Good Practices in Health Technology Assessment: Summary of the ISPOR HTA Council Working Group Report on Good Practices in HTA

The systematic use of evidence to inform healthcare decisions, particularly health technology assessment (HTA), has gained increased recognition. HTA has become a standard policy tool for informing decision makers who must manage the entry and use of pharmaceuticals, medical devices, and other technologies (including complex interventions) within health systems, for example, through reimbursement and pricing. Despite increasing attention to HTA activities, there has been no attempt to comprehensively synthesize good practices or emerging good practices to support population-based decision-making in recent years. After the identification of some good practices through the release of the ISPOR Guidelines Index in 2013, the ISPOR HTA Council identified a need to more thoroughly review existing guidance. The purpose of this effort was to create a basis for capacity building, education, and improved consistency in approaches to HTA-informed decision-making. Our findings suggest that although many good practices have been developed in areas of assessment and some other key aspects of defining HTA processes, there are also many areas where good practices are lacking. This includes good practices in defining the organizational aspects of HTA, the use of deliberative processes, and measuring the impact of HTA. The extent to which these good practices are used and applied by HTA bodies is beyond the scope of this report, but may be of interest to future researchers.

[Experiences with synopses of clinical guidelines using the example of synopses for Disease Management Programmes (DMP)]

Guideline synopses, i.e. the systematic synthesis of clinical practice guidelines, are used as a basis for Disease Management Programmes (DMPs) in Germany. One of the responsibilities of the German Institute for Quality and Efficiency in Health Care (IQWiG) is the preparation of guideline synopses for DMPs. The article describes the experiences with this methodology.

METHODS

For the preparation of guideline synopses, a systematic search for evidence-based guidelines is conducted, and their recommendations are extracted and synthesised. In parallel, the quality of the guidelines is assessed using the AGREE instrument. The need for updating or supplementing a DMP is justified by means of the Grade of Recommendation (GoR) or, alternatively, the Level of Evidence (LoE).

EXPERIENCES

Since 2006 IQWiG has synthesised recommendations from 256 evidence-based guidelines in 12 guideline synopses for DMPs. The results are then used by the Federal Joint Committee (G-BA) to update DMPs. Using the example of the 12 synopses, several analyses were conducted. It was determined that a search for guidelines in guideline databases is sufficient and that a search in bibliographic databases can be dispensed with. Furthermore, the analyses showed that a large proportion of recommendations in evidence-based guidelines are not clearly linked to a GoR or LoE. If GoR and LoE are provided, only about 42% of recommendations with a strong GoR also refer to a strong LoE. It was also shown that only 21 % of the analysed guideline providers supplied information on the handling of unpublished data. With consistent average to high values, the assessment of the methodological quality across all of the prepared synopses allows for the conclusion of a basically acceptable guideline quality, but with a need for improvement.

OUTLOOK

A guideline synopsis is an established tool for identifying health care standards as a basis for developing and updating DMPs. Further methodological development, particularly in collaboration with guideline providers, appears to be reasonable. It should be examined whether guideline synopses are suitable not only for guideline and DMP development, but also for other health care issues.

Tools for assessing the content of guidelines are needed to enable their effective use--a systematic comparison

Background

To ensure that clinical practice guidelines (CPGs) form a sound basis for decision-making in health care, it is necessary to be able to reliably assess and ensure their quality. This results in the need to assess the content of guidelines systematically, particularly with regard to the validity of their recommendations.

The aim of the present analysis was to determine the suitability and applicability of frequently used assessment tools for evidence syntheses with regard to the assessment of guideline content.

Methods

We conducted a systematic comparison and analysis of established tools for the assessment of evidence syntheses (guidelines, systematic reviews, health technology assessments). The tools analyzed were: ADAPTE, AGREE II, AMSTAR, GLIA and the INAHTA checklist. We analyzed methodological steps related to the assessment of the reliability and validity of guideline recommendations. Data were extracted and analyzed by two persons independently of one another.

Results

Widely used tools for the methodological assessment of evidence syntheses are not suitable for a comprehensive content-related assessment. They remain mostly at the level of assessment of the documentation of processes. Some tools assess selected content-related aspects, but operationalization is either unspecific or lacking.

Conclusion

None of the tools analyzed enables the structured and comprehensive assessment of the content of guideline recommendations with special regard to their reliability and validity. All tools contribute towards the judicious use of evidence syntheses by supporting their systematic development or assessment. However, further progress is needed, particularly with regard to the assessment of content quality. This includes comprehensive operationalization and documentation of the assessment process to ensure reliability and validity, and therefore to enable the effective use of trustworthy guidelines in the health care system.

Electronic supplementary material

The online version of this article (doi:10.1186/1756-0500-7-853) contains supplementary material, which is available to authorized users.

[Management of chronic heart failure - a systematic review of guidelines in the context of the DMP revision]

BACKGROUND

Disease Management Programmes (DMPs) are structured treatment programmes for chronic diseases. The DMP requirements are primarily derived from evidence-based guidelines. DMPs are regularly revised to ensure that they reflect current best practice and medical knowledge. The aim of this study was to assess the need for updating the German DMP module on heart failure by comparing it to relevant guidelines and identifying recommendations that should be revised.

METHODS

We systematically searched for clinical guidelines on heart failure published in German, English or French, and extracted relevant guideline recommendations. All included guidelines were assessed for methodological quality. To identify revision needs in the DMP, we performed a synoptic analysis of the extracted guideline recommendations and DMP requirements.

RESULTS

27 guidelines were included. The extracted recommendations covered all aspects of the management of heart failure. The comparison of guideline recommendations with DMP requirements showed that, overall, guideline recommendations were more detailed than DMP requirements, and that the guidelines covered topics not included in the DMP module.

CONCLUSIONS

The DMP module is largely consistent with current guidelines on heart failure. We did not identify any need for significant revision of the DMP requirements. However, some specific recommendations of the DMP module could benefit from revision.

[Methods of health economic evaluation for health services research]

On August 30, 2010, the German Network for Health Services Research [Deutsches Netzwerk Versorgungsforschung e. V. (DNVF e. V.)] approved the Memorandum III "Methods for Health Services Research", supported by the member societies mentioned as authors and published in this Journal [Gesundheitswesen 2010; 72: 739-748]. The present paper focuses on methodological issues of economic evaluation of health care technologies. It complements the Memorandum III "Methods for Health Services Research", part 2. First, general methodological principles of the economic evaluations of health care technologies are outlined. In order to adequately reflect costs and outcomes of health care interventions in the routine health care, data from different sources are required (e. g., comparative efficacy or effectiveness studies, registers, administrative data, etc.). Therefore, various data sources, which might be used for economic evaluations, are presented, and their strengths and limitations are stated. Finally, the need for methodological advancement with regard to data collection and analysis and issues pertaining to communication and dissemination of results of health economic evaluations are discussed.

[The curriculum Health Technology Assessment (HTA)]

Information derived from Health Technology Assessment (HTA) reports is increasingly used for decision-making in healthcare. The production of HTA reports requires well-qualified interdisciplinarily working specialists who are not yet available in sufficient number. The German HTA Society and the German Network for Evidence Based Medicine have now developed an HTA curriculum which may serve as a basis for training courses for users of HTA information as well as HTA authors. Structure and content of the curriculum were inspired by internationally established training courses. However, the specific aspects of technology regulation and decision-making in the healthcare systems of German-speaking countries are emphasised. The curriculum is subdivided into a total of 11 modules. These include principles of HTA, status and regulation of technologies, priority setting, management of knowledge and information, methods of conducting HTA reports and project management. In the next step, the application of the curriculum into an Internet-based training course is planned.

European joint assessments and coordination of findings and resources. Working group 3 report

The aim of Working Group 3 has been to identify possible joint assessments and to coordinate findings and existing resources within the community to support joint assessments.

Best practice in undertaking and reporting health technology assessments. Working group 4 report

The aim of Working Group 4 has been to develop and disseminate best practice in undertaking and reporting assessments, and to identify needs for methodologic development.

The contribution of handsearching European general health care journals to the Cochrane Controlled Trials Register

A fundamental aim of any systematic review is that all relevant studies should be identified and considered for inclusion. Limitations with searching bibliographic databases led the Cochrane Collaboration to search journals by hand for reports of trials. This article presents the results of a 3-year project to identify and make accessible reports of randomized trials published in European general health care journals. Overall, 21,620 reports of controlled trials were identified from 119 journals from 16 countries. More than three quarters (76%) were published in U.K. or German journals. Only 3,640 (17%) reports were indexed in MEDLINE as controlled trials, and 6,554 (30%) were not indexed in MEDLINE at all. Bibliographic details for all reports are available by searching The Cochrane Controlled Trials Register in The Cochrane Library. This project has ensured that a large proportion of trial reports not previously identifiable has been made accessible to those preparing systematic reviews.

[The concept of "evidence-based medicine" in psychiatry. A path to a more rational psychiatry?]

Many of the diagnostic and therapeutic procedures in psychiatry and psychotherapy are based on opinion rather than evidence. The concept of evidence-based medicine aims to bridge the gap between clinical research and clinical decision-making by integrating the best available external evidence with personal expertise. In this article, we demonstrate several examples of the non-evidence-based medicine paradigm. Then we show the usefulness and practicability of the new evidence-based medicine paradigm by using a clinical example. Finally, we discuss the consequences, chances, and limitations of this new model. We also examine the role of the individual clinician's viewpoint as well as the need of institutional re-engineering and the possible restructuring of the entire health care system towards evidence-based methods.

Blockage of longitudinal flow in endolymphatic hydrops

The external shunt operation on the endolymphatic sac is based on the concept that it will drain excess endolymph and thus arrest the progression of endolymphatic hydrops. We performed histological studies on 46 temporal bones from 13 cases of bilateral and 20 cases of unilateral Meniere's disease to evaluate the status of the pathways of longitudinal flow of endolymph to the sac. The endolymphatic ducts were blocked in 8 specimens (17%), the endolymphatic sinuses in 9 (19.5%), the utricular ducts in 12 (26%), the saccular ducts in 7 (15%), and the ductus reuniens in 27 (59%). These blockages arrested longitudinal flow from both the pars superior and inferior in 21 cases (46%), the pars superior only in 3 (6.5%), and the pars inferior only in 16 (35%). In the 6 ears with all pathways open, 2 were found to have fistulae between the saccules and the perilymphatic spaces, which theoretically results in internal shunting, thus alleviating the need for the external shunt procedure. In the aggregate, therefore, 42 (91%) of 46 ears showed either areas of blockage of longitudinal flow or internal shunts that would theoretically negate the value of external endolymphatic shunt procedures. These temporal bones, however, are from an autopsy population and the severity of pathological changes is probably greater than it would have been at an earlier age when external shunt surgery might have been a therapeutic consideration.