Economic analyses of radiological procedures: a methodological evaluation of the medical literature

Traumatic brain injury: methodological approaches to estimate health and economic outcomes

The effort to standardize the methodology and adherence to recommended principles for all economic evaluations has been emphasized in medical literature. The objective of this review is to examine whether economic evaluations in traumatic brain injury (TBI) research have been compliant with existing guidelines. Medline search was performed between January 1, 1995 and August 11, 2012. All original TBI-related full economic evaluations were included in the study. Two authors independently rated each study's methodology and data presentation to determine compliance to the 10 methodological principles recommended by Blackmore et al. Descriptive analysis was used to summarize the data. Inter-rater reliability was assessed with Kappa statistics. A total of 28 studies met the inclusion criteria. Eighteen of these studies described cost-effectiveness, seven cost-benefit, and three cost-utility analyses. The results showed a rapid growth in the number of published articles on the economic impact of TBI since 2000 and an improvement in their methodological quality. However, overall compliance with recommended methodological principles of TBI-related economic evaluation has been deficient. On average, about six of the 10 criteria were followed in these publications, and only two articles met all 10 criteria. These findings call for an increased awareness of the methodological standards that should be followed by investigators both in performance of economic evaluation and in reviews of evaluation reports prior to publication. The results also suggest that all economic evaluations should be made by following the guidelines within a conceptual framework, in order to facilitate evidence-based practices in the field of TBI.

An analysis of the influence of framework aspects on the study design of health economic modeling evaluations

Research and practical guidelines have many implications for how to structure a health economic study. A major focus in recent decades has been the quality of health economic research. In practice, the factors influencing a study design are not limited to the quest for quality. Moreover, the framework of the study is important. This research addresses three major questions related to these framework aspects. First, we want to know whether the design of health economic studies has changed over time. Second, we want to know how the subject of a study, whether it is a process or product innovation, influences the parameters of the study design. Third, one of the most important questions we will answer is whether and how the study's source of funding has an impact on the design of the research. To answer these questions, a total of 234 health economic studies were analyzed using a correspondence analysis and a logistic regression analysis. All three categories of framework factors have an influence on the aspects of the study design. Health economic studies have evolved over time, leading to the use of more advanced methods like complex sensitivity analyses. Additionally, the patient's point of view has increased in importance. The evaluation of product innovations has focused more on utility concepts. On the other hand, the source of funding may influence only a few aspects of the study design, such as the use of evaluation methods, the source of data, and the use of certain utility measures. The most important trends in health care, such as the emphasis on the patients' point of view, become increasingly established in health economic evaluations with the passage of time. Although methodological challenges remain, modern information and communication technologies provide a basis for increasing the complexity and quality of health economic studies if used frequently.

Economic evaluation in radiology: reviewing the literature and examples in oncology

RATIONALE AND OBJECTIVES

To review US health care trends related to medical imaging utilization and costs as well as to present standard methods for conducting economic evaluation for health care interventions and medical imaging specifically.

MATERIALS AND METHODS

A review of the medical literature was performed to assess health policy and health technology assessment trends, expenditures, and cost-effectiveness analysis (CEA) related to medical imaging. Standard approaches to conducting economic evaluation and cost-effectiveness analysis were reviewed and summarized. Examples of CEA evidence related to imaging in select oncology conditions were presented.

RESULTS

Several high-quality methodology publications have provided guidance for conducting economic evaluation and CEA in radiology. There is variability in the quality of CEA models and their dissemination. However, there are numerous methodologically sound cost-effectiveness analyses for radiology procedures, and the evidence base of CEA studies for medical imaging continues to increase. Advanced imaging approaches for diagnosing and staging oncology conditions have the potential to provide cost-effective care when used in appropriate patient subpopulations.

CONCLUSIONS

Additional rigorous comparative effectiveness studies for advanced imaging, including cost-effectiveness analyses, can provide useful information to policy makers and health care providers on the relative effects and costs associated with diagnostic alternatives.

Contrast-enhanced ultrasonography in the characterization of benign focal liver lesions: activity-based cost analysis

PURPOSE

The aim of this study was to perform a cost analysis of contrast-enhanced ultrasonography (CEUS) in the study of benign focal liver lesions (BFLL) with indeterminate appearance on ultrasonography (US).

MATERIALS AND METHODS

A decision model of patients with suspected BFLL on baseline US who subsequently underwent CEUS between 2002 and 2005 was constructed. We analysed the cost effectiveness of CEUS, considering whether or not computed tomography (CT) was necessary for the diagnosis. There were 398 patients with 213 angiomas, 41 focal nodular hyperplasias (FNH) and 154 pseudolesions (focal fatty sparing, focal fatty areas). Each patient underwent CEUS, and 98 of them were also studied by CT. All lesions were followed up.

RESULTS

The cost of a single CEUS examination was 101.51 euros, and that of a single CT scan was 211.48 euros. For diagnosis of haemangiomas, we saved 1,406.97 euros in 2002, 5,315.22 euros in 2003, 10,317.78 euros in 2004 and 9,536.13 euros in 2005. For diagnosis of focal nodular hyperplasias, we saved 781.65 euros in 2003, 781.65 euros in 2004 and 1,406.97 euros in 2005. For diagnosis of pseudolesions, we saved 2,813.94 euros in 2002, 5,158.89 euros in 2003, 5,158.89 euros in 2004 and 4,220.91 euros in 2005. In the period 2002-2005, the introduction of CEUS allowed us to save a total of 47,055.33 euros in the diagnosis of benign focal hepatic liver lesions.

CONCLUSIONS

This cost analysis shows that CEUS is the least expensive second-line modality after baseline US for the diagnosis of BFLL.

Methodologic quality of cost-effectiveness analyses of surgical procedures

OBJECTIVE

The purpose of this study was to evaluate the methodologic quality of economic analyses of surgical procedures and to compare quality across publications.

SUMMARY BACKGROUND DATA

With healthcare resources limited and technologies rapidly advancing across specialties, including surgery, there is increasing demand for evidence of cost-effectiveness.

METHODS

A MEDLINE search identified English-language articles published from 1995 to 2004 that included economic analyses of surgical procedures. Two of the authors reviewed 110 studies and scored each based on compliance with 10 methodologic criteria. Data analyses used Cohen's kappa statistic, regression models, Mann-Whitney U tests, and Kruskal-Wallis tests.

RESULTS

The 110 articles appeared in 79 different journals, including 57 articles in 37 surgical journals. Most journals (75%) had only 1 article eligible for inclusion. The average number of criteria met was 4.1, with 10 articles meeting all 10 methodologic standards. Compliance rates for the 5 methodologic criteria most frequently neglected ranged from 34% to 45% in nonsurgical journals and 9% to 14% in surgical journals (P < 0.001).

CONCLUSIONS

While methodologic guidelines for cost-effectiveness analyses have appeared in the medical literature, studies of cost-effectiveness in surgery often do not meet these criteria. As healthcare policy seeks to incorporate information from economic evaluations, it is increasingly important that surgical journals adhere to accepted guidelines and perform quality assurance on these studies. This may be aided by wider promulgation of the methodologic criteria in surgical journals or at surgical meetings.

Comparative economic analyses of minimally invasive direct coronary artery bypass surgery

OBJECTIVE

This study was undertaken to assess the degree to which published cost comparisons of minimally invasive direct coronary artery bypass through a thoracotomy versus conventional coronary artery bypass grafting, off-pump bypass surgery through a sternotomy, or angioplasty with or without stenting adhered to existing guidelines for performing economic analyses.

METHODS

We used minimally invasive direct coronary artery bypass (MIDCAB), off-pump bypass surgery, cost-effectiveness, economic analysis, and related keywords to search MEDLINE, other literature databases and article reference lists for English-language economic analyses of minimally invasive direct coronary artery bypass procedures versus other procedures that were published from 1990 to February 2002. We critically appraised article adherence to a 10-item methodologic checklist modified to address issues particularly relevant to minimally invasive direct coronary artery bypass evaluations. Assessment discordance was reconciled by consensus.

RESULTS

Ten articles published from June 1997 to March 2001 compared costs and (generally) outcomes of minimally invasive direct coronary artery bypass with those of other procedures. All were nonrandomized comparisons, generally of concurrent intrainstitutional clinical series. Stated results generally favored minimally invasive direct coronary artery bypass, angioplasty, or off-pump bypass surgery through a sternotomy relative to conventional coronary artery bypass grafting. Studies adequately addressed an average of only 24% of applicable checklist items (range 0%-67%). Few studies adequately ensured the comparability of treatment groups, clearly performed intent-to-treat analyses, comprehensively and credibly measured costs that were considered, or clearly addressed costs and results of preprocedural angiography or postprocedural imaging. Only 1 study compared success of revascularization between minimally invasive direct coronary artery bypass and competing alternatives. No studies specified the cost-analysis perspective or included costs of physician or physician assistant care.

CONCLUSIONS

Most published comparative economic analyses of minimally invasive direct coronary artery bypass have failed to adequately address issues crucial to such evaluations. Future studies should more closely follow well-described principles of clinical epidemiology and cost-effectiveness analysis.

Accuracy of terminology and methodology in economic analyses in otolaryngology

OBJECTIVE

Economic studies increasingly guide health care resource allocation decisions. Because rigorous adherence to accepted definitions and research techniques is critical to ensure accuracy, we evaluated the terminology and methods of otolaryngology economic analyses.

STUDY DESIGN

A total of 71 articles published from 1990 to 1999 in 6 peer-reviewed otolaryngology journals with terms such as "cost-effective" in their title or representing economic analyses were reviewed for terminology and use of established methodology guidelines.

RESULTS

Over half (35 of 66) of terms such as "cost-effective" were used incorrectly, and 60% of articles (39 of 64) confused "charge" and "cost" data. Eleven percent (7 of 64) of papers specified the perspective of their analysis. About half (17 of 30) reported a summary measure such as a cost-effectiveness ratio. Only one third (23 of 63) performed sensitivity analyses.

CONCLUSION

Adherence to accepted definitions and research methods is inconsistent, although we did note moderate improvements in making the distinction between costs and charges, defining of study perspective, and performing sensitivity analysis.

SIGNIFICANCE

Greater attention to both terminology and methodology can enhance the quality of economic analyses and ultimately improve certain resource allocation decisions.

Critical issues in the economic evaluation of interventions against communicable diseases

Economic appraisal seeks to provide policy-makers with guidance about how scarce resources can be used to derive the greatest possible social benefit. Its use in the health sector has increased dramatically over the last decade although much of it has been focused on the problems of the more developed countries. The relatively sparse literature on communicable diseases has been dominated by interventions related to HIV/AIDS, hepatitis, malaria and tropical diseases. Reviews of this literature from the perspective of specific conditions such as Hepatitis B are already available, and recently the entire literature has been evaluated against the technical criteria for economic evaluations published in standard textbooks. Accordingly, this paper focuses on issues which would make economic appraisal more useful to policy-makers than it currently is. Given that few countries have the resources to undertake all the necessary analysis in their own settings, it is important that studies in one setting are undertaken in a way that allow generalisability to similar settings. Some of the most important challenges this poses for cost-effectiveness analysis (CEA) are identified. Firstly, incremental analysis is appropriate to local decision making when policy-makers are constrained to keep the current interventions and can consider only marginal improvements. However, it does not allow re-evaluation of existing interventions and is not transferable across settings. A version of Generalised CEA is proposed as an alternative. Secondly, data on costs and effectiveness are often not presented appropriately. The challenge for effectiveness is to adjust the evidence from efficacy studies to allow for different patient or population groups, and local variations in adherence, coverage, and infrastructure. For costs, it is important for studies to report the physical resources used in an intervention as well as unit prices. Thirdly, some long-term effects are still not well incorporated into CEA, especially those affecting child development and drug resistance. These questions are technically challenging and require more concerted efforts over the next few years. Finally, it is important for analysts to provide decision-makers with estimates of the resources that would be required to implement interventions claimed to be cost-effective. These improvements would better enable the evidence from economic analyses to enter the policy debate and be weighed against the other goals and objectives of the health system when allocating scarce resources.

Development of WHO guidelines on generalized cost-effectiveness analysis

The growing use of cost-effectiveness analysis (CEA) to evaluate specific interventions is dominated by studies of prospective new interventions compared with current practice. This type of analysis does not explicitly take a sectoral perspective in which the costs and effectiveness of all possible interventions are compared, in order to select the mix that maximizes health for a given set of resource constraints. WHO guidelines on generalized CEA propose the application of CEA to a wide range of interventions to provide general information on the relative costs and health benefits of different interventions in the absence of various highly local decision constraints. This general approach will contribute to judgements on whether interventions are highly cost-effective, highly cost-ineffective, or something in between. Generalized CEAs require the evaluation of a set of interventions with respect to the counterfactual of the null set of the related interventions, i.e. the natural history of disease. Such general perceptions of relative cost-effectiveness, which do not pertain to any specific decision-maker, can be a useful reference point for evaluating the directions for enhancing allocative efficiency in a variety of settings. The proposed framework allows the identification of current allocative inefficiencies as well as opportunities presented by new interventions.

A critical synopsis of the diagnostic and screening radiology outcomes literature

In summary, the radiology outcomes research literature is both extensive and broad. The methodologic quality, however, is quite variable. Overall, this quality could be improved by intervention in two areas: methodologic dissemination and development. The number of researchers investigating radiology-related outcomes is high, and presently there are over 20 journals devoted exclusively to radiology research. Even with a relatively narrow definition of "outcomes," we identified over 200 radiology outcomes studies, most from the past few years. However, the methodologic quality of most of these articles was relatively low, with important design flaws and biases. Nonetheless, a substantial number of radiology publications do employ state-of-the-art research methods and innovative approaches to methodologic challenges. The quality of radiology outcomes research overall would benefit tremendously from dissemination of such research methods. Instruction in outcomes research methods is accessible to radiologists. For example, there have been several recent articles and series of articles on outcomes research methods in JAMA, including guidelines for the performance and reporting of cost-effectiveness analyses (38-40) and for developing clinical prediction rules (57). Within radiology, several recent articles have appeared on, among other things, cost-effectiveness analysis (34,59,60), assessing quality of life (43), screening for disease (53), and defining the study population (61). The research compendium compiled for the GERRAF (General Electric-Association of University Radiologists Radiology Research Academic Fellowships) program remains a comprehensive methodologic source for many of the issues in radiology outcomes research, and outcomes research methods courses are offered every year at the Society for Health Services Research in Radiology and Society for Medical Decision Making meetings, as well as at the meeting of the Radiological Society of North America. Even so, awareness of the need for such research techniques remains limited. Dissemination of sound research methods is limited at least in part by the current incentives in radiology research. At many institutions, the number of research publications produced, rather than their quality, determines promotion or academic success. Unfortunately, more rigorous study designs often require more time and resources. Further, because peer reviewers are often as uninformed about research methods as the bulk of those who are submitting manuscripts, it may actually be more difficult to publish articles with more advanced methodologic designs. The standard in radiology is the uncontrolled case series, and deviation from the standard may make acceptance for publication more difficult. On a more optimistic note, recent publication of a number of methodology articles suggests that at least some journals are promoting improved research in methodology (43,53,59-61). We hope that time will be available for manuscript reviewers to learn to understand the strengths and weaknesses of various research approaches. If more rigorous study designs were required for publication, radiology outcomes research would probably improve drastically. Nevertheless, the current peer-review system does not effectively promote sound research design. The other great incentive in research is funding. Clearly, if advanced research design is required for funding, then there is incentive for improvement in research quality. Traditionally, National Cancer Institute and other National Institutes of Health and public sector funding has been predicated on a high level of research sophistication. Undoubtedly, availability of grants for diagnostic and screening imaging clinical trials and other research will go far to improve radiology research methods. The other traditional source of research funding is industry.