Quality assessment of economic evaluation studies in pediatric surgery: a systematic review

Methodologic and Reporting Quality of Economic Evaluations in Hand and Wrist Surgery: A Systematic Review

BACKGROUND

Economic evaluations can inform decision-making; however, previous publications have identified poor quality of economic evaluations in surgical specialties.

METHODS

Study periods were from January 1, 2006, to April 20, 2020 (methodologic quality) and January 1, 2014, to April 20, 2020 (reporting quality). Primary outcomes were methodologic quality [Guidelines for Authors and Peer Reviewers of Economic Submissions to The BMJ (Drummond's checklist), 33 points; Quality of Health Economic Studies (QHES), 100 points; Consensus on Health Economic Criteria (CHEC), 19 points] and reporting quality (Consolidated Health Economic Evaluation Standards (CHEERS) statement, 24 points).

RESULTS

Forty-seven hand economic evaluations were included. Partial economic analyses (i.e., cost analysis) were the most common (n = 34; 72 percent). Average scores of full economic evaluations (i.e., cost-utility analysis and cost-effectiveness analysis) were: Drummond's checklist, 27.08 of 33 (82.05 percent); QHES, 79.76 of 100 (79.76 percent); CHEC, 15.54 of 19 (81.78 percent); and CHEERS, 20.25 of 24 (84.38 percent). Cost utility analyses had the highest methodologic and reporting quality scores: Drummond's checklist, 28.89 of 35 (82.54 percent); QHES, 86.56 of 100 (86.56 percent); CHEC, 16.78 of 19 (88.30 percent); and CHEERS, 20.8 of 24 (86.67 percent). The association (multiple R) between CHEC and CHEERS was strongest: CHEC, 0.953; Drummond's checklist, 0.907; and QHES, 0.909.

CONCLUSIONS

Partial economic evaluations in hand surgery are prevalent but not very useful. The Consensus on Health Economic Criteria and Consolidated Health Economic Evaluation Standards should be used in tandem when undertaking and evaluating economic evaluation in hand surgery.

A systematic review of economic evaluation in pancreatic ductal adenocarcinoma

OBJECTIVES

The economic evaluation (EE) of healthcare interventions has become a necessity. However, high quality needs to be ensured in order to achieve validated results and help making informed decisions. Thus, the objective of the present study was to systematically identify and review published pancreatic ductal adenocarcinoma-related EEs and to assess their quality.

METHODS

Systematic literature research was conducted in PubMed and Cochrane to identify published EEs between 2000 and 2015. The quality of each selected EE was assessed by two independent reviewers, using the Drummond's checklist.

RESULTS

Our systematic review was based on 32 EEs and showed a wide variety of methodological approaches, including different perspectives, time horizon, and cost effectiveness analyses. Nearly two-thirds of EEs are full EEs (n = 21), and about one-third of EEs had a Drummond score ≥7, synonymous with 'high quality'. Close to 50% of full EEs had a Drummond score ≥7, whereas all of partial EEs had a Drummond score <7 (n = 11).

CONCLUSIONS

Over the past 15 years, a lot of interest has been evinced over the EE of pancreatic ductal adenocarcinoma (PDAC) and its direct impact on therapeutic advances in PDAC. To provide a framework for health care decision-making, to facilitate transferability and to lend credibility to health EEs, their quality must be improved. For the last 4 years, a tendency towards a quality improvement of these studies has been observed, probably coupled with a context of rational decision-making in health care, a better and wider spread of recommendations and thus, medical practitioners' full endorsement.

A systematic review of the quality of conduct and reporting of systematic reviews and meta-analyses in paediatric surgery

OBJECTIVE

Our objective was to evaluate quality of conduct and reporting of published systematic reviews and meta-analyses in paediatric surgery. We also aimed to identify characteristics predictive of review quality.

BACKGROUND

Systematic reviews summarise evidence by combining sources, but are potentially prone to bias. To counter this, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was published to aid in reporting. Similarly, the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) measurement tool was designed to appraise methodology. The paediatric surgical literature has seen an increasing number of reviews over the past decade, but quality has not been evaluated.

METHODS

Adhering to PRISMA guidelines, we performed a systematic review with a priori design to identify systematic reviews and meta-analyses of interventions in paediatric surgery. From 01/2010 to 06/2016, we searched: MEDLINE, EMBASE, Cochrane, Centre for Reviews and Dissemination, Web of Science, Google Scholar, reference lists and journals. Two reviewers independently selected studies and extracted data. We assessed conduct and reporting using AMSTAR and PRISMA. Scores were calculated as the sum of reported items. We also extracted author, journal and article characteristics, and used them in exploratory analysis to determine which variables predict quality.

RESULTS

112 articles fulfilled eligibility criteria (53 systematic reviews; 59 meta-analyses). Overall, 68% AMSTAR and 56.8% PRISMA items were reported adequately. Poorest scores were identified with regards a priori design, inclusion of structured summaries, including the grey literature, citing excluded articles and evaluating bias. 13 reviews were pre-registered and 6 in PRISMA-endorsing journals. The following predicted quality in univariate analysis:, word count, Cochrane review, journal h-index, impact factor, journal endorses PRISMA, PRISMA adherence suggested in author guidance, article mentions PRISMA, review includes comparison of interventions and review registration. The latter three variables were significant in multivariate regression.

CONCLUSIONS

There are gaps in the conduct and reporting of systematic reviews in paediatric surgery. More endorsement by journals of the PRISMA guideline may improve review quality, and the dissemination of reliable evidence to paediatric clinicians.

A Systematic Review of the Level of Evidence in Economic Evaluations of Medical Devices: The Example of Vertebroplasty and Kyphoplasty

CONTEXT

Economic evaluations are far less frequently reported for medical devices than for drugs. In addition, little is known about the quality of existing economic evaluations, particularly for innovative devices, such as those used in vertebroplasty and kyphoplasty.

OBJECTIVE

To assess the level of evidence provided by the available economic evaluations for vertebroplasty and kyphoplasty.

DATA SOURCES

A systematic review of articles in English or French listed in the MEDLINE, PASCAL, COCHRANE and National Health Service Economic Evaluation databases, with limits on publication date (up to the date of the review, March 2014).

STUDY SELECTION

We included only economic evaluations of vertebroplasty or kyphoplasty. Editorial and methodological articles were excluded.

DATA EXTRACTION

Data were extracted from articles by two authors working independently and using two analysis grids to measure the quality of economic evaluations.

DATA SYNTHESIS

Twenty-one studies met our inclusion criteria. All were published between 2008 and 2014. Eighteen (86%) were full economic evaluations. Cost-effectiveness analysis (CEA) was the most frequent type of economic evaluation, and was present in 11 (52%) studies. Only three CEAs complied fully with the British Medical Journal checklist. The quality of the data sources used in the 21 studies was high, but the CEAs conforming to methodological guidelines did not use high-quality data sources for all components of the analysis.

CONCLUSIONS

This systematic review shows that the level of evidence in economic evaluations of vertebroplasty and kyphoplasty is low, despite the recent publication of a large number of studies. This finding highlights the challenges to be faced to improve the quality of economic evaluations of medical devices.