Assessment of publication bias for the surgeon scientist

A Review of Meta-Analyses in Plastic Surgery: Need for Adequate Assessment of Publication Bias

INTRODUCTION

Publication bias describes a phenomenon in which significant positive results have a higher likelihood of being published compared to negative or nonsignificant results. Publication bias can confound the estimated therapeutic effect in meta-analyses and needs to be adequately assessed in the surgical literature.

METHODS

A review of meta-analyses published in five plastic surgery journals from 2002 to 2022 was conducted. The inclusion criteria for meta-analyses were factors that demonstrated an obligation to assess publication bias, such as interventions with comparable treatment groups and enough power for statistical analysis. Acknowledgment of publication bias risk, quality of bias assessment, methods used in assessment, and individual article factors were analyzed.

RESULTS

318 unique meta-analyses were identified in literature search, and after full-text reviews, 143 met the inclusion criteria for obligation to assess publication bias. 64% of eligible meta-analyses acknowledged the confounding potential of publication bias, and only 46% conducted a formal assessment. Of those who conducted an assessment, 49% used subjective inspection of funnel plots alone, while 47% used any statistical testing in analysis. Overall, only 9/143 (6.3%) assessed publication bias and attempted to correct for its effect. Journals with a higher average impact factor were associated with mention and assessment of publication bias, but more recent publication year and higher number of primary articles analyzed were not.

CONCLUSIONS

This review identified low rates of proper publication bias assessment in meta-analyses published in five major plastic surgery journals. Assessment of publication bias using objective statistical testing is necessary to ensure quality literature within surgical disciplines.

The Necessity of Implant Removal after Fixation of Thoracolumbar Burst Fractures—A Systematic Review

Background: Thoracolumbar burst fractures are a common traumatic vertebral fracture in the spine, and pedicle screw fixation has been widely performed as a safe and effective procedure. However, after the stabilization of the thoracolumbar burst fractures, whether or not to remove the pedicle screw implant remains controversial. This review aimed to assess the benefits and risks of pedicle screw instrument removal after fixation of thoracolumbar burst fractures. Methods: Data sources, including PubMed, EMBASE, Cochrane Library, Web of Science, Google Scholar, and Clinical trials.gov, were comprehensively searched. All types of human studies that reported the benefits and risks of implant removal after thoracolumbar burst fractures, were selected for inclusion. Clinical outcomes after implant removal were collected for further evaluation. Results: A total of 4051 papers were retrieved, of which 35 studies were eligible for inclusion in the review, including four case reports, four case series, and 27 observational studies. The possible risks of pedicle screw removal after fixation of thoracolumbar burst fractures include the progression of the kyphotic deformity and surgical complications (e.g., surgical site infection, neurovascular injury, worsening pain, revision surgery), while the potential benefits of pedicle screw removal mainly include improved segmental range of motion and alleviated pain and disability. Therefore, the potential benefits and possible risks should be weighed to support patient-specific clinical decision-making about the removal of pedicle screws after the successful fusion of thoracolumbar burst fractures. Conclusions: There was conflicting evidence regarding the benefits and harms of implant removal after successful fixation of thoracolumbar burst fractures, and the current literature does not support the general recommendation for removal of the pedicle screw instruments, which may expose the patients to unnecessary complications and costs. Both surgeons and patients should be aware of the indications and have appropriate expectations of the benefits and risks of implant removal. The decision to remove the implant or not should be made individually and cautiously by the surgeon in consultation with the patient. Further studies are warranted to clarify this issue. Level of evidence: level 1.

Development and validation of a recommended checklist for assessment of surgical videos quality: the LAParoscopic surgery Video Educational GuidelineS (LAP-VEGaS) video assessment tool

INTRODUCTION

There has been a constant increase in the number of published surgical videos with preference for open-access sources, but the proportion of videos undergoing peer-review prior to publication has markedly decreased, raising questions over quality of the educational content presented. The aim of this study was the development and validation of a standard framework for the appraisal of surgical videos submitted for presentation and publication, the LAParoscopic surgery Video Educational GuidelineS (LAP-VEGaS) video assessment tool.

METHODS

An international committee identified items for inclusion in the LAP-VEGaS video assessment tool and finalised the marking score utilising Delphi methodology. The tool was finally validated by anonymous evaluation of selected videos by a group of validators not involved in the tool development.

RESULTS

9 items were included in the LAP-VEGaS video assessment tool, with every item scoring from 0 (item not presented in the video) to 2 (item extensively presented in the video), with a total marking score ranging from 0 to 18. The LAP-VEGaS video assessment tool resulted highly accurate in identifying and selecting videos for acceptance for conference presentation and publication, with high level of internal consistency and generalisability.

CONCLUSIONS

We propose that peer review in adherence to the LAP-VEGaS video assessment tool could enhance the overall quality of published video outputs.

LAP-VEGaS Practice Guidelines for Reporting of Educational Videos in Laparoscopic Surgery: A Joint Trainers and Trainees Consensus Statement

OBJECTIVE

Consensus statement by an international multispecialty trainers and trainees expert committee on guidelines for reporting of educational videos in laparoscopic surgery.

SUMMARY OF BACKGROUND DATA

Instructive laparoscopy videos with appropriate exposition could be ideal for initial training in laparoscopic surgery, but there are no guidelines for video annotation or procedural educational and safety evaluation.

METHODS

Delphi questionnaire of 45 statements prepared by a steering group and voted on over 2 rounds by committee members using an electronic survey tool. Committee selection design included representative surgical training experts worldwide across different laparoscopic specialties, including general surgery, lower and upper gastrointestinal surgery, gynecology and urology, and a proportion of aligned surgical trainees.

RESULTS

All 33 committee members completed both the first and the second round of the Delphi questionnaire related to 7 major domains: Video Introduction/Authors' information; Patient Details; Procedure Description; Procedure Outcome; Associated Educational Content; Peer Review; and Use in Educational Curriculae. The 17 statements that did not reach at least 80% agreement after the first round were revised and returned into the second round. The committee consensus approved 37 statements to at least an 82% agreement.

CONCLUSION

Consensus guidelines on how to report laparoscopic surgery videos for educational purposes have been developed. We anticipate that following our guidelines could help to improve video quality.These reporting guidelines may be useful as a standard for reviewing videos submitted for publication or conference presentation.

Medication adherence interventions that target subjects with adherence problems: Systematic review and meta-analysis

BACKGROUND

Inadequate medication adherence is a pervasive, under-recognized cause of poor health outcomes. Many intervention trials designed to improve medication adherence have targeted adults with adherence problems. No previous reviews have synthesized the effectiveness of medication adherence interventions focused on subjects with medication adherence difficulties.

OBJECTIVE

This systematic review and meta-analysis synthesized findings from medication adherence intervention studies conducted among adults with medication adherence difficulties.

METHODS

Primary research studies were eligible for inclusion if they tested an intervention designed to increase medication adherence among adults with documented adherence difficulties and reported medication adherence behavior outcomes. Comprehensive search strategies of 13 computerized databases, author and ancestry searches, and hand searches of 57 journals were used to locate eligible primary research. Participant demographics, intervention characteristics, and methodological features were reliably coded from reports along with medication adherence outcomes. Effect sizes for outcomes were calculated as standardized mean differences, and random effects models were used to estimate overall mean effects. Exploratory dichotomous and continuous variable moderator analyses were employed to examine potential associations between medication adherence effect size and sample, intervention, and methodological characteristics.

RESULTS

Data were extracted from 53 reports of studies involving 8243 individual primary study participants. The overall standardized mean difference effect size for treatment vs. control subjects was 0.301. For treatment pre- vs. post-intervention comparisons, the overall effect size was 0.533. Significantly larger effect sizes were associated with interventions incorporating prompts to take medications than interventions lacking medication prompts (0.497 vs. 0.234). Larger effect sizes were also found for interventions that linked medication taking with existing habits compared to interventions that did not (0.574 vs. 0.222). Effect sizes were largest among studies that measured adherence by pill counts or electronic event monitoring systems. Analysis of study design features identified several potential risks of bias. Statistically significant publication bias was detected, but adherence effect sizes were not significantly associated with other risks of bias.

CONCLUSIONS

These findings document that interventions targeting individuals with medication adherence problems can have modest but significant effects on medication-taking behavior. The findings support the use of behavioral strategies such as prompts and linking medications to habits to increase medication adherence in adults with adherence challenges. Face-to-face interventions appear to be critical for patients who have experienced past problems with medication adherence.

Hand suture versus stapler for closure of loop ileostomy--a systematic review and meta-analysis of randomized controlled trials

PURPOSE

The aims of this study are to compare the 30-day rate of bowel obstruction for stapled vs. handsewn closure of loop ileostomy, and to further assess efficacy and safety for each technique by secondary endpoints such as operative time, rates of anastomotic leakage, and other post-operative complications within 30 days.

METHODS

A systematic literature search (MEDLINE, The Cochrane Library, EMBASE and ISI Web of Science) was performed to identify randomized controlled trials (RCTs) comparing stapled and handsewn closure of loop ileostomy after low anterior resection. Random effects meta-analyses were calculated and presented as risk ratio (RR) and mean difference (MD) with corresponding 95 % confidence intervals.

RESULTS

Forty publications were retrieved and 4 RCTs (649 patients) were included. There was methodological and clinical heterogeneity of included trials, but statistical heterogeneity was low for most endpoints. Stapler use significantly reduced the rate of bowel obstruction compared to hand-sewn closure (RR 0.53 [0.32, 0.88]; P = 0.01). The operation time was significantly lower for stapling compared to hand suture (MD -15.5 min [-18.4, 12.6]; P < 0.001). All other secondary outcomes did not show significant differences.

CONCLUSIONS

This meta-analysis shows superiority of stapled closure of loop ileostomy compared to handsewn closure in terms of bowel obstruction rate and mean operation time. Other relevant complications such as anastomotic leakage are equivalent. Even so, both techniques are options with opposing advantages and disadvantages.

Meta-epidemiology

The concept of meta-epidemiology has been introduced with considering the methodological limitations of systematic review for intervention trials. The paradigm of meta-epidemiology has shifted from a statistical method into a new methodology to close gaps between evidence and practice. Main interest of meta-epidemiology is to control potential biases in previous quantitative systematic reviews and draw appropriate evidences for establishing evidence-base guidelines. Nowadays, the network meta-epidemiology was suggested in order to overcome some limitations of meta-epidemiology. To activate meta-epidemiologic studies, implementation of tools for risk of bias and reporting guidelines such as the Consolidated Standards for Reporting Trials (CONSORT) should be done.

Acellular vaccines for preventing whooping cough in children

BACKGROUND

Routine use of whole-cell pertussis (wP) vaccines was suspended in some countries in the 1970s and 1980s because of concerns about adverse effects. Following this action, there was a resurgence of whooping cough. Acellular pertussis (aP) vaccines, containing purified or recombinant Bordetella pertussis (B. pertussis) antigens, were developed in the hope that they would be as effective, but less reactogenic than the whole-cell vaccines. This is an update of a Cochrane review first published in 1999, and previously updated in 2012. In this update, we included no new studies.

OBJECTIVES

To assess the efficacy and safety of acellular pertussis vaccines in children and to compare them with the whole-cell vaccines.

SEARCH METHODS

We searched CENTRAL (2013, Issue 12), MEDLINE (1950 to January week 2, 2014), EMBASE (1974 to January 2014), Biosis Previews (2009 to January 2014) and CINAHL (2009 to January 2014).

SELECTION CRITERIA

We selected double-blind randomised efficacy and safety trials of aP vaccines in children up to six years old, with active follow-up of participants and laboratory verification of pertussis cases.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data and assessed the risk of bias in the studies. Differences in trial design precluded a meta-analysis of the efficacy data. We pooled the safety data from individual trials using a random-effects meta-analysis model.

MAIN RESULTS

We included six efficacy trials with a total of 46,283 participants and 52 safety trials with a total of 136,541 participants. Most of the safety trials did not report the methods for random sequence generation, allocation concealment and blinding, which made it difficult to assess the risk of bias in the studies. The efficacy of multi-component (≥ three) vaccines varied from 84% to 85% in preventing typical whooping cough (characterised by 21 or more consecutive days of paroxysmal cough with confirmation of B. pertussis infection by culture, appropriate serology or contact with a household member who has culture-confirmed pertussis), and from 71% to 78% in preventing mild pertussis disease (characterised by seven or more consecutive days of cough with confirmation of B. pertussis infection by culture or appropriate serology). In contrast, the efficacy of one- and two-component vaccines varied from 59% to 78% against typical whooping cough and from 41% to 58% against mild pertussis disease. Multi-component acellular vaccines are more effective than low-efficacy whole-cell vaccines, but may be less effective than the highest-efficacy whole-cell vaccines. Most systemic and local adverse events were significantly less common with aP vaccines than with wP vaccines for the primary series as well as for the booster dose.

AUTHORS' CONCLUSIONS

Multi-component (≥ three) aP vaccines are effective in preventing whooping cough in children. Multi-component aP vaccines have higher efficacy than low-efficacy wP vaccines, but they may be less efficacious than the highest-efficacy wP vaccines. Acellular vaccines have fewer adverse effects than whole-cell vaccines for the primary series as well as for booster doses.

Acellular vaccines for preventing whooping cough in children

BACKGROUND

Routine use of whole-cell pertussis (wP) vaccines was suspended in some countries in the 1970s and 1980s because of concerns about adverse effects. Following such action, there was a resurgence of whooping cough. Acellular pertussis (aP) vaccines, containing purified or recombinant Bordetella pertussis (B. pertussis) antigens, were developed in the hope that they would be as effective, but less reactogenic than the whole-cell vaccines.

OBJECTIVES

To assess the efficacy and safety of acellular pertussis vaccines in children.

SEARCH METHODS

We searched the Cochrane Register of Controlled Trials (CENTRAL) (The Cochrane Library 2011, Issue 4) which contains the Cochrane Acute Respiratory Infections Group's Specialised Register, MEDLINE (1950 to December week 4, 2011), EMBASE (1974 to January 2012), Biosis Previews (2009 to January 2012), and CINAHL (2009 to January 2012).

SELECTION CRITERIA

We selected double-blind randomised efficacy and safety trials of aP vaccines in children up to six years old, with active follow-up of participants and laboratory verification of pertussis cases.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data and assessed the risk of bias in the studies. Differences in trial design precluded a meta-analysis of the efficacy data. We pooled the safety data from individual trials using a random-effects meta-analysis model.

MAIN RESULTS

We included six efficacy trials with a total of 46,283 participants and 52 safety trials with a total of 136,541 participants. Most of the safety trials did not report the methods for random sequence generation, allocation concealment and blinding, which made it difficult to assess the risk of bias in the studies. The efficacy of multi-component (≥ three) vaccines varied from 84% to 85% in preventing typical whooping cough (characterised by 21 or more consecutive days of paroxysmal cough with confirmation of B. pertussis infection by culture, appropriate serology or contact with a household member who has culture-confirmed pertussis), and from 71% to 78% in preventing mild pertussis disease (characterised by seven or more consecutive days of cough with confirmation of B. pertussis infection by culture or appropriate serology). In contrast, the efficacy of one- and two-component vaccines varied from 59% to 75% against typical whooping cough and from 13% to 54% against mild pertussis disease. Multi-component acellular vaccines are more effective than low-efficacy whole-cell vaccines, but may be less effective than the highest-efficacy whole-cell vaccines. Most systemic and local adverse events were significantly less common with aP vaccines than with wP vaccines for the primary series as well as for the booster dose.

AUTHORS' CONCLUSIONS

Multi-component (≥ three) aP vaccines are effective and show less adverse effects than wP vaccines for the primary series as well as for booster doses.

Diagnostic accuracy and receiver-operating characteristics curve analysis in surgical research and decision making

In surgical research, the ability to correctly classify one type of condition or specific outcome from another is of great importance for variables influencing clinical decision making. Receiver-operating characteristic (ROC) curve analysis is a useful tool in assessing the diagnostic accuracy of any variable with a continuous spectrum of results. In order to rule a disease state in or out with a given test, the test results are usually binary, with arbitrarily chosen cut-offs for defining disease versus health, or for grading of disease severity. In the postgenomic era, the translation from bench-to-bedside of biomarkers in various tissues and body fluids requires appropriate tools for analysis. In contrast to predetermining a cut-off value to define disease, the advantages of applying ROC analysis include the ability to test diagnostic accuracy across the entire range of variable scores and test outcomes. In addition, ROC analysis can easily examine visual and statistical comparisons across tests or scores. ROC is also favored because it is thought to be independent from the prevalence of the condition under investigation. ROC analysis is used in various surgical settings and across disciplines, including cancer research, biomarker assessment, imaging evaluation, and assessment of risk scores.With appropriate use, ROC curves may help identify the most appropriate cutoff value for clinical and surgical decision making and avoid confounding effects seen with subjective ratings. ROC curve results should always be put in perspective, because a good classifier does not guarantee the expected clinical outcome. In this review, we discuss the fundamental roles, suggested presentation, potential biases, and interpretation of ROC analysis in surgical research.

Acellular vaccines for preventing whooping cough in children

BACKGROUND

Routine use of whole-cell pertussis vaccines was suspended in some countries in the 1970s/1980s because of concerns about adverse effects. There was a resurgence of whooping cough. Acellular pertussis vaccines (containing purified or recombinant Bordetella pertussis antigens) were developed in the hope that they would be as effective but less reactogenic than the whole-cell vaccines.

OBJECTIVES

To assess the efficacy and safety of acellular pertussis vaccines in children.

SEARCH STRATEGY

We searched the Cochrane Register of Controlled Trials (CENTRAL) (The Cochrane Library 2009, issue 2) which contains the Acute Respiratory Infections Group's Specialised Register; MEDLINE (1950 to April week 2 2009) and EMBASE (1974 to April 2009).

SELECTION CRITERIA

Double-blind randomised efficacy and safety trials of acellular pertussis vaccines in children up to six years old, with active follow-up of participants and laboratory verification of pertussis cases.

DATA COLLECTION AND ANALYSIS

Two review authors independently performed data extraction and study quality assessment. Differences in trial design precluded pooling of the efficacy data. The safety data from individual trials were pooled using the Cochrane statistical package Review Manager 5.

MAIN RESULTS

Six efficacy trials and 52 safety trials were included. The efficacy of multi-component (≥ 3) vaccines varied from 84% to 85% in preventing typical whooping cough, and from 71% to 78% in preventing mild pertussis disease. In contrast, the efficacy of one- and two-component vaccines varied from 59% to 75% against typical whooping cough, and from 13% to 54% against mild pertussis disease. Multi-component acellular vaccines is more effective than low-efficacy whole-cell vaccines, but may be less effective than the highest-efficacy whole-cell vaccines. Most systemic and local adverse events were significantly less common with acellular than with whole-cell pertussis vaccines for the primary series as well as for the booster dose.

AUTHORS' CONCLUSIONS

Multi-component acellular pertussis vaccines are effective, and show less adverse effects than whole-cell pertussis vaccines for the primary series as well as for booster doses.

Meta-analysis of the effect of peritoneal lavage on survival in experimental peritonitis

Background

Morbidity and mortality associated with bacterial peritonitis remain a challenge for contemporary surgery. Despite great surgical improvements, death rates have not improved. A secondary debate concerns the volume and nature of peritoneal lavage or washout—what volume, what carrier and what, if any, antibiotic or antiseptic?

Methods

A literature search of experimental studies assessing the effect of peritoneal lavage following peritonitis was conducted using Medline, EMBASE and Cochrane databases. Twenty-three trials met predetermined inclusion criteria. Data were pooled and relative risks calculated.

Results

In an experimental peritonitis setting a mortality rate of 48·9 per cent (238 of 487) was found for saline lavage compared with 16·4 per cent (106 of 647) for antibiotic lavage (absolute risk reduction (ARR) 32·5 (95 per cent confidence interval (c.i.) 27·1 to 37·7) per cent; (P &lt; 0·001). An ARR of 25·0 (95 per cent c.i. 17·9 to 31·7) per cent P &lt; 0·001) was found for the use of saline compared with no lavage at all. The survival benefit persisted regardless of systemic antibiotic therapy. Antiseptic lavage was associated with a very high mortality rate (75·0 per cent).

Conclusion

Pooled data from studies in experimental peritonitis demonstrated a significant reduction in mortality with antibiotic lavage.

Publication bias in oral and maxillofacial surgery journals: an observation on published controlled trials

BACKGROUND

Publication bias (PB) diminishes the full distribution of research, distorts and discredits the scientific record, and thus compromises evidence-based practice. The objective of this study was to analyse published controlled trials with regard to PB in leading oral and maxillofacial surgery (OMS) journals.

METHODS

All controlled trials published in the International Journal of Oral and Maxillofacial Surgery, Journal of Cranio-Maxillofacial Surgery, Journal of Oral and Maxillofacial Surgery, and British Journal of Oral and Maxillofacial Surgery in 2008 were analysed for a primary outcome, country of authors, sample size, gender of the first author, funding source and location of the study.

RESULTS

Of 952 published articles, 53 controlled trials (5.7%) were identified. The OMS journals preferentially published controlled trials with a positive outcome (77.4%) and from high-income countries (73.6%). Single-centred trials (86.8%) with low sample size (n<100; 69.8%) were published more frequently. The majority of the first authors were male (75.5%). Funding source disclosure was missing in most studies (73.6%) [corrected].

CONCLUSIONS

Our results suggest the possible existence of PB in the OMS literature. Hence, it should be borne in mind that the published articles may not be representative of all scientific works, especially when systematic reviews and meta-analyses are conducted or read. In the meantime, journals should establish measures to eliminate PB to uphold scientific integrity. However, this study was an observation based on the published articles. An analysis of all submitted manuscripts would provide more accurate estimates of PB. Ethical considerations on PB are also discussed.

WITHDRAWN: Acellular vaccines for preventing whooping cough in children

BACKGROUND

Routine use of whole cell pertussis vaccines was suspended in some countries in the late 1970s and early 1980s, leading to a resurgence of whooping cough. Acellular pertussis vaccines containing purified or recombinant Bordetella pertussis antigens were developed in the hope that they would be as effective but less toxic than the whole cell vaccines.

OBJECTIVES

The objective of this review was to assess the effects of acellular pertussis vaccines in children.

SEARCH STRATEGY

The Cochrane Controlled Trials Register and Medline were searched up to January 1998.

SELECTION CRITERIA

Double-blind randomised efficacy and safety trials of acellular pertussis vaccines in children, with active follow-up of participants and laboratory verification of pertussis cases.

DATA COLLECTION AND ANALYSIS

One reviewer assessed trial quality and extracted data.

MAIN RESULTS

Six efficacy trials and 45 safety trials were included. Acellular pertussis vaccines with three or more pertussis vaccines were more effective than those with one or two antigens. They were also more effective than one type of whole cell pertussis vaccine, but less effective than two other types of whole cell vaccines. Differences in trial design precluded pooling of the efficacy data and results should be interpreted with caution. Most systemic and local adverse events were significantly less common with acellular than with whole cell pertussis vaccines.

AUTHORS' CONCLUSIONS

Multi-component acellular pertussis vaccines are effective, and show less adverse effects than whole cell pertussis vaccines. However in areas where whooping cough is more likely to be fatal, the higher toxicity of some whole cell vaccines may be offset by their increased effectiveness.