Randomized clinical trials in the evaluation of surgical innovation

Are preoperative chlorhexidine gluconate showers associated with a reduction in surgical site infection following craniotomy? A retrospective cohort analysis of 3126 surgical procedures

OBJECTIVE

Surgical site infection (SSI) is a complication linked to increased costs and length of hospital stay. Prevention of SSI is important to reduce its burden on individual patients and the healthcare system. The authors aimed to assess the efficacy of preoperative chlorhexidine gluconate (CHG) showers on SSI rates following cranial surgery.

METHODS

In November 2013, a preoperative CHG shower protocol was implemented at the authors' institution. A total of 3126 surgical procedures were analyzed, encompassing a time frame from April 2012 to April 2016. Cohorts before and after implementation of the CHG shower protocol were evaluated for differences in SSI rates.

RESULTS

The overall SSI rate was 0.6%. No significant differences (p = 0.11) were observed between the rate of SSI of the 892 patients in the preimplementation cohort (0.2%) and that of the 2234 patients in the postimplementation cohort (0.8%). Following multivariable analysis, implementation of preoperative CHG showers was not associated with decreased SSI (adjusted OR 2.96, 95% CI 0.67-13.1; p = 0.15).

CONCLUSIONS

This is the largest study, according to sample size, to examine the association between CHG showers and SSI following craniotomy. CHG showers did not significantly alter the risk of SSI after a cranial procedure.

Defining Innovation in Neurosurgery: Results from an International Survey

BACKGROUND

Innovation is a part of the daily practice of neurosurgery. However, a clear definition of what constitutes innovation is lacking and opinions vary from continent to continent, from hospital to hospital, and from surgeon to surgeon.

METHODS

In this study, we distributed an online survey to neurosurgeons from multiple countries to investigate what neurosurgeons consider innovative, by gathering opinions on several hypothetical cases. The anonymous survey consisted of 52 questions and took approximately 10 minutes to complete.

RESULTS

A total of 355 neurosurgeons across all continents excluding Antarctica completed the survey. Neurosurgeons achieved consensus (>75%) in considering specific cases to be innovative, including laser resection of meningioma, focused ultrasonography for tumor, oncolytic virus, deep brain stimulation for addiction, and photodynamic therapy for tumor. Although the new dura substitute case was not considered innovative, there was consensus among neurosurgeons indicating that institutional review board approval was still necessary to maintain ethical standards. Furthermore, although 90% of neurosurgeons considered an oncolytic virus for glioblastoma multiforme to be innovative, only 78% believed that institutional review board approval was necessary before treatment.

CONCLUSIONS

Our results indicate that innovation is a heterogeneous concept among neurosurgeons that necessitates standardization to ensure appropriate patient safety without stifling progress. We discuss both the ethical drawbacks of not having a clear definition of innovation and the challenges in achieving a unified understanding of innovation in neurosurgery and offer suggestions for uniting the field.

Barriers and facilitators experienced in collaborative prospective research in orthopaedic oncology: A qualitative study

Objectives

As tumours of bone and soft tissue are rare, multicentre prospective collaboration is essential for meaningful research and evidence-based advances in patient care. The aim of this study was to identify barriers and facilitators encountered in large-scale collaborative research by orthopaedic oncological surgeons involved or interested in prospective multicentre collaboration.

Methods

All surgeons who were involved, or had expressed an interest, in the ongoing Prophylactic Antibiotic Regimens in Tumour Surgery (PARITY) trial were invited to participate in a focus group to discuss their experiences with collaborative research in this area. The discussion was digitally recorded, transcribed and anonymised. The transcript was analysed qualitatively, using an analytic approach which aims to organise the data in the language of the participants with little theoretical interpretation.

Results

The 13 surgeons who participated in the discussion represented orthopaedic oncology practices from seven countries (Argentina, Brazil, Italy, Spain, Denmark, United States and Canada). Four categories and associated themes emerged from the discussion: the need for collaboration in the field of orthopaedic oncology due to the rarity of the tumours and the need for high level evidence to guide treatment; motivational factors for participating in collaborative research including establishing proof of principle, learning opportunity, answering a relevant research question and being part of a collaborative research community; barriers to participation including funding, personal barriers, institutional barriers, trial barriers, and administrative barriers and facilitators for participation including institutional facilitators, leadership, authorship, trial set-up, and the support of centralised study coordination.

Conclusions

Orthopaedic surgeons involved in an ongoing international randomised controlled trial (RCT) were motivated by many factors to participate. There were a number of barriers to and facilitators for their participation. There was a collective sense of fatigue experienced in overcoming these barriers, which was mirrored by a strong collective sense of the importance of, and need for, collaborative research in this field. The experiences were described as essential educational first steps to advance collaborative studies in this area. Knowledge gained from this study will inform the development of future large-scale collaborative research projects in orthopaedic oncology.

Cite this article: J. S. Rendon, M. Swinton, N. Bernthal, M. Boffano, T. Damron, N. Evaniew, P. Ferguson, M. Galli Serra, W. Hettwer, P. McKay, B. Miller, L. Nystrom, W. Parizzia, P. Schneider, A. Spiguel, R. Vélez, K. Weiss, J. P. Zumárraga, M. Ghert. Barriers and facilitators experienced in collaborative prospective research in orthopaedic oncology: A qualitative study. Bone Joint Res 2017;6:–314. DOI: 10.1302/2046-3758.65.BJR-2016-0192.R1.

Randomized controlled trials in neurosurgery--how good are we?

BACKGROUND

The strongest evidence in medical clinical literature is represented by randomized controlled trials (RCTs). This study was designed to evaluate neurosurgically relevant RCTs published recently by neurosurgeons.

METHOD

A literature search in MEDLINE and EMBASE included all clinical studies published up to 30 June 2006. RCTs with neurosurgical relevance published by at least one author with affiliation to a neurosurgical department were selected. The number and characteristics of individual trials were recorded, and the quality of the trials with regard to study design, quality of reporting, and relevance for clinical practice was assessed by two different investigators using a modification of the Scottish Intercollegiate Guidelines Network methodology checklist. Changes of RCT quality over time as well as factors influencing the quality were analyzed.

FINDINGS

From the initial search results (MEDLINE n = 3,860, EMBASE n = 3,113 articles), 159 RCTs published by neurosurgeons were extracted for final evaluation. Of the RCTs, 62% have been published since 1995; 52% came from the USA, UK, and Germany. The median RCT sample size was 78 patients and the median follow-up 35.7 weeks. Fifty-two percent of all RCTs were of good, 37% of moderate, and 11% of bad quality, with an improvement over time. RCTs with financial funding and RCTs with a sample size of >78 patients were of significantly better quality. There were no major differences in the rating of the studies between the two investigators.

CONCLUSIONS

Only a fraction of neurosurgically relevant literature consists of RCTs, but the quality is satisfying and has significantly improved over the last years. An adequate sample size and sufficient financial support seem to be of substantial importance with regard to the quality of the study. Our data also show that by using a standardized checklist, the quality of trials can be reliably assessed by observers of different experience and educational levels.

The challenges faced in the design, conduct and analysis of surgical randomised controlled trials

Randomised evaluations of surgical interventions are rare; some interventions have been widely adopted without rigorous evaluation. Unlike other medical areas, the randomised controlled trial (RCT) design has not become the default study design for the evaluation of surgical interventions. Surgical trials are difficult to successfully undertake and pose particular practical and methodological challenges. However, RCTs have played a role in the assessment of surgical innovations and there is scope and need for greater use. This article will consider the design, conduct and analysis of an RCT of a surgical intervention. The issues will be reviewed under three headings: the timing of the evaluation, defining the research question and trial design issues. Recommendations on the conduct of future surgical RCTs are made. Collaboration between research and surgical communities is needed to address the distinct issues raised by the assessment of surgical interventions and enable the conduct of appropriate and well-designed trials.

Randomization procedures in orthopaedic trials

Historically, the surgical literature has lacked in reports of randomized clinical trials. This deficit is now being addressed, but the best methods are not always followed. One opportunity for improvement is in the area of randomization. Randomization is of central importance in clinical trials because it reduces bias and represents a basis for ensuring the validity of data analysis using statistical testing. Randomization requires a table of random numbers. Simple randomization is adequate for large trials. Block randomization is a method of balancing equal numbers of patients in each treatment group. Stratification allows balanced distribution of one or more confounding prognostic variables among treatment groups to ensure that groups have similar prognoses (minimizing selection bias). Block randomization and stratification improve validity in trials with fewer patients. Commercially available computer software facilitates randomization.

Randomised controlled trials in orthopaedic surgery and traumatology: overview of parameters and pitfalls

There is a growing consensus that randomised controlled clinical trial (RCT) provide a secure basis for determining treatment effects. Prospective randomised clinical trials can be a powerful tool in medical science and evidence-based medicine. A well-defined study hypothesis, with a prospectively applied study design, blinded and randomised treatment allocation and assessment, with appropriate control groups can provide strong evidence in support of treatment decisions. However, the recent reviews of the medical literature indicate that the study design itself does not ensure the quality of science or useful and valid scientific data. Thus, regardless of the study design or level of evidence, it remains imperative for the physician and surgeon to critically evaluate a scientific report. Moreover, as randomisation, concealment of treatment allocation and blinding are difficult issues to resolve in orthopaedic surgery, future trials should focus on detailed and correct reporting of outcome measures.

Adoption of an innovation to repair aortic aneurysms at a Canadian hospital: a qualitative case study and evaluation

Background

Priority setting in health care is a challenge because demand for services exceeds available resources. The increasing demand for less invasive surgical procedures by patients, health care institutions and industry, places added pressure on surgeons to acquire the appropriate skills to adopt innovative procedures. Such innovations are often initiated and introduced by surgeons in the hospital setting. Decision-making processes for the adoption of surgical innovations in hospitals have not been well studied and a standard process for their introduction does not exist. The purpose of this study is to describe and evaluate the decision-making process for the adoption of a new technology for repair of abdominal aortic aneurysms (endovascular aneurysm repair [EVAR]) in an academic health sciences centre to better understand how decisions are made for the introduction of surgical innovations at the hospital level.

Methods

A qualitative case study of the decision to adopt EVAR was conducted using a modified thematic analysis of documents and semi-structured interviews. Accountability for Reasonableness was used as a conceptual framework for fairness in priority setting processes in health care organizations.

Results

There were two key decisions regarding EVAR: the decision to adopt the new technology in the hospital and the decision to stop hospital funding. The decision to adopt EVAR was based on perceived improved patient outcomes, safety, and the surgeons' desire to innovate. This decision involved very few stakeholders. The decision to stop funding of EVAR involved all key players and was based on criteria apparent to all those involved, including cost, evidence and hospital priorities. Limited internal communications were made prior to adopting the technology. There was no formal means to appeal the decisions made.

Conclusion

The analysis yielded recommendations for improving future decisions about the adoption of surgical innovations. ese empirical findings will be used with other case studies to help develop guidelines to help decision-makers adopt surgical innovations in Canadian hospitals.

Evidence-Based Practice in Oral and Maxillofacial Surgery: Audit of 1 Training Center

PURPOSE

To evaluate the proportion of evidence-based interventions in the field of oral and maxillofacial surgery in a regional training center.

PATIENTS AND METHODS

A prospective clinical audit was carried out within the discipline of Oral and Maxillofacial Surgery, University of Hong Kong in February 2004 for a period of 6 months to investigate the extent of evidence-based practice. Consecutive diagnosis and intervention pairs were identified and recorded through standardized charts in randomly selected clinical sessions. A corresponding literature search using Medline and the Cochrane Library was performed to identify best current evidence. Each pair was then analyzed and graded according to the best current evidence.

RESULTS

Of 500 cases, 273 were eligible for evaluation while the rest were excluded based on 4 defined exclusion criteria. A majority of interventions (n = 195, 71.4%) were found to be evidence-based. Seventy-eight (28.6%) interventions were found to be not evidence-based. Among the evidence, a majority (56.1%) was level 5 evidence, which are case series or systematic review/meta-analysis of case series, and 36% were level 3 or above, which are randomized control trial (RCT) (level 3), meta-analysis of RCTs (level 2), or systematic review of RCTs (level 1). There was no statistically significant difference in the proportion of evidence-based practice between specialists and trainees in oral and maxillofacial surgery who saw and treated patients.

CONCLUSION

This study demonstrated that most interventions prescribed in this oral and maxillofacial surgery training center were evidence-based, and the proportion was comparable with that reported by other specialties.

Levels of evidence and journal impact factor in oral and maxillofacial surgery

The aim of this study was to identify the relationship between quality of research, in terms of levels of evidence, and journal impact factor (IF), and to describe the pattern of publications in oral and maxillofacial surgery. All four major journals in this subject area only, and with a published IF, were included in the study. Latest published IF dated 2004 was chosen, and all articles related to its calculation were accessed and classified into four levels of evidence. Correlation between levels of evidence and IF was investigated and the pattern of publications was described. All eligible 932 published articles were analysed. None (0%) were level I evidence, 20 (2%) were Level II, 70 (8%) level III and 337 (40%) level IV; 465 (50%) articles were classified as non-evidence. IF ranged from 0.689 to 1.154. There were statistically significant correlations between levels of evidence and IF (rho=1.0, P<0.01). Among the 465 non-evidence articles, there were 219 (47%) case reports, 91 (20%) animal studies, 52 (11%) laboratory studies, 35 (8%) technical notes, 24 (5%) tutorial articles, and 16 (3%) reviews articles.

Randomized Controlled Trials in Pediatric Urology: Room for Improvement

PURPOSE

The primary purpose of this study was to ascertain what proportion of the body of published literature in pediatric urology is represented by randomized controlled trials. The secondary purpose was to assess the quality of these trials.

MATERIALS AND METHODS

Using a predefined strategy, we conducted systematic computerized searches of the MEDLINE (years 1966 to 2004) and EMBASE (1980 to 2004) databases to identify all English language randomized controlled trials related to pediatric urology. Full text versions of identified studies were reviewed in blinded fashion for key demographic, methodological and statistical characteristics. Trial quality was assessed with the previously validated Jadad tool.

RESULTS

The 77 identified randomized controlled trials represented only 0.4% to 0.9% of the indexed pediatric urology literature. The origins of these trials were Europe (40%), North America (26%) and a variety of other geographic centers (34%). A primarily surgical focus was present in 43% of the studies. Trials with negative results represented only 19% of the total randomized controlled trials. Generally, the trials were of low to fair quality (median Jadad score 3), with substandard methodological reporting and planning. There was not a significant trend toward improved quality in recent years. Trials from North America and Europe had higher quality (p = 0.007), as did those reporting negative results (p = 0.0001).

CONCLUSIONS

Randomized controlled trials in pediatric urology constitute only a small proportion of the body of published literature in the field. High quality studies are uncommon. Efforts should be made to increase the number of well designed, randomized controlled trials in pediatric urology.

Ethical issues in evidence-based surgery

Evidence-based medicine, although ostensibly concerned with the research evidence underlying claims of efficacy for surgical procedures,has a direct connection with the ethics of surgical decision making. Questions of whether new procedures should ever be performed on patients outside of a formal research protocol, what the patient should be told about the uncertainties inherent in the use of nonvalidated innovative procedures, when formal evaluation is necessary, what form that evaluation should take, and how the burdens and results of such research can be distributed fairly all involve balancing competing ethical principles. Good ethics requires good facts, and evidence from well-controlled experiments provides best information upon which to base decisions in these areas and to build ethical surgical practice.

Clinical trials in head and neck oncology: An evaluation of clinicians' willingness to participate

BACKGROUND

This study investigated the individual and collective ("community") equipoise of surgeons and oncologists and their willingness to take part in each of six hypothetical randomized controlled trials in head and neck oncology.

METHODS

A survey was mailed to Australasian head and neck specialists.

RESULTS

Among 109 respondents (74% response), the scenario with the highest level of individual equipoise pertained to the use of adjuvant interferon for patients with high-risk malignant melanoma, with 45% indicating complete uncertainty between treatment approaches. Significant differences in levels of community equipoise were demonstrated between surgeons and oncologists for three of the scenarios. Willingness to participate in randomized controlled trials ranged from 39% to 72%. Increasing strength of treatment preference was associated with unwillingness to participate in randomized controlled trials for two of six scenarios.

CONCLUSION

High levels of equipoise and willingness to participate in clinical research augur well for future randomized controlled trials in head and neck oncology.

Developing measures of surgeons' equipoise to assess the feasibility of randomized controlled trials in vascular surgery

BACKGROUND

Equipoise is defined medically as a state of genuine uncertainty about the relative benefits of alternative treatment options. This study investigated individual and collective equipoise among vascular surgeons for controversial clinical questions to assess the feasibility of conducting randomized controlled trials.

METHODS

Vascular surgeons throughout Australia and New Zealand received a survey by mail.

RESULTS

Vascular surgeons (n=146, 77% response fraction) were able to quantify the strength of their treatment preferences and did so differentially between clinical scenarios using a simple scale. Almost one quarter (24%; 95% CI, 18%-32%) were completely undecided about whether carotid endarterectomy or carotid stenting was preferable to treat carotid stenosis in high-risk patients, indicating individual equipoise. In contrast, the vast majority of respondents (89%; 95% CI, 82%-93%) favored carotid endarterectomy over carotid stenting for average-risk patients, suggesting lack of community equipoise for this patient group. Similarly, there was lack of community equipoise for treatments for abdominal aortic aneurysm in high-risk patients with 88% (95% CI, 81%-92%) favoring a minimally invasive approach. Older respondents were consistently less willing to take part in randomized trials, with strength of treatment preference also independently predicting willingness to participate in 4 of 6 trials.

CONCLUSIONS

Individual and community equipoise can be measured in a representative sample of surgeons as part of the feasibility assessment for future randomized controlled trials.

Randomized Trials of Neurosurgical Interventions: A Systematic Appraisal

OBJECTIVE

To systematically appraise the study design and quality of reporting of randomized controlled trials (RCTs) on neurosurgical procedures and to identify potential defects and biases.

METHODS

Randomized controlled trials with at least five patients comparing any neurosurgical procedure against another procedure, nonsurgical treatment, or no treatment were retrieved from MEDLINE, EMBASE, and the Cochrane Library. We analyzed study design, quality of reporting, and trial results.

RESULTS

The median sample size in the 108 eligible reports was 68 patients. Ninety-nine trials (91.7%) reported inclusion and exclusion criteria, 55 (50.9%) mentioned the randomization mode, and 87 (80.6%) adequately described withdrawals, but only 31 (28.7%) described allocation concealment, only 23 (21.3%) gave power calculations, and only 20 (18.5%) were adequately powered. Significant efficacy or trend for efficacy was claimed in 46 reports (42.6%), and no difference between the compared procedures was found in 60 trials (55.6%). Trials with a larger sample size were more likely to report withdrawals (P = 0.02) and power calculations (P = 0.006). Only 14 trials (13.6%) were double-blind, and this was less frequent in longer trials (P = 0.02). Among quality criteria, only the reporting of randomization mode improved significantly over time (P = 0.015).

CONCLUSION

Several aspects of the design and reporting of randomized controlled trials on neurosurgical procedures can be improved. Larger, adequately powered, and accurately reported trials are needed.

Surgical innovation or surgical evolution: an ethical and practical guide to handling novel neurosurgical procedures

Object. Surgical innovation is an important driver of improvements in technique and technology, which ultimately translates into improvements in patients' outcomes. Nevertheless, patients may face new risks of morbidity and mortality when surgical innovation is used, and well-intentioned surgical “experimentation” on patients must be regulated and monitored. In this paper the authors examine the challenges of defining surgical innovation and briefly review the literature on this challenging subject.

Methods. Using examples from the field of neurosurgery and in part from the personal experience of the senior author, the authors develop a model of levels of experimental acuity of surgical procedures and offer recommendations on how these procedures would best be regulated.

Conclusions. The authors propose guidelines for determining the need for regulation of innovation. The potential role of institutional review boards in this process is highlighted.

Clinical trials in orthodontics I: demographic details of clinical trials published in three orthodontic journals between 1989 and 1998

AIM

To test the hypothesis that there is insufficient evidence available, from clinical trials, to allow evidence-based decisions to be made on the effectiveness of orthodontic treatment.

OBJECTIVES

To identify reports of orthodontic clinical trials and assess their demographic characteristics.

DESIGN

A retrospective, observational study.

SETTING

The American Journal of Orthodontics and Dentofacial Orthopedics, British Journal of Orthodontics, and European Journal Orthodontics.

DATA SOURCE

Clinical trials published between 1989 and 1998.

METHOD

A hand-search was performed to identify all clinical trials. The journal and year of publication, research method, interventions, and sample size of the trials reported were recorded.

RESULTS

One-hundred-and-fifty-five trial reports were identified of which 56 (36.1%) were published from 1989 to 1993 and 99 (69%) from 1994 to 1998. Ninety-nine (69%) reports were published in the AJO-DO, 18 (11.6%) in the BJO and 38 (24.5%) in the EJO. Eighty-five (54.8%) were reports of randomized controlled trials and 70 (45.2%) of controlled clinical trials. The interventions most frequently assessed were bonding materials (21.9%), growth modification treatments (21.3%), and oral hygiene procedures (9.0%). The median sample size was 32 (IQR 19.5, 50).

CONCLUSION

There is sufficient evidence available from clinical trials to warrant doing systematic reviews of orthodontic clinical trials to aid decision-making.

The quality of reporting of randomized trials in the Journal of Bone and Joint Surgery from 1988 through 2000

BACKGROUND

The purpose of this study was threefold: (1) to determine the scientific quality of published randomized trials in the American Volume of The Journal of Bone and Joint Surgery from 1988 through 2000, (2) to identify predictors of study quality, and (3) to evaluate inter-rater agreement in the scoring of study quality with use of a simple scale.

METHODS

Hand searches of The Journal of Bone and Joint Surgery were conducted in duplicate to identify randomized clinical trials. Of 2468 studies identified, seventy-two (2.9%) met all eligibility criteria. Two investigators each assessed the quality of the study under blinded conditions and abstracted relevant data.

RESULTS

The mean score (and standard error) for the quality of the seventy-two randomized trials was 68.1% plus minus 1.6%; 60% (forty-three) scored <75%. Drug trials had a significantly higher mean quality score than did surgical trials (72.8% compared with 63.9%, p < 0.05). Regression analysis revealed that cited affiliation with an epidemiology department and cited funding were associated with higher quality scores. Failure to conceal randomization, to blind outcome assessors, and to describe why patients were excluded resulted in significantly lower quality scores (p < 0.05), more than the 5% decrease expected by removal of each item. A priori calculations of sample size were rarely performed in the reviewed studies, and only 2% of the studies with negative results included a post hoc power analysis. The Detsky quality scale met accepted standards of interobserver reliability (kappa, 0.87; 95% confidence interval, 0.70 to 0.95).

CONCLUSIONS

Few studies published in The Journal of Bone and Joint Surgery were randomized trials. More than half of the trials were limited by a lack of concealed randomization, lack of blinding of outcome assessors, or failure to report reasons for excluding patients. Application of standardized guidelines for the reporting of clinical trials in orthopaedics should improve quality.

Innovation and research in pediatric surgery

Progress in pediatric surgery has relied primarily on the diffusion of innovations as reported in case series in the literature. The standards applied to clinical research that predominate in medical specialties have not become common in surgery, despite agreement that comparative trials produce the best evidence. Many pediatric surgical interventions compete with similar or even radically different surgical and medical approaches to the same condition. The resulting confusion about how to proceed raises serious ethical questions for physicians and families facing major decisions about surgery, medical therapy, or comfort care. Pediatric surgeons have a moral obligation to undertake formal research comparing their preferred operations to alternative approaches.

The role of prospective randomized clinical trials in pediatric surgery: state of the art?

PURPOSE

This study sought to determine the role of randomized controlled trials (RCT) in the evolution of pediatric surgical practice.

METHODS

The authors used a computer-assisted literature search to identify all clinical trials related to pediatric surgery published in the English-language literature from 1966 through 1999. Each article was reviewed in detail for purpose, content, conduct, and quality of the trial. The authors assessed quality with a previously validated instrument (Chalmers Qualitative Assessment).

RESULTS

The authors identified 134 RCTs related to pediatric surgery over the past 33 years. This accounts for 0.17% of 80,377 articles published in the field. The areas of surgery studied were analgesia 65 (49%), antibiotics 17 (13%), extracorporeal membrane oxygenation (ECMO) 9 (7%), gastrointestinal, burns, oncology, minimally invasive surgery, vascular access, congenital anomalies, and trauma (each <5%). Only 16 (12%) trials compared 2 surgical therapies, 9 (7%) compared a medical versus a surgical therapy, and 109 (81%) compared 2 medical therapies in surgical patients. Fourteen (10%) RCTs were funded by peer-reviewed agencies. Only 17 (13%) RCTs included a biostatistician as an author or a consultant. Trial design included calculation of sample size and statistical power in 21 (16%) RCTs. Method of randomization was reported in only 51 (38%). The test statistic and observed probability value was reported in 15 (11%).

CONCLUSIONS

Clinical trials are used infrequently to answer questions related to pediatric surgery. When RCTs are utilized, they often suffer from poor trial design, inadequate statistical analysis, and incomplete reporting. Pediatric surgery could benefit from increased expertise, funding, and participation in clinical trials.

In search of evidence-based plastic surgery: the problems faced by the specialty

Recently, there has been significant interest both from government and medical practitioners in the discipline of evidence-based medicine. In this article we discuss the problems faced by the plastic surgeon when trying to ensure that practice is evidence-based and highlight some of the reasons behind these difficulties. With the rapid growth of the Internet we also outline its use to access high quality information for the plastic surgeon.

Surgery and the randomised controlled trial: past, present and future

Randomised controlled trials (RCTs), with their prospective definition of methods and outcome measures, double-blind assessment of outcomes and unbiased selection of subjects and controls, provide the best possible evidence for deciding the value of a medical or surgical intervention. Few surgical studies are designed as RCTs, and those that are should be of a higher quality. The lack of good surgical RCTs may be a result of surgeons lacking the necessary training, expertise and desire to perform RCTs, inadequate funding from granting agencies, difficulties in securing patient consent or a lack of sufficient patient numbers. If an RCT is not feasible for a particular study, then alternative research designs, such as prospective matched-pair trials, may need to be better developed and used. If RCTs can be performed, other strategies to increase the number and quality of RCTs may be needed: Education of surgeons in clinical research methods Improved funding of surgical RCTs Compulsory evaluation of new techniques and technology before their general adoption is permitted.

Evidence-based medicine: study design for evaluation of treatment

Obstetrician-gynecologists frequently rely on results of published studies to guide decisions regarding clinical management of their patients; that is, to practice evidence-based medicine. Therefore, it is essential that these studies be carried out meticulously. One of the first tasks in performing research is to select an appropriate study design. The principal designs are survey, which is a descriptive method; observational, which generates hypotheses; and experimental, which tests hypotheses, and is commonly called a randomized, clinical trial. Each one has advantages and limitations that must be considered carefully to achieve the most applicable effective results.

An analysis of papers published in the British and European Journals of Orthodontics

The aims of this study were to assess the type, subject, setting and methods of papers published in British Journal of Orthodontics (BJO) and European Journal of Orthodontics (EJO) between 1989 and 1993 to allow all published randomized controlled trials (RCTs) to be identified and a comparison of the papers published in the journals to be made. A hand search of all papers published in BJO and EJO between 1989 and 1993 was performed, and the type, subject, setting, and methods of each paper were classified and recorded. Of the studies, 59.3 per cent related to clinical orthodontics, but only three RCTs were identified in each journal. This comprised 2.8 per cent of the clinical research papers which were analysed. The remaining studies used non-randomized controls or were uncontrolled. Significant differences were found between the type (P < 0.001), subject (P < 0.001), setting (P < 0.01) and methods (P < 0.05) of papers published in the two journals. Relatively more papers in BJO were case reports, clinical opinions and update articles, reported on orthodontic materials or assessed methods of measuring the outcome of treatment. Ninety per cent of papers in EJO reported the results of research projects and relatively more papers, than in BJO, were related to animal studies, and were laboratory based or epidemiological.

OBJECTIVES

To identify all randomized controlled trials (RCTs) and compare papers published in two orthodontic journals.

DESIGN

A retrospective, observational study.

SETTING

The British Journal of Orthodontics (BJO) and European Journals of Orthodontics) (EJO).

DATA SOURCE

Papers published between 1989 and 1993.

METHOD

A hand search of all papers was performed. The type, subject, setting and methods of each paper were classified and recorded.

RESULTS

200 papers were identified in BJO and 275 in EJO. Six RCTs were identified which represents 2.8 per cent of clinical research papers. Significant differences were found between the type (P < 0.001), subject (P < 0.001), setting (P < 0.01), and methods (P < 0.05) of papers published in the two journals. More papers in BJO were case reports, clinical opinions, and update articles, and reported on orthodontic materials or assessed methods of measuring the outcome of treatment. Ninety per cent of papers in EJU reported the results of research projects. More papers were related to animal studies; were laboratory based on epidemiological.

CONCLUSION

Despite the RCT being regarded as the 'Gold Standard' for the evaluation of therapeutic interventions and materials only six (5.1 per cent) of such studies used this method. Significant differences in the type, setting and subject of papers published in BJO and EJO between 1989 and 1993 were found.

Should we be performing more randomized controlled trials evaluating surgical operations?

BACKGROUND

The objective of this study was, first to determine what proportion of clinical treatment evaluation questions involving surgical operations could be answered by a randomized controlled trial (RCT). Second, for those questions not amenable to a RCT, to determine the problems that potentially preclude the initiation of RCT in an ideal clinical research setting.

METHODS

A sample of treatment evaluation questions involving a surgical procedure was obtained by a computerized search of the surgical literature. Problems precluding a RCT were defined. Their face validity and interobserver and intraobserver reliability were assessed. By use of these criteria, the sample questions were evaluated to determine whether a RCT could be performed and, if not, the predominant reasons precluding RCT of surgical procedures.

RESULTS

Only 38.8% of treatment evaluation questions could have been answered by a RCT in an ideal clinical research setting. Patient preference was the most common precluding problem encountered (40% of all problems). The principal precluding problem was patient preference in 23.1%, an uncommon condition in 24.2%, and lack of community (clinical) equipoise in 10%. Methodologic issues (1.2%) and surgical preference (2.3%) were infrequent precluding problems. Questions evaluating therapy for malignant disease, comparing surgical with nonsurgical therapies, and where survival was the primary outcome were more likely to have problems precluding RCT.

CONCLUSIONS

In the ideal situation RCT can be performed to evaluate only 40% of treatment questions involving surgical procedures. Patient preferences, uncommon conditions, and lack of surgical community equipoise appear to be the most common reasons precluding the of RCT of surgical operations.

Clinical studies in surgical journals--have we improved?

A critical appraisal of all clinical studies published in 1980 and 1990 in three surgical journals--Diseases of the Colon and Rectum (DCR), Surgery (SURG), and the British Journal of Surgery (BJS)--was made to ascertain the frequency with which various research designs appeared, the standard of individual clinical studies, and a comparison of changes in the past decade. Clinical studies were classified into case studies or comparative studies. Comparative studies included randomized controlled trials (RCT), nonrandomized controlled trials, retrospective cohorts, and case-control studies. A 10-point index score (range, 0-10) was used to assess the comparative studies. A sample of articles was analyzed for interobserver and intraobserver variation, with strong agreement between reviewers for classification of trials (unweighted kappa, 0.87) and index scores (0.67). Of 1,481 articles reviewed, 1,060 were classified as clinical studies. Sixteen percent of all clinical studies were comparative studies in 1980, compared with 17 percent in 1990. Of these, 7 percent were RCT in both years. In 1980, 6 percent of clinical studies in DCR were comparative studies, 19 percent in BJS, and 18 percent in SURG. In 1990, 11 percent, 18 percent, and 18 percent, respectively, were comparative studies. In 1980, the proportion of RCT in DCR was 0 percent, in BJS 12 percent, and in SURG 4 percent, compared with 3 percent, 8 percent, and 8 percent, respectively, in 1990. Overall, 52 of 76 (68 percent) RCT were published in BJS. The standard of comparative studies increased overall from 5.49 to 6.04 (P = NS), and that of RCT increased from 7.06 to 7.70 (P = NS). The standard of comparative studies in DCR in 1980 was lower than those in BJS (P < 0.001) and SURG (P < 0.001). The standard of comparative studies in DCR improved from 1.67 in 1980 to 5.47 in 1990 (P < 0.001). There was no significant difference in the standard of comparative studies among the three journals in 1990. In conclusion, there has been no overall increase in the proportion of stronger clinical trial designs in the journals reviewed. A small increase seen in the overall standard of comparative studies was not statistically significant.

Dependency on bypass circulation: a case study

In a case of consecutive occlusion of both internal carotid arteries bilateral STA-MCA anastomoses were established. Since a series of angiographic examinations were undertaken, dynamics and probable efficacy of both natural and artificial collateral circulations could be followed up in a two-year period. Objectively in the evaluation of anastomoses was enhanced by direct percutaneous STA-angiography (STAG), in order to delineate the vascular territory irrigated exclusively by the anastomosis. By demonstrating exactly the functional capacity of anastomoses in correlation with the clinical course, dependency on bypass circulation could be assessed.

Hypothesis testing in neurosurgical trials

Controlled clinical trials represent the most scientific methods of evaluating a new form of treatment. In designing such a trial, one must avoid committing two kinds of errors. The Type I error is defined as falsely concluding that a difference between two treatments exists, when they are equal. The Type II error is committed when one concludes that two treatments are the same, when a real difference exists. To reduce the probability of committing these errors, large sample sizes are required. A survey of neurosurgical trials showed that the majority of these trials have an unacceptably high probability of committing a Type II error because of inadequate sample size.

Treatment of spinal epidural metastases. Randomized prospective comparison of laminectomy and radiotherapy

Metastases to the spinal epidural space with compression of the spinal cord or cauda equina are commonly encountered by physicians in a variety of clinical field. In the recent past, decompressive laminectomy followed by radiotheray was thought to be the best available treatment. More recently, radiotherapy alone has been advocated as an alternative treatment mode with a similar rate of effectiveness. This study compares laminectomy followed by radiotherapy to radiotherapy alone in the treatment of spinal epidural metastases in a randomized, prospective clinical trial. No significant difference was found in the effectiveness of the two treatment methods in regard to pain relief, improved ambulation, or improved sphincter function. Patients with an incomplete myelographic block fared well regardless of treatment, and those with a complete block fared poorly. Because of the limited size of this study and because of certain unforeseen design defects, the results are suggestive but not conclusive. Suggestions are made for a future randomized, prospective multicenter study that would conclusively answer the perplexing question as to the most efficacious method for treating spinal epidural metastases.

Speech changes in cerebral-palsied patients after cerebellar stimulation

A quantitative study was made of the effects of focal cerebellar stimulation on oral-motor control, duration of phonation, articulation and vocal characteristics in 10 patients with cerebral palsy. The patients were evaluated prior to surgery and again after approximately two and six months of cerebellar stimulation. One patient had normal speech, which was not affected by the stimulation; another case with moderate dysarthria due to severe hearing loss was not helped by the stimulation. Seven patients increased their duration of vowel phonation by about two seconds, a significant amount. Four of the patients with moderate dysarthria improved their articulation, particularly for the consonants S, Sh and Th, after two months of stimulation. Two patients had changes in oral-motor control, which included better tongue and lip movements, and two other cases had small alterations in hypernasality or breathiness. Most of the changes in sound production and speech intelligibility appear to be related to improved intra-oral breath control.