Randomized clinical trials in neurosurgery

Randomized controlled trials in neurosurgery

Randomized controlled trials (RCTs) have become the standard method of evaluating new interventions (whether medical or surgical), and the best evidence used to inform the development of new practice guidelines. When we review the history of medical versus surgical trials, surgical RCTs usually face more challenges and difficulties when conducted. These challenges can be in blinding, recruiting, funding, and even in certain ethical issues. Moreover, to add to the complexity, the field of neurosurgery has its own unique challenges when it comes to conducting an RCT. This paper aims to provide a comprehensive review of the history of neurosurgical RCTs, focusing on some of the most critical challenges and obstacles that face investigators. The main domains this review will address are: (1) Trial design: equipoise, blinding, sham surgery, expertise-based trials, reporting of outcomes, and pilot trials, (2) trial implementation: funding, recruitment, and retention, and (3) trial analysis: intention-to-treat versus as-treated and learning curve effect.

Achieving high research reporting quality through the use of computational ontologies

Systematic reviews and meta-analyses constitute one of the central pillars of evidence-based medicine. However, clinical trials are poorly reported which delays meta-analyses and consequently the translation of clinical research findings to clinical practice. We propose a Center of Excellence in Research Reporting in Neurosurgery (CERR-N) and the creation of a clinically significant computational ontology to encode Randomized Controlled Trials (RCT) studies in neurosurgery. A 128 element strong computational ontology was derived from the Trial Bank ontology by omitting classes which were not required to perform meta-analysis. Three researchers from our team tagged five randomly selected RCT's each, published in the last 5 years (2004-2008), in the Journal of Neurosurgery (JoN), Neurosurgery Journal (NJ) and Journal of Neurotrauma (JoNT). We evaluated inter and intra observer reliability for the ontology using percent agreement and kappa coefficient. The inter-observer agreement was 76.4%, 75.97% and 74.9% and intra-observer agreement was 89.8%, 80.8% and 86.56% for JoN, NJ and JoNT respectively. The inter-observer kappa coefficient was 0.60, 0.54 and 0.53 and the intra-observer kappa coefficient was 0.79, 0.82 and 0.79 for JoN, NJ and JoNT journals respectively. The high degree of inter and intra-observer agreement confirms tagging consistency in sections of a given scientific manuscript. Standardizing reporting for neurosurgery articles can be reliably achieved through the integration of a computational ontology within the context of a CERR-N. This approach holds potential for the overall improvement in the quality of reporting of RCTs in neurosurgery, ultimately streamlining the translation of clinical research findings to improvement in patient care.

Randomized controlled trials in neurosurgery--how good are we?

BACKGROUND

The strongest evidence in medical clinical literature is represented by randomized controlled trials (RCTs). This study was designed to evaluate neurosurgically relevant RCTs published recently by neurosurgeons.

METHOD

A literature search in MEDLINE and EMBASE included all clinical studies published up to 30 June 2006. RCTs with neurosurgical relevance published by at least one author with affiliation to a neurosurgical department were selected. The number and characteristics of individual trials were recorded, and the quality of the trials with regard to study design, quality of reporting, and relevance for clinical practice was assessed by two different investigators using a modification of the Scottish Intercollegiate Guidelines Network methodology checklist. Changes of RCT quality over time as well as factors influencing the quality were analyzed.

FINDINGS

From the initial search results (MEDLINE n = 3,860, EMBASE n = 3,113 articles), 159 RCTs published by neurosurgeons were extracted for final evaluation. Of the RCTs, 62% have been published since 1995; 52% came from the USA, UK, and Germany. The median RCT sample size was 78 patients and the median follow-up 35.7 weeks. Fifty-two percent of all RCTs were of good, 37% of moderate, and 11% of bad quality, with an improvement over time. RCTs with financial funding and RCTs with a sample size of >78 patients were of significantly better quality. There were no major differences in the rating of the studies between the two investigators.

CONCLUSIONS

Only a fraction of neurosurgically relevant literature consists of RCTs, but the quality is satisfying and has significantly improved over the last years. An adequate sample size and sufficient financial support seem to be of substantial importance with regard to the quality of the study. Our data also show that by using a standardized checklist, the quality of trials can be reliably assessed by observers of different experience and educational levels.

The Quality of Randomized Controlled Trials in Major Anesthesiology Journals

Increased attention has been directed at the quality of randomized controlled trials (RCTs) and how they are being reported. We examined leading anesthesiology journals to identify if there were specific areas for improvement in the design and analysis of published clinical studies. All RCTs that appeared between January 2000 and December 2000 in leading anesthesiology journals (Anesthesiology,Anesthesia & Analgesia,Anaesthesia, and Canadian Journal of Anaesthesia) were retrieved by a MEDLINE search. We used a previously validated assessment tool, including 14 items associated with study quality, to determine a quality score for each article. The overall mean weighted quality score was 44% +/- 16%. Overall average scores were relatively high for appropriate controls (77% +/- 7%) and discussions of side effects (67% +/- 6%). Scores were very low for randomization blinding (5% +/- 2%), blinding observers to results (1% +/- 1%), and post-beta estimates (16% +/- 13%). Important pretreatment clinical predictors were absent in 32% of all studies. Significant improvement in the reporting and conduct of RCTs is required and should focus on randomization methodology, the blinding of investigators, and sample size estimates. Repeat assessments of the literature may improve the adoption of guidelines for the improvement of the quality of randomized controlled trials.

Randomized Trials of Neurosurgical Interventions: A Systematic Appraisal

OBJECTIVE

To systematically appraise the study design and quality of reporting of randomized controlled trials (RCTs) on neurosurgical procedures and to identify potential defects and biases.

METHODS

Randomized controlled trials with at least five patients comparing any neurosurgical procedure against another procedure, nonsurgical treatment, or no treatment were retrieved from MEDLINE, EMBASE, and the Cochrane Library. We analyzed study design, quality of reporting, and trial results.

RESULTS

The median sample size in the 108 eligible reports was 68 patients. Ninety-nine trials (91.7%) reported inclusion and exclusion criteria, 55 (50.9%) mentioned the randomization mode, and 87 (80.6%) adequately described withdrawals, but only 31 (28.7%) described allocation concealment, only 23 (21.3%) gave power calculations, and only 20 (18.5%) were adequately powered. Significant efficacy or trend for efficacy was claimed in 46 reports (42.6%), and no difference between the compared procedures was found in 60 trials (55.6%). Trials with a larger sample size were more likely to report withdrawals (P = 0.02) and power calculations (P = 0.006). Only 14 trials (13.6%) were double-blind, and this was less frequent in longer trials (P = 0.02). Among quality criteria, only the reporting of randomization mode improved significantly over time (P = 0.015).

CONCLUSION

Several aspects of the design and reporting of randomized controlled trials on neurosurgical procedures can be improved. Larger, adequately powered, and accurately reported trials are needed.

What we research: survey of American Association of Neurological Surgeons and Congress of Neurological Surgeons member publications

OBJECT

The goal of this study was to create a searchable database of research manuscripts authored by members of the American Association of Neurological Surgeons and the Congress of Neurological Surgeons (AANS/CNS) to describe the nature and character of the research currently being undertaken by neurosurgeons.

METHODS

Manuscripts published by all physician members listed in the 2001 AANS/CNS Membership Directory (6921 physicians) were gathered into a database through individual literature searches of the author name for the calendar year 2001. Duplicate publications were purged and the database was reviewed for accuracy. An internal verification of the database revealed a 4% underreporting rate. Statistics from the database were compiled and displayed with information about AANS/CNS members and their clinical activities. The AANS/CNS members published a total of 2748 research the manuscripts in 479 different journals during 2001. Thirty-eight percent of the manuscripts (1042 of 2748) were authored by US members and 62% (1706 of 2748) by non-US members. The focus of the majority of manuscripts included the areas of brain tumor (26%; 707 of 2748), vascular disease (20%; 558 of 2748), spine (10%; 282 of 2748), and trauma (8%; 233 of 2748). Sixty-nine percent of manuscripts (1897 of 2748) were retrospective and technical clinical studies, and of these 39% (744 of 1897) were case reports. Laboratory investigations made up 15% (414 of 2748) of all manuscripts, whereas prospective randomized clinical trials represented 1% (34 of 2748).

CONCLUSIONS

The majority of AANS/CNS member manuscripts are authored by non-US members despite their small AANS/CNS representation. Most research is clinical, based on retrospective data, and includes a large number of case reports. A disparity exists between what neurosurgeons do clinically and both the quantity and subject of their research.

Science in neurosurgery: the importance of the scientific method

The scientific method offers a way for a neurosurgeon to assess objectively his or her professional activities, especially in regard to the results of treatment. For this reason, all neurosurgical trainees should be instructed in at least the basic tenets of the scientific method, and all neurosurgeons should be guided by these tenets in their daily practice.

A comparative study of the Reaction Level Scale (RLS85) with Glasgow Coma Scale (GCS) and Edinburgh-2 Coma Scale (modified) (E2CS(M))

In this work a new coma scale for the assessment of responsiveness in acute brain disorders, constructed near the year 1985 by Scandinavian investigators, the Reaction Level Scale (RLS85), is compared with two other coma scales namely: (i) the Glasgow Coma Scale: (GCS); (ii) the Edinburgh-2 Coma Scale, after modification: (E2CS(M)). The study proceeded in the form of a statistical analysis of assessments made on 46 patients according to RLS85 and GCS (i.e., when comparison was with GCS) and on 28 patients according to RLS85 and E2CS(M). In all 74 cases two physicians participating as "observers" carried out the assessments. They were both contacting the patient--not together but--successively within a time interval of less than 20'. Hence the data appeared as "pairwise" observations for any of the three scales above. The results of the analysis, arising from a rather strict statistical reasoning, can be summarized as follows: (1) The rank correlation coefficient r(s) between: (i) RLS85 and GCS sum score, (ii) RLS85 and E2CS(M), was found to be at a satisfactory level meaning that all three scales indicate almost the same "ranking order of severity". (2) Reliability was compared by taking into account as to what extent the two observers agreed on RLS85 and--simultaneously--disagreed on the other scale. The "sign" test was applied and as a result RLS85 proved to be more reliable than; (i) GCS (EMY profile), (ii) GCS sum score, and (iii) E2CS(M), in all the above mentioned at a high level of significance. (3) Apart from the test above, some values of the index kappa (kappa) of interobserver agreement were calculated. Those corresponding to RLS85 are considerably higher. In particular the overall value based on 74 pairwise assessments amounted to kappa = 0.733 associated with a standard error sigma(kappa) = 0.061. This was a satisfactory result regarding the features of RLS85. (4) As far as coverage is concerned, again--by the "sign" test--the predominance of RLS85 versus GCS (EMY profile) was accepted.

The role of surgery in the management of supratentorial intermediate and high-grade astrocytomas in adults

In this analysis, the authors review studies over the last 50 years addressing the association between long-term survival and type of surgical management in adults with supratentorial intermediate or high-grade astrocytomas. Earlier reports are included because they are repeatedly referenced in current works and clearly are an important basis upon which present attitudes are predicated. Because recent work has definitively demonstrated the significance of prognostic variables on outcome, the handling of such factors in studies that investigated survival data according to degree of surgery is emphasized. Study design, experimental methods used, and methods of data analysis are also examined. This analysis shows that there is little justification for dogmatic statements concerning the relationship between increasing patient survival times and aggressive surgical management in adults with supratentorial intermediate or high-grade astrocytomas, if patients receive postoperative radiotherapy.

Current reporting of responsiveness in acute cerebral disorders. A survey of the neurosurgical literature

One hundred sixty-six papers published in seven neurosurgical journals from 1983 through 1985 have been surveyed to determine the methods used for assessment of overall patient responsiveness in acute cerebral disorders (coma grading). Fifty-one different coma scales or modifications were found. The Glasgow Coma Scale (GCS) sum score (that is, the sum of the scores of the individual eye, verbal, and motor scales) dominated (54%), and was used in 73 (76%) of 96 of the head-injury studies; in 56 (77%) of these 73 studies it was the single method of grading neurological status. The GCS sum score was used in 16 (23%) of 70 studies in patients with other etiologies. The Hunt and Hess scale was used in 26 (57%) of 46 reports of patients with subarachnoid hemorrhage. In 31 (55%) of the 56 studies of head injuries using the GCS alone, it was not obvious if the 12- or 13-grade scale was used. In 13 studies (23%) no reference to methodological investigations was made. In 44 papers (79%) the handling of untestable features, such as intubation or swollen eyes, was not reported. In the 56 studies using the GCS alone, coma was defined in many different ways and in 22 studies the definition of coma was not specified. In 63% of reports, the GCS sum score scale was combined in one to five groups of scores and this was done in 32 different ways. No information was available to describe the procedure of data aggregation or the reliability of the 13-grade GCS sum score. The lack of standardization makes it unnecessarily difficult to perform valid comparisons between different series of patients. Since the GCS sum score is the most widely used scale, it is suggested that the reporting of the GCS sum score should be standardized regarding pseudoscoring, coma definition, and use of combined scores. Further studies on the reliability of the GCS sum score are needed.

Clinical trials and cerebral resuscitation research

Resuscitation research trials must have adequate scientific basis, sound ethics, feasibility, and potential impact on health care. The organization and methodology of the Pittsburgh-based Brain Resuscitation Clinical Trials (BRCT I) are reviewed in detail. This type of international collaborative study will be useful for evaluation of new cardiopulmonary-cerebral resuscitation therapies.