The Management of Persistent Distal Occlusions after Mechanical Thrombectomy and Thrombolysis: An Inter- and Intrarater Agreement Study

BACKGROUND AND PURPOSE

The best management of patients with persistent distal occlusion after mechanical thrombectomy with or without IV thrombolysis remains unknown. We sought to evaluate the variability and agreement in decision-making for persistent distal occlusions.

MATERIALS AND METHODS

A portfolio of 60 cases was sent to clinicians with varying backgrounds and experience. Responders were asked whether they considered conservative management or rescue therapy (stent retriever, aspiration, or intra-arterial thrombolytics) a treatment option as well as their willingness to enroll patients in a randomized trial. Agreement was assessed using κ statistics.

RESULTS

The electronic survey was answered by 31 physicians (8 vascular neurologists and 23 interventional neuroradiologists). Decisions for rescue therapies were more frequent (n = 1116/1860, 60%) than for conservative management (n = 744/1860, 40%; P < .001). Interrater agreement regarding the final management decision was "slight" (κ = 0.12; 95% CI, 0.09-0.14) and did not improve when subgroups of clinicians were studied according to background, experience, and specialty or when cases were grouped according to the level of occlusion. On delayed re-questioning, 23 of 29 respondents (79.3%) disagreed with themselves on at least 20% of cases. Respondents were willing to offer trial participation in 1295 of 1860 (69.6%) cases.

CONCLUSIONS

Individuals did not agree regarding the best management of patients with persistent distal occlusion after mechanical thrombectomy and IV thrombolysis. There is sufficient uncertainty to justify a dedicated randomized trial.

Novices cannot fill the examiners' shoes: Evidence of footwear examiners' expertise in shoe comparisons

The value of a footwear examiner's opinion centres on their ability to determine whether a particular shoe made an impression with greater accuracy than a novice. However, there has been limited research on the expertise of footwear examiners and the accuracy and reproducibility of their decisions. In the current study, we measured the accuracy and consensus of 31 footwear examiners versus a comparison group of 29 novices. Participants completed 20 ground truth known mock shoe comparisons. Results demonstrated that footwear examiners were more accurate than novices, regardless of comparison difficulty. Overall, on trials where probative decisions were given, examiners made false identifications and false exclusions on a total of 3% and 2% of trials, while novices made false identifications and false exclusions on a total of 19% and 17% of trials. Examiners also demonstrated better consensus in their opinions than novices, although both groups demonstrated low levels of agreement in their responses and variability in their interpretation of the conclusion scale. In summary, these findings support the proposition that footwear examiners show expert-level performance in matching known and unknown footwear impressions. These performance estimates may help the criminal justice system to appropriately value footwear examination evidence.

Understanding crossovers and potential ways to mitigate the problem: Lessons from influential trials on lumbar microdiscectomy

BACKGROUND

Lumbar microdiscectomy is the most frequent surgical intervention used in the treatment of sciatica from herniated lumbar discs. Many discectomy trials have been plagued with an excessive number of crossovers that have rendered results inconclusive.

METHODS

We review the design and results of influential lumbar microdiscectomy trials. We also discuss the various strategies that have been used to decrease the number of crossovers or to mitigate the effects of crossovers on analyses.

RESULTS

Randomized trials on lumbar discectomy were affected by crossover rates of 8% to 42%. Various strategies that have been used to decrease that number or to mitigate the effects on results include: patient selection, blinding (placebo-controlled trials), an immediate access to surgery for the surgical group (but limited access to surgery for the conservative group), shortening the follow-up period necessary to reach the primary outcome measure, postponing crossovers to surgery after determination of the primary outcome, and modifying the primary outcome measure to include treatment failures. Crossovers should be anticipated and compensated for by increasing the number of participants.

CONCLUSION

Non-adherence to randomly allocated management options can deprive trials of the statistical power needed to inform clinical care. Crossovers and ways to mitigate related problems should be anticipated at the time of trial design.

Understanding how the research question impacts trial design: Examples from discectomy trials

BACKGROUND

Formulating a pertinent research question is of the utmost importance in clinical research. An ill-conceived question may lead to an erroneous trial design, which may adversely affect the care of patients and provide uninformative or even misleading results.

METHODS

We review the research question of a randomized trial on the timing of lumbar discectomy. We compare the resulting design with other trials, real or hypothetical, that would have been more appropriate.

RESULTS

The RCT we examine randomly allocated patients to early or delayed surgery to answer a theoretical question of the effect of time on the efficacy of surgery. The trial was interpreted to have shown that early surgery was associated with better clinical and functional outcomes as compared to delayed surgery. This conclusion is clinically misleading. Valid comparisons between groups should be performed on intent-to-treat analyses and at the same time points after randomization (and not at a fixed follow-up period after surgery). The clinically pertinent comparison is not between the theoretical efficacy of surgery performed at various times, but between surgery and conservative management in patients presenting at various times. Better-designed trials on the clinical benefits of lumbar discectomy, including the treatment of chronic sciatica, have been published.

CONCLUSION

Theoretical research questions inspired from observational data can lead to erroneous trial design. Prospective randomized trials impact practice immediately: they are unique occasions to address clinical problems and optimize care under uncertainty in real time. However, they require the research question to be formulated with great care.

A Pragmatic Randomized Trial Comparing Surgical Clipping and Endovascular Treatment of Unruptured Intracranial Aneurysms

BACKGROUND AND PURPOSE

Surgical clipping and endovascular treatment are commonly used in patients with unruptured intracranial aneurysms. We compared the safety and efficacy of the 2 treatments in a randomized trial.

MATERIALS AND METHODS

Clipping or endovascular treatments were randomly allocated to patients with one or more 3- to 25-mm unruptured intracranial aneurysms judged treatable both ways by participating physicians. The study hypothesized that clipping would decrease the incidence of treatment failure from 13% to 4%, a composite primary outcome defined as failure of aneurysm occlusion, intracranial hemorrhage during follow-up, or residual aneurysms at 1 year, as adjudicated by a core lab. Safety outcomes included new neurologic deficits following treatment, hospitalization of >5 days, and overall morbidity and mortality (mRS > 2) at 1 year. There was no blinding.

RESULTS

Two hundred ninety-one patients were enrolled from 2010 to 2020 in 7 centers. The 1-year primary outcome, ascertainable in 290/291 (99%) patients, was reached in 13/142 (9%; 95% CI, 5%-15%) patients allocated to surgery and in 28/148 (19%; 95% CI, 13%-26%) patients allocated to endovascular treatments (relative risk: 2.07; 95% CI, 1.12-3.83; P = .021). Morbidity and mortality (mRS >2) at 1 year occurred in 3/143 and 3/148 (2%; 95% CI, 1%-6%) patients allocated to surgery and endovascular treatments, respectively. Neurologic deficits (32/143, 22%; 95% CI, 16%-30% versus 19/148, 12%; 95% CI, 8%-19%; relative risk: 1.74; 95% CI, 1.04-2.92; P = .04) and hospitalizations beyond 5 days (69/143, 48%; 95% CI, 40%-56% versus 12/148, 8%; 95% CI, 5%-14%; relative risk: 0.18; 95% CI, 0.11-0.31; P < .001) were more frequent after surgery.

CONCLUSIONS

Surgical clipping is more effective than endovascular treatment of unruptured intracranial aneurysms in terms of the frequency of the primary outcome of treatment failure. Results were mainly driven by angiographic results at 1 year.

Stent-Assisted Coiling in the Treatment of Unruptured Intracranial Aneurysms: A Randomized Clinical Trial

BACKGROUND AND PURPOSE

Stent-assisted coiling may improve angiographic results of endovascular treatment of unruptured intracranial aneurysms compared with coiling alone, but this has never been shown in a randomized trial.

MATERIALS AND METHODS

The Stenting in the Treatment of Aneurysm Trial was an investigator-led, parallel, randomized (1:1) trial conducted in 4 university hospitals. Patients with intracranial aneurysms at risk of recurrence, defined as large aneurysms (≥10 mm), postcoiling recurrent aneurysms, or small aneurysms with a wide neck (≥4 mm), were randomly allocated to stent-assisted coiling or coiling alone. The composite primary efficacy outcome was "treatment failure," defined as initial failure to treat the aneurysm; aneurysm rupture or retreatment during follow-up; death or dependency (mRS > 2); or an angiographic residual aneurysm adjudicated by an independent core laboratory at 12 months. The primary hypothesis (revised for slow accrual) was that stent-assisted coiling would decrease treatment failures from 33% to 15%, requiring 200 patients. Primary analyses were intent to treat.

RESULTS

Of 205 patients recruited between 2011 and 2021, ninety-four were allocated to stent-assisted coiling and 111 to coiling alone. The primary outcome, ascertainable in 203 patients, was reached in 28/93 patients allocated to stent-assisted coiling (30.1%; 95% CI, 21.2%-40.6%) compared with 30/110 (27.3%; 95% CI, 19.4%-36.7%) allocated to coiling alone (relative risk = 1.10; 95% CI, 0.7-1.7; P  = .66). Poor clinical outcomes (mRS >2) occurred in 8/94 patients allocated to stent-assisted coiling (8.5%; 95% CI, 4.0%-16.6%) compared with 6/111 (5.4%; 95% CI, 2.2%-11.9%) allocated to coiling alone (relative risk = 1.6; 95% CI, 0.6%-4.4%; P  = .38).

CONCLUSIONS

The STAT trial did not show stent-assisted coiling to be superior to coiling alone for wide-neck, large, or recurrent unruptured aneurysms.

Understanding the difference between theory and practice: The extracranial-intracranial bypass trials in prevention of ischemic stroke

BACKGROUND

Patients with atherosclerotic carotid or middle cerebral artery occlusions suffer ischemic events that might theoretically be preventable with a surgical extracranial-intracranial bypass, but theory by itself does not justify surgical interventions.

METHODS

We review landmark randomized trials on EC-IC bypass surgery for the treatment of ischemic stroke in patients with atherosclerotic stenoses or occlusions.

RESULTS

The initial EC-IC bypass trial from 1985 did not show any clinical benefit from surgery. The carotid occlusion surgery study (COSS) performed more than 20 years later included only patients highly selected to potentially benefit from bypass by using modern perfusion studies. While EC-IC bypasses were successfully created and they did improve cerebral perfusion, the COSS study also failed to show any clinical benefit to the participating patients.

CONCLUSION

Neurosurgical interventions must not only work in theory; they must improve patient outcomes in real practice.

Research participants may not recall their participation but have a better understanding of alternative management options than patients in routine care

BACKGROUND

Patient understanding of care interventions, of the clinical uncertainty, and of their participation in clinical research is often poor. We hypothesized that compared to routine care, patients would better understand the prevailing uncertainty when they participated in research.

METHODS

A questionnaire was administered to patients at the time they attended a follow-up neurovascular clinic 4 to 52 weeks after a care episode where they did or did not participate in a clinical trial. Patients were not reminded whether they had previously participated in a clinical trial. Questions concerned their understanding of the risks/benefits of interventions, the availability of alternative options, whether their personal opinion was taken into consideration, the reason for the final decision, their confidence at having received the best management, and whether they had been research participants.

RESULTS

Between June 2019 and June 2020, 167 patients were recruited; 71 had truly been research participants, while 96 had not. A greater proportion of research patients were aware of the existence of management alternatives (65% versus 44%; P=0.008). Patients of both groups believed their personal opinion counted in the final decision (76% versus 70%), and patients were equally confident that they had received the best management (94%). Research patients believed they had participated in research 46% of the time, compared to 12% of routine care patients (P=0.003).

CONCLUSION

Many patients do not recall that they participated in a clinical trial, but they have a better understanding of the clinical uncertainty and of the availability of alternative management options.

Understanding how the primary endpoint impacts the interpretation of trial results: The Japanese Adult Moyamoya bypass trial

BACKGROUND

The meaning of a clinical trial depends to a large extent on the choice of the primary outcome measure, which can be explanatory or pragmatic.

METHODS

We review the Japanese Adult Moyamoya (JAM) trial, that compared surgical extracranial to intracranial (EC-IC) bypass and medical management of hemorrhagic moyamoya disease. We also review some principles which guide the selection of the primary trial endpoint.

DISCUSSION

The main component of the primary outcome measure in JAM was rebleeding, a surrogate outcome that allowed investigators to demonstrate that surgical bypass had causal efficacy. However, the number of patients with a poor outcome, defined as those with a modified Rankin score (mRS)>2, would have been a more pragmatic choice. Unfortunately, the trial was too small to show that patients benefited from surgery.

CONCLUSION

The JAM trial showed that EC-IC bypass can decrease rebleeding in moyamoya patients, but whether patients have better outcomes with surgery remains uncertain. Hard pragmatic clinical primary outcome measures are necessary to guide surgical care.

Study of "ALS reversals": LifeTime environmental exposures (StARLiTE)

We previously reported on a series of patients diagnosed with ALS whom had an extraordinary course defined by substantial and sustained improvement in weakness and function. For this study, twenty-five of these "ALS Reversals" completed extensive environmental exposure questionnaires. These responses were then compared to a large database of prior responses from patients with typically progressive ALS (n = 6187). The results demonstrated that the "Reversal" participants have had a diverse number of exposures with substantial heterogeneity. In general, this was similar to the control group; however, there were a few specific differences that could be further explored in future research.

Understanding explanatory and pragmatic trials: Examples from randomized controlled trials on vertebroplasty

OBJECTIVES

To better understand the explanatory-pragmatic distinction in the design and interpretation of randomized controlled trials (RCTs).

METHODS

We review the explanatory-pragmatic distinction in clinical trial design. We use the PRECIS-2 tool to evaluate the trial design of selected RCTs on percutaneous vertebroplasty for osteoporotic vertebral compression fractures. We discuss difficulties in the selection of criteria and in the construction of PRECIS diagrams. We also examine how inconsistency in the selection of various items of trial design can cause confusion in the interpretation of results.

RESULTS

The selection of criteria and the scoring of multiple PRECIS domains were subjective and thus debatable. The pragmascope patterns of various vertebroplasty trials were heterogeneous. Many trials had both pragmatic and explanatory components. Some placebo-controlled trial goals seem to have been explanatory, but their design actually included enough pragmatic items such that the meaning of negative trial results remains ambiguous.

CONCLUSION

The results of a trial cannot be interpreted without understanding the various design choices made along the explanatory-pragmatic spectrum.

Understanding power in randomized trials: The example of vertebroplasty

OBJECTIVES

Randomized trials (RCTs) should include a sufficient number of patients to reduce the risk that the observed outcome is a result of chance rather than from a truly different treatment effect. Trials must be even larger to claim an absence of treatment effect in a placebo-controlled trial. To estimate the size of the trial and maximize power, it is often suggested to use a comparison between the means of a continuous variable.

METHODS

We examine the RCTs that have compared vertebroplasty and placebo for patients with osteoporotic fractures. Most trials compared the means of a continuous pain score to yield implausibly small trials, as small as 24 patients per group.

RESULTS

The minimally significant difference between groups has no precise clinical meaning for patients when it is based on a comparison of means of pain scores. A comparison of the proportions of patients reaching a per-patient outcome measure of treatment success is much more pertinent if the trial is to inform the care of future patients. The resulting trials will admittedly need to be larger, but they will be much less likely to fall prey to the 'evidence of absence' fallacy. Furthermore, trial size should also take into consideration harder clinical outcome measures, such as death and disability.

CONCLUSION

When the goal of a trial is to inform outcome-based medical care, comparing the proportions of patients reaching a clinically pertinent outcome is more appropriate than comparing the means of a continuous variable.

Understanding the research-care demarcation and why it must be revised

BACKGROUND

A clean-cut separation between research and care was artificially created at the time of the Belmont report more than 40 years ago. The demarcation was initially controversial but eventually was implemented for political reasons. We examine why it must be revised.

METHODS

We review historical research scandals as well as the theoretical basis for the Belmont demarcation. We then discuss consequences on medical practice and propose an alternative.

DISCUSSION

Most research scandals involved abusing human beings supposedly for the sake of science. Belmont commissioners were aware the research/care problem was double-headed. While research subjects should be protected from abuse in the research context, patients need to be protected from unvalidated medical and surgical interventions in the care context. For political reasons the Commission recommended the regulation of research but to leave medical practice untouched. Thus the Commission had to distinguish research from care. The notion of 'generalizable knowledge' was introduced to define and regulate research, but the inadvertent result was that by trying to protect research subjects, the regulation has not only failed to protect all other patients, but also encouraged the widespread practice of unvalidated interventions within the care context. The notion of validated care should be re-introduced into a proper analysis of the care-research demarcation, for care research is an integral ingredient of a good medical practice.

CONCLUSION

The research-care demarcation should be revised to leave room for the validated/unvalidated care distinction. Care research, essential to guide medical practice, should be facilitated at all levels.

Understanding the choice of control group: A systematic review of vertebroplasty trials for osteoporotic vertebral compression fractures

OBJECTIVES

To better understand the choice of the comparator intervention in the design of clinical trials and its impact on the meaning of results we review randomized trials on vertebroplasty.

METHODS

We conducted a systematic and narrative review of all randomized trials on vertebroplasty. Trials are categorized according to the comparator intervention (non-surgical management, placebo/sham vertebroplasty, and kyphoplasty).

RESULTS

All trials were too small to show a difference in objective clinical outcomes, and 20 of 23 RCTs used mean pain scores to compare interventions. Most trials comparing vertebroplasty with non-surgical management concluded that vertebroplasty was superior. Trials comparing kyphoplasty with vertebroplasty showed similar results for both interventions. However, 4 of 5 trials comparing vertebroplasty with placebo surgery failed to show a significant difference between groups.

CONCLUSION

The clinical results of an intervention cannot be interpreted without a comparison that involves a control group. The choice of comparator intervention can change the meaning of the trial. A large pragmatic trial, using hard clinical outcomes such as morbidity and mortality as a primary outcome measure, would be needed to assess the potential clinical benefits of vertebroplasty.

Successful Reperfusion is Associated with Favorable Functional Outcome despite Vessel Perforation during Thrombectomy: A Case Series and Systematic Review

BACKGROUND

Arterial perforation is a potentially serious complication during endovascular thrombectomy.

PURPOSE

Our aim was to describe interventional approaches after arterial perforation during endovascular thrombectomy and to determine whether reperfusion remains associated with favorable outcome despite this complication.

DATA SOURCES

Data from consecutive patients with acute stroke undergoing endovascular thrombectomy were retrospectively collected between 2015 to 2020 from a single-center cohort, and a systematic review was performed using PubMed, EMBASE, and Ovid MEDLINE up to June 2020.

STUDY SELECTION

Articles reporting functional outcome after arterial perforation during endovascular thrombectomy were selected.

DATA ANALYSIS

Functional outcomes of patients achieving successful reperfusion (TICI 2b/3) were compared with outcomes of those with unsuccessful reperfusion in our single-center cohort. We then summarized the literature review to describe interventional approaches and outcomes after arterial perforation during endovascular thrombectomy.

DATA SYNTHESIS

In our single-center cohort, 1419 patients underwent endovascular thrombectomy, among whom 32 (2.3%) had vessel perforation and were included in the analysis. The most common hemostatic strategy was watchful waiting (71% of cases). Patients with successful reperfusion had a higher proportion of favorable 90-day mRS scores (60% versus 12.5%; P = .006) and a lower mortality rate (13.3% versus 56.3%, P = .01) than patients without successful reperfusion. Thirteen articles were included in the systematic review. Successful reperfusion also appeared to be associated with better outcomes.

LIMITATIONS

Given the low number of published reports, we performed only a descriptive analysis.

CONCLUSIONS

Arterial perforation during endovascular thrombectomy is rare but is associated with high mortality rates and poor outcome. However, successful reperfusion remains correlated with favorable outcome in these patients.

Surgical or Endovascular Treatment of MCA Aneurysms: An Agreement Study

BACKGROUND AND PURPOSE

MCA aneurysms are still commonly clipped surgically despite the recent development of a number of endovascular tools and techniques. We measured clinical uncertainty by studying the reliability of decisions made for patients with middle cerebral artery (MCA) aneurysms.

MATERIALS AND METHODS

A portfolio of 60 MCA aneurysms was presented to surgical and endovascular specialists who were asked whether they considered surgery or endovascular treatment to be an option, whether they would consider recruitment of the patient in a randomized trial, and whether they would provide their final management recommendation. Agreement was studied using κ statistics. Intrarater reliability was assessed with the same, permuted portfolio of cases of MCA aneurysm sent to the same specialists 1 month later.

RESULTS

Surgical management was the preferred option for neurosurgeons (n = 844/1320; [64%] responses/22 raters), while endovascular treatment was more commonly chosen by interventional neuroradiologists (1149/1500 [76.6%] responses/25 raters). Interrater agreement was only "slight" for all cases and all judges (κ = 0.094; 95% CI, 0.068-0.130). Agreement was no better within specialties or with more experience. On delayed requestioning, 11 of 35 raters (31%) disagreed with themselves on at least 20% of cases. Surgical management and endovascular treatment were always judged to be a treatment option, for all patients. Trial participation was offered to patients 65% of the time.

CONCLUSIONS

Individual clinicians did not agree regarding the best management of patients with MCA aneurysms. A randomized trial comparing endovascular with surgical management of patients with MCA aneurysms is in order.

Flow Diversion in the Treatment of Intracranial Aneurysms: A Pragmatic Randomized Care Trial

BACKGROUND AND PURPOSE

Flow diversion is a recent endovascular treatment for intracranial aneurysms. We compared the safety and efficacy of flow diversion with the alternative standard management options.

MATERIALS AND METHODS

A parallel group, prerandomized, controlled, open-label pragmatic trial was conducted in 3 Canadian centers. The trial included all patients considered for flow diversion. A Web-based platform 1:1 randomly allocated patients to flow diversion or 1 of 4 alternative standard management options (coiling with/without stent placement, parent vessel occlusion, surgical clipping, or observation) as prespecified by clinical judgment. Patients ineligible for alternative standard management options were treated with flow diversion in a registry. The primary safety outcome was death or dependency (mRS > 2) at 3 months. The composite primary efficacy outcome included the core lab-determined angiographic presence of a residual aneurysm, aneurysm rupture, progressive mass effect during follow-up, or death or dependency (mRS > 2) at 3-12 months.

RESULTS

Between May 2011 and November 2020, three hundred twenty-three patients were recruited: Two hundred seventy-eight patients (86%) had treatment randomly allocated (139 to flow diversion and 139 to alternative standard management options), and 45 (14%) received flow diversion in the registry. Patients in the randomized trial frequently had unruptured (83%), large (52% ≥10 mm) carotid (64%) aneurysms. Death or dependency at 3 months occurred in 16/138 patients who underwent flow diversion and 12/137 patients receiving alternative standard management options (relative risk, 1.33; 95% CI, 0.65-2.69; P = .439). A poor primary efficacy outcome was found in 30.9% (43/139) with flow diversion and 45.6% (62/136) of patients receiving alternative standard management options, with an absolute risk difference of 14.7% (95% CI, 3.3%-26.0%; relative risk, 0.68; 95% CI, 0.50-0.92; P = .014).

CONCLUSIONS

For patients with mostly unruptured, large, anterior circulation (carotid) aneurysms, flow diversion was more effective than the alternative standard management option in terms of angiographic outcome.

Understanding the problems with recruitment in surgical randomized trials: A lesson from landmark trials on temporal lobe epilepsy

BACKGROUND

Surgical randomized trials are difficult to accomplish. One major problem is recruitment of a sufficient number of patients to address the clinical problem.

METHODS

We review the various ways patient recruitment in surgical RCTs can be promoted. We examine two landmark trials on the surgical treatment of temporal lobe epilepsy (TLE), one that was successful, and one which did not attain the target number of participants.

DISCUSSION

Both designs of the Canadian and American trials of surgery for TLE included a benefit to participants: the Canadian trial gave a chance to have immediate access to investigation and treatment, as compared to a 1 year delay (considered 'standard care' in that center), while the American trial offered free surgical management to both arms. Patients were recruited and treatments randomly allocated prior to knowing for certain whether they were surgical candidates or not. This design choice may have helped circumvent the 'equipoise problem'. The Canadian trial offered participation to drug-resistant patients that were already routinely referred to surgical centers, while the success of the American trial which limited recruitment to the early period of drug resistance was dependent on a change of practice of referring clinicians which did not materialize.

CONCLUSION

The surgical treatment of drug-resistant temporal lobe epilepsy has been validated using RCT methods. Ways to promote participation in surgical trials should be further investigated.

Understanding burden of proof and equipoise in the design of pragmatic clinical trials: An example from a trial on brain arteriovenous malformations

BACKGROUND AND PURPOSE

The burden of proof principle is rarely discussed and poorly understood, but central to the proper design of pragmatic clinical trials. A better understanding of the principle could play an important role in the re-introduction of scientific methods within practice and in revising fundamental problems with the current research-care separation.

METHODS

We analyze the design of the ARUBA trial on the management of unruptured brain arteriovenous malformations. We also review how the concept of clinical equipoise was introduced to address a misconceived problem of research ethics.

RESULTS

The ARUBA trial hypothesis in favour of conservative management of brain arteriovenous malformations failed to take into account the fact that the burden of proof was on surgery, endovascular treatment or radiation therapy. Thus, results remained inconclusive and other trials are needed. The equipoise notion fails to take into account that the burden of proof is on unvalidated medical or surgical interventions, if we want to provide outcome-based medical care that patients can trust.

CONCLUSION

The burden of proof principle is essential to properly design pragmatic trials. This principle also explains why in certain circumstances optimal care is a randomized care trial.

Understanding the reliability of trial outcome measures: The example of angiographic results of surgical or endovascular treatments of aneurysms

BACKGROUND

The reliability of outcome measures is of central importance in clinical research. Studies of reliability remain rare in the neurovascular field.

METHODS

A narrative review of the history (1997-2021) of reporting angiographic results of the surgical or endovascular treatments of aneurysms serves to illustrate the importance of precisely defining outcome measures. We also review how the reliability of an angiographic classification system was studied.

DISCUSSION

Outcome measures are commonly used without precise definitions. When definitions or classification systems exist, they are rarely verified for their reliability. Twenty-five years following its introduction, a classification of angiographic results of aneurysm treatments is still being studied and modified.

CONCLUSION

The reliability of outcome measures should be made a research priority if we are to practice outcome-based medical or surgical care.

Transcranial Doppler Velocities and Angiographic Vasospasm after SAH: A Diagnostic Accuracy Study

BACKGROUND AND PURPOSE

After aneurysmal SAH, transcranial Doppler is commonly used to monitor cerebral vasospasm. The diagnostic accuracy of transcranial Doppler flow velocity values in detecting angiographic vasospasm in patients requiring urgent endovascular intervention has not been established.

MATERIALS AND METHODS

We performed a retrospective analysis of a consecutive series of patients with aneurysmal SAH who underwent transcranial Doppler (index test) within 24 hours of conventional angiography (reference test). The judgment of 33%, 50%, and 66% degree of vessel narrowing on angiography was independently established by multiple neuroendovascular clinicians. Vessel-specific per-segment and per-patient transcranial Doppler velocities were studied using receiver operating characteristic curves, the Youden index, and minimal acceptable sensitivity models. Optimal mean flow-velocity thresholds were explored to calculate sensitivity and specificity using a per-patient judgment of vasospasm of at least 50% angiographic narrowing in any large arterial segment except A1.

RESULTS

In 221 patients, vasospasm was found in 15%, 8%, and 4% of arteries when the degree of reference angiographic luminal narrowing was 33%, 50%, and 66%, respectively. Mean flow velocities were significantly higher in vasospastic segments (P = . 001), but per-segment exploratory analyses yielded unsound mean flow velocity thresholds. The Youden and minimal acceptable sensitivity models proposed mean flow velocity thresholds of approximately 160 cm/s for the anterior circulation and 80 cm/s for the posterior circulation in the per-patient diagnosis of angiographic vasospasm (≥50%), yielding a sensitivity of 80%-90% (95% CI, 0.77-0.96), but with a corresponding specificity of 50% (95% CI, 0.40-0.56).

CONCLUSIONS

In this study, a threshold transcranial Doppler mean flow-velocity value that would accurately diagnose ≥50% angiographic vasospasm remained elusive.

Noninvasive Angiographic Results of Clipped or Coiled Intracranial Aneurysms: An Inter- and Intraobserver Reliability Study

BACKGROUND AND PURPOSE

Noninvasive angiography is commonly used to assess the outcome of surgical or endovascular treatment of intracranial aneurysms in clinical series or randomized trials. We sought to assess whether a standardized 3-grade classification system could be reliably used to compare the CTA and MRA results of both treatments.

MATERIALS AND METHODS

An electronic portfolio composed of CTAs of 30 clipped and MRAs of 30 coiled aneurysms was independently evaluated by 24 raters of diverse experience and training backgrounds. Twenty raters performed a second evaluation 1 month later. Raters were asked which angiographic grade and management decision (retreatment; close or long-term follow-up) would be most appropriate for each case. Agreement was analyzed using the Krippendorff α (α_K) statistic, and the relationship between angiographic grade and clinical management choice, using the Fisher exact and Cramer V tests.

RESULTS

Interrater agreement was substantial (α_K = 0.63; 95% CI, 0.55-0.70); results were slightly better for MRA results of coiling (α_K = 0.69; 95% CI, 0.56-0.76) than for CTA results of clipping (α_K = 0.58; 95% CI, 0.44-0.69). Intrarater agreement was substantial to almost perfect. Interrater agreement regarding clinical management was moderate for both clipped (α_K = 0.49; 95% CI, 0.32-0.61) and coiled subgroups (α_K = 0.47; 95% CI, 0.34-0.54). The choice of clinical management was strongly associated with the size of the residuum (mean Cramer V = 0.77 [SD, 0.14]), but complete occlusions (grade 1) were followed more closely after coiling than after clipping (P = .01).

CONCLUSIONS

A standardized 3-grade scale was found to be a reliable and clinically meaningful tool to compare the results of clipping and coiling of aneurysms using CTA or MRA.

Sticholysin II shows similar immunostimulatory properties to LLO stimulating dendritic cells and MHC-I restricted T cell responses of heterologous antigen

Induction of CD8⁺ T cell responses against tumor cells and intracellular pathogens is an important goal of modern vaccinology. One approach of translational interest is the use of liposomes encapsulating pore-forming proteins (PFPs), such as Listeriolysin O (LLO), which has shown efficacy at priming strong and sustained CD8⁺ T cell responses. Recently, we have demonstrated that Sticholysin II (StII), a PFP from the sea anemone Stichodactyla helianthus, co-encapsulated into liposomes with ovalbumin (OVA) was able to stimulate, antigen presenting cells, antigen-specific CD8⁺ T cells and anti-tumor activity in mice. In the present study, we aimed to compare StII and LLO in terms of their abilities to stimulate dendritic cells and to induce major histocompatibility complex (MHC) class I restricted T cell responses against OVA. Interestingly, StII exhibited similar abilities to LLO in vitro of inducing dendritic cells maturation, as measured by increased expression of CD40, CD80, CD86 and MHC-class II molecules, and of stimulating OVA cross-presentation to a CD8⁺ T cell line. Remarkably, using an ex vivo Enzyme-Linked ImmunoSpot Assay (ELISPOT) to monitor gamma interferon (INF-γ) producing effector memory CD8⁺ T cells, liposomal formulations containing either StII or LLO induced comparable frequencies of OVA-specific INF-γ producing CD8⁺ T cells in mice that were sustained in time. However, StII-containing liposomes stimulated antigen-specific memory CD8⁺ T cells with a higher potential to secrete IFN-γ than liposomes encapsulating LLO. This StII immunostimulatory property further supports its use for the rational design of T cell vaccines against cancers and intracellular pathogens. In summary, this study indicates that StII has immunostimulatory properties similar to LLO, despite being evolutionarily distant PFPs.

Douglas Altman's 2009 Grand Lecture: Can we trust our literature?

Recent studies of the medical literature have revealed numerous and serious problems. Errors in the design, methods and interpretation of studies can frequently be identified. A huge hidden problem is publication bias, the tendency for positive articles to be published, while negative articles are either not written or submitted. This can systematically lead to an overestimation of the value of treatments, of diagnostic or prognostic studies. Even more worrisome is selective reporting: only a subset of a wide array of tested hypotheses are presented (the ones that turned out to be positive with significance testing). This is particularly true for secondary endpoints and subgroup findings, but even the primary endpoints of trials have been modified when publications are compared to protocols. The peer-review process is fallible. Even if it were strengthened, reviewers cannot examine what is not reported. Hence many problems can only be mitigated with better reporting. Numerous initiatives have proposed guidelines to promote transparent reporting, but progress is slow.

Interobserver Agreement in Scoring Angiographic Results of Basilar Artery Occlusion Stroke Therapy

BACKGROUND AND PURPOSE

The modified TICI Infarction grading system is a metric currently used to evaluate angiographic results of thrombectomy for large-vessel occlusion in ischemic stroke. Originally designed for evaluating MCA territories, it is currently used for other vessel occlusions, including the posterior circulation. We postulate that the modified TICI use for the posterior circulation is not accurate due to the different vascular territories supplied by vertebrobasilar vasculature, making grading more complex.

MATERIALS AND METHODS

We collected angiographic results from 30 patients who presented with acute posterior circulation occlusions between 2015 and 2018 and underwent thrombectomy in our institution. Eight observers were asked to evaluate the TICI scores before and after thrombectomy. The multirater statistics were computed using Fleiss κ analysis. Further data were collected regarding the potential brain territories at risk and the existence of atherosclerotic disease in the basilar artery.

RESULTS

The overall agreement κ reached 0.277 (SD, 0.013), which suggests a "fair" agreement among the raters. On average, 45% of observers achieved a high accuracy in predicting brain areas at risk of ischemia. As for the existence of basilar atherosclerotic disease, a high agreement (defined as at least 5 of 6 observers) was seen in 20 of the 30 patients.

CONCLUSIONS

Despite TICI being ubiquitous in stroke diagnostics, the high variability of posterior circulation TICI scores calls into question its use in these strokes. Other methods should be developed to assess recanalization in the posterior circulation.

Unruptured aneurysms: Why observational studies fall short no matter how "Big" the Data

BACKGROUND

The best management of unruptured intracranial aneurysms (UIAs) remains unknown, despite multiple observational studies. A randomized trial (RCT) is in order. Yet, a National Institute Neurological Disorders and Stroke workshop has once again proposed to use prospective observational studies (POS) of large databases to address such problems.

METHODS

We review the historical misconceptions that have been associated with observations of UIAs and their treatments. We critically examine some recent methods that have been proposed to address shortcomings of observational studies. We finally review the ethical principles underlying the use of trial methods in the care of patients.

RESULTS

Replacing RCTs with POS submits patients to management options that have never been proven beneficial, while making them involuntary research subjects of studies that are inevitably biased. A science of practice cannot be an outsider's examination of the behavior of clinicians incapable of questioning their practice. The thesis we propose is that a science of practice must not only eventually determine what best practice will be; It must engage agents involved in medical practice to transparently reveal the uncertainty that calls for management options to be offered under the guidance of declared and controlled care research, to optimize patient outcomes in spite of the uncertainty.

CONCLUSION

To use POS rather than RCTs in medical practice is to renege on scientific and ethical principles that characterize modern medicine. Instead, we must learn to integrate care research into our practice to provide optimal medical care in real time.

Reliability of the Diagnosis of Cerebral Vasospasm Using Catheter Cerebral Angiography: A Systematic Review and Inter- and Intraobserver Study

BACKGROUND AND PURPOSE

Conventional angiography is the benchmark examination to diagnose cerebral vasospasm, but there is limited evidence regarding its reliability. Our goals were the following: 1) to systematically review the literature on the reliability of the diagnosis of cerebral vasospasm using conventional angiography, and 2) to perform an agreement study among clinicians who perform endovascular treatment.

MATERIALS AND METHODS

Articles reporting a classification system on the degree of cerebral vasospasm on conventional angiography were systematically searched, and agreement studies were identified. We assembled a portfolio of 221 cases of patients with subarachnoid hemorrhage and asked 17 raters with different backgrounds (radiology, neurosurgery, or neurology) and experience (junior ≤10 and senior >10 years) to independently evaluate cerebral vasospasm in 7 vessel segments using a 3-point scale and to evaluate, for each case, whether findings would justify endovascular treatment. Nine raters took part in the intraobserver reliability study.

RESULTS

The systematic review showed a very heterogeneous literature, with 140 studies using 60 different nomenclatures and 21 different thresholds to define cerebral vasospasm, and 5 interobserver studies reporting a wide range of reliability (κ = 0.14-0.87). In our study, only senior raters reached substantial agreement (κ ≥ 0.6) on vasospasm of the supraclinoid ICA, M1, and basilar segments and only when assessments were dichotomized (presence or absence of ≥50% narrowing). Agreement on whether to proceed with endovascular management of vasospasm was only fair (κ ≤ 0.4).

CONCLUSIONS

Research on cerebral vasospasm would benefit from standardization of definitions and thresholds. Dichotomized decisions by experienced readers are required for the reliable angiographic diagnosis of cerebral vasospasm.

Exophthalmos following mechanical thrombectomy for anterior circulation stroke: A retrospective study and review of literature

BACKGROUND

Anecdotal cases of exophthalmos after acute mechanical thrombectomy have been described. We sought to estimate the incidence in a large cohort of patients with acute anterior circulation stroke treated with mechanical thrombectomy. Secondarily, we aimed to evaluate the underlying mechanism and to differentiate it on imaging from other pathology with similar clinical orbital features.

METHODS

Between November 2016 and November 2018, we performed a retrospective single-center study of 250 patients who underwent anterior circulation mechanical thrombectomy. Development of exophthalmos was independently evaluated by two readers on preprocedure and 24-h postprocedure non-contrast cerebral CT.

RESULTS

In the mechanical thrombectomy cohort, six individuals (2.4%) developed interval ipsilateral exophthalmos at 24 h. Of these, at least two patients developed clinical symptoms. There was almost perfect agreement between assessments of the two readers (Cohen's kappa = 0.907 (95% confidence interval: 0.726, 1.000)). In two patients, there was delayed ophthalmic artery filling on digital subtraction angiography. None of the patients had features of a direct carotid-cavernous fistula.

CONCLUSIONS

Exophthalmos is not uncommon after mechanical thrombectomy (2.4%). The underlying mechanism is difficult to confirm, but it is most likely due to orbital ischemia from hypoperfusion or distal emboli.

Reliability of CT Angiography in Cerebral Vasospasm: A Systematic Review of the Literature and an Inter- and Intraobserver Study

BACKGROUND AND PURPOSE

Computed tomography angiography offers a non-invasive alternative to DSA for the assessment of cerebral vasospasm following subarachnoid hemorrhage but there is limited evidence regarding its reliability. Our aim was to perform a systematic review (Part I) and to assess (Part II) the inter- and intraobserver reliability of CTA in the diagnosis of cerebral vasospasm.

MATERIALS AND METHODS

In Part I, articles reporting the reliability of CTA up to May 2018 were systematically searched and evaluated. In Part II, 11 raters independently graded 17 arterial segments in each of 50 patients with SAH for the presence of vasospasm using a 4-category scale. Raters were additionally asked to judge the presence of any moderate/severe vasospasm (≥ 50% narrowing) and whether findings would justify augmentation of medical treatment or conventional angiography ± balloon angioplasty. Four raters took part in the intraobserver reliability study.

RESULTS

In Part I, the systematic review revealed few studies with heterogeneous vasospasm definitions. In Part II, we found interrater reliability to be moderate at best (κ ≤ 0.6), even when results were stratified according to specialty and experience. Intrarater reliability was substantial (κ > 0.6) in 3/4 readers. In the per arterial segment analysis, substantial agreement was reached only for the middle cerebral arteries, and only when senior raters' judgments were dichotomized (presence or absence of ≥50% narrowing). Agreement on the medical or angiographic management of vasospasm based on CTA alone was less than substantial (κ ≤ 0.6).

CONCLUSIONS

The diagnosis of vasospasm using CTA alone was not sufficiently repeatable among observers to support its general use to guide decisions in the clinical management of patients with SAH.

Does Increasing Packing Density Using Larger Caliber Coils Improve Angiographic Results of Embolization of Intracranial Aneurysms at 1 Year: A Randomized Trial

BACKGROUND AND PURPOSE

The impact of increased aneurysm packing density on angiographic outcomes has not been studied in a randomized trial. We sought to determine the potential for larger caliber coils to achieve higher packing densities and to improve the angiographic results of embolization of intracranial aneurysms at 1 year.

MATERIALS AND METHODS

Does Embolization with Larger Coils Lead to Better Treatment of Aneurysms (DELTA) was an investigator-initiated multicenter prospective, parallel, randomized, controlled clinical trial. Patients had 4- to 12-mm unruptured aneurysms. Treatment allocation to either 15- (experimental) or 10-caliber coils (control group) was randomized 1:1 using a Web-based platform. The primary efficacy outcome was a major recurrence or a residual aneurysm at follow-up angiography at 12 ± 2 months adjudicated by an independent core lab blinded to the treatment allocation. Secondary outcomes included indices of treatment success and standard safety outcomes. Recruitment of 564 patients was judged necessary to show a decrease in poor outcomes from 33% to 20% with 15-caliber coils.

RESULTS

Funding was interrupted and the trial was stopped after 210 patients were recruited between November 2013 and June 2017. On an intent-to-treat analysis, the primary outcome was reached in 37 patients allocated to 15-caliber coils and 36 patients allocated to 10-caliber coils (OR = 0.931; 95% CI, 0.528-1.644; P = .885). Safety and other clinical outcomes were similar. The 15-caliber coil group had a higher mean packing density (37.0% versus 26.9%, P = .0001). Packing density had no effect on the primary outcome when adjusted for initial angiographic results (OR = 1.001; 95% CI, 0.981-1.022; P = .879).

CONCLUSIONS

Coiling of aneurysms randomized to 15-caliber coils achieved higher packing densities compared with 10-caliber coils, but this had no impact on the angiographic outcomes at 1 year, which were primarily driven by aneurysm size and initial angiographic results.