Evidence based purchasing: understanding results of clinical trials and systematic reviews

Number needed to treat and associated cost analysis of cenobamate versus third-generation anti-seizure medications for the treatment of focal-onset seizures in patients with drug-resistant epilepsy in Spain

INTRODUCTION

Epilepsy is a serious neurological disease, ranking high in the top causes of disability. The main goal of its treatment is to achieve seizure freedom without intolerable adverse effects. However, approximately 40% of patients suffer from Drug-Resistant Epilepsy (DRE) despite the availability of the latest options called third-generation Anti-Seizure Medications(ASMs). Cenobamate is the first ASM approved in Spain for the adjunctive treatment of Focal-Onset Seizures (FOS) in adult patients with DRE. The introduction of a new drug increases the number of therapeutic options available, making it important to compare it with existing alternatives in terms of clinical benefit and efficiency.

PURPOSE

This study aimed to compare the clinical benefit, in terms of the Number Needed to Treat (NNT), and the efficiency, in terms of Cost per NNT (CNT), associated with cenobamate versus third-generation ASMs used in Spain for the adjunctive treatment of FOS in patients with DRE.

METHODS

The Number Needed to Treat data was calculated based on the ≥50% responder rate and seizure freedom endpoints (defined as the percentage of patients achieving 50% and 100% reduction in seizure frequency, respectively), obtained from pivotal clinical trials performed with cenobamate, brivaracetam, perampanel, lacosamide, and eslicarbazepine acetate. The NNT was established as the inverse of the treatment responder rate minus the placebo responder rate and was calculated based on the minimum, mid-range Daily Defined Dose (DDD), and maximum doses studied in the pivotal clinical trials of each ASM. CNT was calculated by multiplying the annual treatment cost by NNT values for each treatment option.

RESULTS

In terms of NNT, cenobamate was the ASM associated with the lowest values at all doses for both ≥50% responder rate and seizure freedom compared with the alternatives. In terms of CNT, for ≥50% responder rate, cenobamate was the ASM associated with the lowest CNT values at DDD and lacosamide and eslicarbazepine acetate at the minimum and maximum dose, respectively. For seizure freedom, cenobamate was associated with the lowest CNT value at DDD and the maximum dose and lacosamide at the minimum dose.

CONCLUSIONS

Cenobamate could represent the most effective ASM in all doses studied compared to the third-generation ASMs and the most efficient option at DDD for both ≥50% responder rate and seizure freedom. This study could represent an important contribution towards informed decision-making regarding the selection of the most appropriate therapy for FOS in adult patients with DRE from a clinical and economical perspective in Spain.

How much is enough? Considering minimally important change in youth mental health outcomes

To make decisions in mental health care, service users, clinicians, and administrators need to make sense of research findings. Unfortunately, study results are often presented as raw questionnaire scores at different time points and regression coefficients, which are difficult to interpret with regards to their clinical meaning. Other commonly reported treatment outcome indicators in clinical trials or meta-analyses do not convey whether a given change score would make a noticeable difference to service users. There is an urgent need to improve the interpretability and relevance of outcome indicators in youth mental health (aged 12-24 years), in which shared decision making and person-centred care are cornerstones of an ongoing global transformation of care. In this Personal View, we make a case for considering minimally important change (MIC) as a meaningful, accessible, and user-centred outcome indicator. We discuss what the MIC represents, how it is calculated, and how it can be implemented in dialogues between clinician and researcher, and between youth and clinician. We outline how use of the MIC could enhance reporting in clinical trials, meta-analyses, clinical practice guidelines, and measurement-based care. Finally, we identify current methodological challenges around estimating the MIC and areas for future research. Efforts to select outcome domains and valid measurement instruments that resonate with youth, families, and clinicians have increased in the past 5 years. In this context, now is the time to define demarcations of changes in outcome scores that are clinically relevant, and meaningful to youth and families. Through the use of MIC, youth-centred outcome measurement, analysis, and reporting would support youth-centred therapeutic decision making.

[Interpretation of systematic review articles and meta-analyses : Clinical trials in trauma surgery and orthopedics]

Clinical trials must be planned and interpreted in the context of current best clinical and scientific evidence, undoubtedly provided by systematic reviews and meta-analyses, especially Cochrane Reviews. While many clinicians feel overwhelmed by this complex data source, few visualElements (e.g., the traffic light system of the Cochrane risk of bias [RoB‑2] tool, forest plots, etc.), together with indices such as the I2 heterogeneity statistic, allow for a quick appraisal of all critical and necessary qualitative and quantitative information. The effectiveness of different treatment options can indirectly be assessed by methodological advancements like network meta-analyses.Point estimates of percentages are insufficient to describe the utility and value of a proposed novel intervention, which, in orthopedic and trauma surgery, often represents a step innovation. 95% confidence intervals and the so-called fragility index are helpful in determining the ultimate patient benefit.

[Evidence-based medicine in orthopedics and trauma surgery]

Medical knowledge is rapidly expanding but which research results in orthopedics and trauma surgery are actually suitable to change treatment decisions in the daily routine? Evidence-based medicine is a process to search for reliable answers to this and in the assessment of scientific data relies on an individual systematic evaluation with quality criteria and defined statistics. The core procedure is to filter out the true patient benefits from study results by a structured analysis in order to derive an algorithm to solve the clinical problem. The clinical expertise, the knowledge and the strength of judgement of orthopedic and trauma surgeons play a central role in the assessment of the calculated evidence-based key figures.

Problems with Using Polygenic Scores to Select Embryos

Companies have recently begun to sell a new service to patients considering in vitro fertilization: embryo selection based on polygenic scores (ESPS). These scores represent individualized predictions of health and other outcomes derived from genomewide association studies in adults to partially predict these outcomes. This article includes a discussion of many factors that lower the predictive power of polygenic scores in the context of embryo selection and quantifies these effects for a variety of clinical and nonclinical traits. Also discussed are potential unintended consequences of ESPS (including selecting for adverse traits, altering population demographics, exacerbating inequalities in society, and devaluing certain traits). Recommendations for the responsible communication about ESPS by practitioners are provided, and a call for a society-wide conversation about this technology is made. (Funded by the National Institute on Aging and others.).

The Barrier to Informed Choice in Cancer Screening: Statistical Illiteracy in Physicians and Patients

An efficient health care requires both informed doctors and patients. Our current healthcare system falls short on both counts. Most doctors and patients do not understand the available medical evidence. To illustrate the extent of the problem in the setting of cancer screening: In a representative sample of some 5000 women in nine European countries, 92% overestimated the reduction of breast cancer mortality by mammography by a factor of 10-200, or did not know. For a similar sample of about 5000 men with respect to PSA screening, this number was 89%. Of more than 300 US citizens who regularly attended one or more cancer screening test, more than 90% had never been informed about the biggest harms of screening-overdiagnosis and overtreatment-by their physicians. Among 160 German gynecologists, some 80% did not understand the positive predictive value of a positive mammogram, with estimates varying between 1 and 90%. In a national sample of 412 US primary care physicians, 47% mistakenly believed that if more cancers are detected by a screening test, this proves that the test saves lives, and 76% wrongly thought that if screen-detected cancers have better 5-year survival rates than cancers detected by symptoms, this would prove that the screening test saves lives. And of 20 German gynecologists, not a single one provided a woman with all information on the benefits and harms of cancer screening required in order to make an informed choice. Why is risk literacy so scarce in health care? One frequently discussed explanation assumes that people suffer from cognitive deficits that make them predictably irrational and basically hopeless at dealing with risks, so that they need to be "nudged" into healthy behavior. Yet research has demonstrated that the problem lies less in stable cognitive deficits than in how information is presented to physicians and patients. This includes biased reporting in medical journals, brochures, and the media that uses relative risks and other misleading statistics, motivated by conflicts of interest and defensive medicine that do not promote informed physicians and patients. What can be done? Every medical school should teach its students how to understand evidence in general and health statistics in particular. To cultivate informed patients, elementary and high schools should start teaching the mathematics of uncertainty-statistical thinking. Guidelines about complete and transparent reporting in journals, brochures, and the media need to be better enforced, and laws need to be changed in order to protect patients and doctors alike against the practice of defensive medicine instead of encouraging it. A critical mass of informed citizens will not resolve all healthcare problems, but it can constitute a major triggering factor for better care.

Do evidence summaries increase health policy-makers' use of evidence from systematic reviews? A systematic review

This review summarizes the evidence from six randomized controlled trials that judged the effectiveness of systematic review summaries on policymakers' decision making, or the most effective ways to present evidence summaries to increase policymakers' use of the evidence. This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one. The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary. None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?" Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.

Plain language summary

Policy briefs make systematic reviews easier to understand but little evidence of impact on use of study findings: It is likely that evidence summaries are easier to understand than complete systematic reviews. Whether these summaries increase the use of evidence from systematic reviews in policymaking is not clear.What is this review about?: Systematic reviews are long and technical documents that may be hard for policymakers to use when making decisions. Evidence summaries are short documents that describe research findings in systematic reviews. These summaries may simplify the use of systematic reviews.Other names for evidence reviews are policy briefs, evidence briefs, summaries of findings, or plain language summaries. The goal of this review was to learn whether evidence summaries help policymakers use evidence from systematic reviews. This review also aimed to identify the best ways to present the evidence summary to increase the use of evidence.What are the main findings of this review?: This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one.The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary.None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?"Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read.. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.What do the findings of this review mean?: Our review suggests that evidence summaries help policymakers to better understand the findings presented in systematic reviews. In short, evidence summaries should be developed to make it easier for policymakers to understand the evidence presented in systematic reviews. However, right now there is very little evidence on the best way to present systematic review evidence to policymakers.How up to date is this review?: The authors of this review searched for studies through June 2016.

Executive summary/Abstract

Background: Systematic reviews are important for decision makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which makes them hard to use for decision-making. Strategies to promote the use of evidence to decision makers are required, and evidence summaries have been suggested as a facilitator. Evidence summaries include policy briefs, briefing papers, briefing notes, evidence briefs, abstracts, summary of findings tables, and plain language summaries. There are many organizations developing and disseminating systematic review evidence summaries for different populations or subsets of decision makers. However, evidence on the usefulness and effectiveness of systematic review summaries is lacking. We present an overview of the available evidence on systematic review evidence summaries.Objectives: This systematic review aimed to 1) assess the effectiveness of evidence summaries on policy-makers' use of the evidence and 2) identify the most effective summary components for increasing policy-makers' use of the evidence.Search methods: We searched several online databases (Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, Global Health Library, Popline, Africa-wide, Public Affairs Information Services, Worldwide Political Science Abstracts, Web of Science, and DfiD), websites of research groups and organizations which produce evidence summaries, and reference lists of included summaries and related systematic reviews. These databases were searched in March-April, 2016.Selection criteria: Eligible studies included randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-after (CBA) studies, and interrupted time series (ITS) studies. We included studies of policymakers at all levels as well as health system managers. We included studies examining any type of "evidence summary", "policy brief", or other product derived from systematic reviews that presented evidence in a summarized form. These interventions could be compared to active comparators (e.g. other summary formats) or no intervention.The primary outcomes were: 1) use of systematic review summaries decision-making (e.g. self-reported use of the evidence in policy-making, decision-making) and 2) policymaker understanding, knowledge, and/or beliefs (e.g. changes in knowledge scores about the topic included in the summary). We also assessed perceived relevance, credibility, usefulness, understandability, and desirability (e.g. format) of the summaries.Results: Our database search combined with our grey literature search yielded 10,113 references after removal of duplicates. From these, 54 were reviewed in full text and we included 6 studies (reported in 7 papers, 1661 participants) as well as protocols from 2 ongoing studies. Two studies assessed the use of evidence summaries in decision-making and found little to no difference in effect. There was also little to no difference in effect for knowledge, understanding or beliefs (4 studies) and perceived usefulness or usability (3 studies). Summary of Findings tables and graded entry summaries were perceived as slightly easier to understand compared to complete systematic reviews. Two studies assessed formatting changes and found that for Summary of Findings tables, certain elements, such as reporting study event rates and absolute differences were preferred as well as avoiding the use of footnotes. No studies assessed adverse effects. The risks of bias in these studies were mainly assessed as unclear or low however, two studies were assessed as high risk of bias for incomplete outcome data due to very high rates of attrition.Authors' conclusions: Evidence summaries may be easier to understand than complete systematic reviews. However, their ability to increase the use of systematic review evidence in policymaking is unclear.

Baseline risk of recurrence in stage I-II endometrial carcinoma

Objective

Though there are no evidences that postoperative therapy improves overall survival (OS) in stage I–II endometrial carcinoma, many women receive postoperative radiation or chemotherapy. This study aimed to investigate the baseline risk of recurrence after complete resection without any adjuvant therapies and to suppose the validity of postoperative therapy for stage I–II endometrial carcinoma.

Methods

Charts for patients with stage I–II endometrial carcinoma who underwent operation without postoperative therapy between January 2005 and December 2011 were retrospectively reviewed and the baseline risk of recurrence and prognosis were assessed. Risk classifications were performed according to European Society for Medical Oncology (ESMO) clinical practice guidelines and Japanese guideline written by Japan Society of Gynecologic Oncology Group.

Results

Among 374 patients who underwent complete resection, 311 were evaluable. Five-year recurrence rates by ESMO and Japanese were 2.6% and 3.1% in low-risk, 9.2% and 6.6% in intermediate-risk and 13.5% and 13.8% in high-risk group (p=0.003 and 0.015, respectively). High-risk group had worse OS compared with low- and intermediate-risk groups (5-year OS, low: 97.9% and 97.6%, intermediate: 97.9% and 98.8%, and high: 89.5% and 87.5%; p=0.003 and 0.008, respectively). Independent predictive factors of recurrence were age over 60 years, type 2 (estrogen-independent) and peritoneal cytology.

Conclusion

ESMO and Japanese risk classification similarly stratify the baseline risk of recurrence. Patients with stage I–II endometrial carcinoma, especially low- and intermediate-risk diseases, have low recurrence rate and favorable OS, and the benefit of postoperative therapy might be small.

Brentuximab vedotin consolidation post-autologous stem cell transplant in Hodgkin lymphoma patients at risk of residual disease: number needed to treat

The number needed to treat (NNT) with brentuximab vedotin consolidation therapy post-autologous stem cell transplant (ASCT) versus placebo in the phase 3 AETHERA trial to avoid one additional event of disease progression/death was evaluated. AETHERA included 329 Hodgkin lymphoma patients at increased risk of progression post-ASCT who received brentuximab vedotin 1.8 mg/kg (n = 165) or placebo (n = 164) on day 1 of each 21-d cycle (up to 16 cycles). Over 60 months, the NNT with brentuximab vedotin ranged from 4.08 to 7.79 for the intent-to-treat population, 3.18-6.07 for patients with ≥2 risk factors, and 2.98-5.65 for patients with ≥3 risk factors. At various time points, and dependent on the risk group, 3-8 patients would need to be treated with brentuximab vedotin consolidation therapy to prevent a disease progression/death, compared with placebo. Patients with increased risk of relapse may benefit most from brentuximab vedotin.

A SOA-Based Platform to Support Clinical Data Sharing

The eSource Data Interchange Group, part of the Clinical Data Interchange Standards Consortium, proposed five scenarios to guide stakeholders in the development of solutions for the capture of eSource data. The fifth scenario was subdivided into four tiers to adapt the functionality of electronic health records to support clinical research. In order to develop a system belonging to the "Interoperable" Tier, the authors decided to adopt the service-oriented architecture paradigm to support technical interoperability, Health Level Seven Version 3 messages combined with LOINC (Logical Observation Identifiers Names and Codes) vocabulary to ensure semantic interoperability, and Healthcare Services Specification Project standards to provide process interoperability. The developed architecture enhances the integration between patient-care practice and medical research, allowing clinical data sharing between two hospital information systems and four clinical data management systems/clinical registries. The core is formed by a set of standardized cloud services connected through standardized interfaces, involving client applications. The system was approved by a medical staff, since it reduces the workload for the management of clinical trials. Although this architecture can realize the "Interoperable" Tier, the current solution actually covers the "Connected" Tier, due to local hospital policy restrictions.

Comparing and Contrasting A Priori and A Posteriori Generalizability Assessment of Clinical Trials on Type 2 Diabetes Mellitus

Clinical trials are indispensable tools for evidence-based medicine. However, they are often criticized for poor generalizability. Traditional trial generalizability assessment can only be done after the trial results are published, which compares the enrolled patients with a convenience sample of real-world patients. However, the proliferation of electronic data in clinical trial registries and clinical data warehouses offer a great opportunity to assess the generalizability during the design phase of a new trial. In this work, we compared and contrasted a priori (based on eligibility criteria) and a posteriori (based on enrolled patients) generalizability of Type 2 diabetes clinical trials. Further, we showed that comparing the study population selected by the clinical trial eligibility criteria to the real-world patient population is a good indicator of the generalizability of trials. Our findings demonstrate that the a priori generalizability of a trial is comparable to its a posteriori generalizability in identifying restrictive quantitative eligibility criteria.

Perception of risk and shared decision making process in multiple sclerosis

INTRODUCTION

Multiple sclerosis (MS) treatment has changed considerably in recent years thanks to the introduction of ever-more-powerful drugs. Unfortunately, the higher efficacies of these therapies are associated with increased risks of severe adverse events. In this scenario, neurologists and persons with MS (pwMSs) must now balance benefits and risks when making decisions regarding MS management. Areas covered: This review highlights the importance of the risk perception of pwMSs and their neurologists in the shared decision-making process in MS management, taking into account different ways improve the empowerment and engagement of pwMSs. Expert commentary: The shared decision-making process in MS is strongly influenced by an individual's risk perception, which is dynamic and influenced by the personal, emotional, social, and experiential factors of both the pwMSs and neurologists.

Helping Doctors and Patients Make Sense of Health Statistics

Many doctors, patients, journalists, and politicians alike do not understand what health statistics mean or draw wrong conclusions without noticing. Collective statistical illiteracy refers to the widespread inability to understand the meaning of numbers. For instance, many citizens are unaware that higher survival rates with cancer screening do not imply longer life, or that the statement that mammography screening reduces the risk of dying from breast cancer by 25% in fact means that 1 less woman out of 1,000 will die of the disease. We provide evidence that statistical illiteracy (a) is common to patients, journalists, and physicians; (b) is created by nontransparent framing of information that is sometimes an unintentional result of lack of understanding but can also be a result of intentional efforts to manipulate or persuade people; and (c) can have serious consequences for health.

The causes of statistical illiteracy should not be attributed to cognitive biases alone, but to the emotional nature of the doctor–patient relationship and conflicts of interest in the healthcare system. The classic doctor–patient relation is based on (the physician's) paternalism and (the patient's) trust in authority, which make statistical literacy seem unnecessary; so does the traditional combination of determinism (physicians who seek causes, not chances) and the illusion of certainty (patients who seek certainty when there is none). We show that information pamphlets, Web sites, leaflets distributed to doctors by the pharmaceutical industry, and even medical journals often report evidence in nontransparent forms that suggest big benefits of featured interventions and small harms. Without understanding the numbers involved, the public is susceptible to political and commercial manipulation of their anxieties and hopes, which undermines the goals of informed consent and shared decision making.

What can be done? We discuss the importance of teaching statistical thinking and transparent representations in primary and secondary education as well as in medical school. Yet this requires familiarizing children early on with the concept of probability and teaching statistical literacy as the art of solving real-world problems rather than applying formulas to toy problems about coins and dice. A major precondition for statistical literacy is transparent risk communication. We recommend using frequency statements instead of single-event probabilities, absolute risks instead of relative risks, mortality rates instead of survival rates, and natural frequencies instead of conditional probabilities. Psychological research on transparent visual and numerical forms of risk communication, as well as training of physicians in their use, is called for.

Statistical literacy is a necessary precondition for an educated citizenship in a technological democracy. Understanding risks and asking critical questions can also shape the emotional climate in a society so that hopes and anxieties are no longer as easily manipulated from outside and citizens can develop a better-informed and more relaxed attitude toward their health.

Buyer beware: health choices information broadcast to the public

Purpose

– This paper is based on the author’s 2015 Northwest Patient Safety Conference presentation, consistent with a conference theme of improving doctor-patient communication. The paper aims to discuss these issues.

Design/methodology/approach

– Ongoing systematic accumulation and critical review of research literature regarding design of effective public information websites, conducted from 2008-2015 while the author was supervising the prototyping, refinement and evaluation of healthcare-associated infections public information websites.

Findings

– In 2005, the US Centers for Medicare and Medicaid Services launched its Hospital Compare website, announced as an enormous step forward by providing objective information to inform consumer choices. Subsequently, many other websites and programs emerged to report quality-of-care ratings of hospitals and doctors, and provide other advice intended to help the public inform their choices. When objectively evaluated to a scientific publication-level standard, websites like Hospital Compare show relatively low usage and disappointing impact; individual providers rank so differently across ratings websites that it is difficult to see how trustworthy conclusions could be drawn; and much of the advice offered through popular media is not supported by believable evidence. Further, research shows healthcare professionals and members of the lay public view concepts of evidence and evidence-based decisions quite differently. Badly informed misguided decisions can have negative consequences for providers, patients and public trust.

Originality/value

– Populism and celebrity seem to have trumped science during recent growth of public information resources for health choices. This paper summarizes serious flaws underlying resulting information products, indicating necessary changes to better serve a legitimate need.

Application of Lepidic Component Predominance to Adjuvant Chemotherapy with Oral Fluoropyrimidines for Stage I Lung Adenocarcinoma

BACKGROUND

In the present study we aimed to investigate whether the predominance of the lepidic component in tumors was associated with the outcome of postoperative adjuvant chemotherapy for stage I lung adenocarcinoma.

PATIENTS AND METHODS

Charts for patients with pathological stage I lung adenocarcinoma were retrospectively reviewed and then outcomes of adjuvant chemotherapy were assessed according to the lepidic component predominance in tumors. Prognostic factors were evaluated using a Cox proportional hazard model. Propensity scores were determined using the optimal matching method on the basis of Cox modeling and matched (1:1) analysis was applied after classification into lepidic and nonlepidic predominant tumors.

RESULTS

Among 798 patients with stage I lung adenocarcinoma, 168 received adjuvant chemotherapy. Although adjuvant chemotherapy conferred no disease-free survival (DFS) advantage upon patients with lepidic predominant tumors, it improved DFS in T1b and T2a nonlepidic predominant tumors (P = .045 and P = .029, respectively). Propensity score matched analysis revealed no survival benefits of adjuvant oral fluoropyrimidines in lepidic predominant tumors (DFS, P = .461 and overall survival, P = .983) and the positive survival advantages in nonlepidic predominant tumors (DFS, P = .015 and overall survival, P = .027).

CONCLUSION

Adjuvant oral fluoropyrimidines conferred a better survival advantage upon patients with nonlepidic predominant tumors than patients with lepidic predominant tumors. The predominance of a lepidic component could serve as an indicator of adjuvant chemotherapy with oral fluoropyrimidines in stage I lung adenocarcinoma.

Integration of PKPD relationships into benefit-risk analysis

AIM

Despite the continuous endeavour to achieve high standards in medical care through effectiveness measures, a quantitative framework for the assessment of the benefit-risk balance of new medicines is lacking prior to regulatory approval. The aim of this short review is to summarise the approaches currently available for benefit-risk assessment. In addition, we propose the use of pharmacokinetic-pharmacodynamic (PKPD) modelling as the pharmacological basis for evidence synthesis and evaluation of novel therapeutic agents.

METHODS

A comprehensive literature search has been performed using MESH terms in PubMed, in which articles describing benefit-risk assessment and modelling and simulation were identified. In parallel, a critical review of multi-criteria decision analysis (MCDA) is presented as a tool for characterising a drug's safety and efficacy profile.

RESULTS

A definition of benefits and risks has been proposed by the European Medicines Agency (EMA), in which qualitative and quantitative elements are included. However, in spite of the value of MCDA as a quantitative method, decisions about benefit-risk balance continue to rely on subjective expert opinion. By contrast, a model-informed approach offers the opportunity for a more comprehensive evaluation of benefit-risk balance before extensive evidence is generated in clinical practice.

CONCLUSIONS

Benefit-risk balance should be an integral part of the risk management plan and as such considered before marketing authorisation. Modelling and simulation can be incorporated into MCDA to support the evidence synthesis as well evidence generation taking into account the underlying correlations between favourable and unfavourable effects. In addition, it represents a valuable tool for the optimization of protocol design in effectiveness trials.

When is rational to order a diagnostic test, or prescribe treatment: the threshold model as an explanation of practice variation

BACKGROUND

The threshold model represents an important advance in the field of medical decision-making. It is a linchpin between evidence (which exists on the continuum of credibility) and decision-making (which is a categorical exercise - we decide to act or not act). The threshold concept is closely related to the question of rational decision-making. When should the physician act, that is order a diagnostic test, or prescribe treatment? The threshold model embodies the decision theoretic rationality that says the most rational decision is to prescribe treatment when the expected treatment benefit outweighs its expected harms. However, the well-documented large variation in the way physicians order diagnostic tests or decide to administer treatments is consistent with a notion that physicians' individual action thresholds vary.

METHODS

We present a narrative review summarizing the existing literature on physicians' use of a threshold strategy for decision-making.

RESULTS

We found that the observed variation in decision action thresholds is partially due to the way people integrate benefits and harms. That is, explanation of variation in clinical practice can be reduced to a consideration of thresholds. Limited evidence suggests that non-expected utility threshold (non-EUT) models, such as regret-based and dual-processing models, may explain current medical practice better. However, inclusion of costs and recognition of risk attitudes towards uncertain treatment effects and comorbidities may improve the explanatory and predictive value of the EUT-based threshold models.

CONCLUSIONS

The decision when to act is closely related to the question of rational choice. We conclude that the medical community has not yet fully defined criteria for rational clinical decision-making. The traditional notion of rationality rooted in EUT may need to be supplemented by reflective rationality, which strives to integrate all aspects of medical practice - medical, humanistic and socio-economic - within a coherent reasoning system.

Consensus development for healthcare professionals

Consensus development sprang from a desire to synthesize clinician and expert opinions on clinical practice and research agendas in the 1950s. And since the American Institute of Medicine formally defined "guidelines" in 1990, there has been a proliferation of clinical practice guidelines (CPG) both formally and informally. This modern decision-making tool used by both physicians and patients, requires extensive planning to overcome the challenges of consensus development while reaping its rewards. Consensus allows for a group approach of multiple experts sharing ideas to form consensus on topics ranging from appropriateness of procedures to research agenda development. Disagreements can shed light on areas of controversy and launch further discussions. It has five main components: three inputs (defining the task, participant identification and recruitment, and information synthesis), the approach (consensus development by explicit or implicit means), and the output (dissemination of results). Each aspect requires extensive planning a priori as they influence the entire process, from how information will be interpreted, the interaction of participants, the resulting judgment, to whether there will be uptake of results. Implicit approaches utilize qualitative methods and/or a simple voting structure of majority wins, and are used in informal consensus development methods and consensus development conferences. Explicit approaches aggregate results or judgments using explicit rules set a priori with definitions of "agreement" or consensus. Because the implicit process can be more opaque, unforeseen challenges can emerge such as the undue influence of a minority. And yet, the logistics of explicit approaches may be more time consuming and not appropriate when speed is a priority. In determining which method to use, it is important to understand the pros and cons of different approaches and how it will affect the overall input, approach, and outcome.

Doctors and numbers: an assessment of the critical risk interpretation test

BACKGROUND

Risk interpretation affects decision making. Yet, there is no valid assessment of how clinicians interpret the risk data that they commonly encounter.

OBJECTIVE

To establish the reliability and validity of a 20-item test of clinicians' risk interpretation.

METHODS

The Critical Risk Interpretation Test (CRIT) measures clinicians' abilities to 1) modify the interpretation based on meaningful differences in the outcome (e.g., disease specific v. all-cause mortality) and time period (e.g., lifetime v. 10-year mortality), 2) maintain a stable interpretation for different risk framings (e.g., relative v. absolute risk), and 3) correctly interpret how diagnostic testing modifies risk. There were 658 clinicians and medical trainees who participated: 116 nurse practitioners (NPs) at a national conference, 273 medical students at 1 institution, 148 residents in internal medicine at 2 institutions, and 121 internists at 1 institution. Participants completed a self-administered paper test during educational conferences. Seventeen evidence-based medicine experts took the test online and formally assessed content validity. Eighteen second-year medical students were recruited to take the test and a retest 3 weeks later to explore test-retest correlation.

RESULTS

Expert review supported test clarity and content validity. Factor analysis supported that the CRIT identifies at least 3 separable areas of clinician knowledge. Test-retest correlation was fair (intraclass correlation coefficient = 0.65; standard error = 0.15). Scores on our test correlated with other tests of related abilities. Mean test scores varied among groups, with differences in prior evidence-based medicine training and experience (93 for NPs, 101 for medical students, 101 for residents, 103 for academic internists, and 110 for physician experts; P < 0.001).

CONCLUSIONS

Our results provide supporting evidence for the reliability and validity of the CRIT as an index of critical risk interpretation abilities, which is acceptable and feasible to administer in an educational setting.

Subgroup analysis in randomized controlled trials appeared to be dependent on whether relative or absolute effect measures were used

OBJECTIVES

To assess whether relative or absolute effect measures were used in subgroup analyses of randomized controlled trials (RCTs) and study whether conclusions would change if subgroup effects were calculated on a different scale than reported.

STUDY DESIGN AND SETTING

We studied all 327 RCTs published in 2010 in five major medical journals. For trials with a dichotomous primary outcome, we extracted reported main and subgroup effect measures. If crude subgrouping data were reported, we calculated the subgroup effects on both relative and absolute scales.

RESULTS

Of the 229 RCTs with a dichotomous primary outcome, 120 (52%) performed subgroup analyses. In 106 of these 120 (88%) RCTs, relative effect measures were used for subgroup analyses, whereas an absolute scale was used in 9 (8%) trials. Two (2%) RCTs reported both relative and absolute subgroup effects. Crude data of the subgroups could be extracted in 41 of the 120 (34%) RCTs. Calculating subgroup effects on a different scale than reported lead to a change in conclusion in 17% of the 41 trials.

CONCLUSION

Almost all RCTs used relative effect measures for subgroup analyses. Interpretation of subgroup effects, however, appeared to be dependent on whether relative or absolute effect measures were used.

Making evidence more wanted: a systematic review of facilitators to enhance the uptake of evidence from systematic reviews and meta-analyses

CONTEXT

The increased uptake of evidence from systematic reviews is advocated because of their potential to improve the quality of decision making for patient care. Systematic reviews can do this by decreasing inappropriate clinical variation and quickly expediting the application of current, effective advances to everyday practice. However, research suggests that evidence from systematic reviews has not been widely adopted by health professionals. Little is known about the facilitators to uptake of research evidence from systematic reviews and meta-analyses.

OBJECTIVE

To review the facilitators to the uptake by decision makers, of evidence from systematic, meta-analyses and the databases containing them.

SEARCH STRATEGY

We searched 19 databases covering the full range of publication years, utilised three search engines and also personally contacted investigators. Grey literature and knowledge translation research was particularly sought. Reference lists of primary studies and related reviews were also searched.

SELECTION CRITERIA

Studies were included if they reported on the views and perceptions of decision makers on the uptake of evidence from systematic reviews, meta-analyses and the databases associated with them. One investigator screened titles to identify candidate articles, and then two reviewers independently assessed the relevance of retrieved articles to exclude studies that did not meet the inclusion criteria. Quality of the included studies was also assessed.

DATA EXTRACTION

Using a pre-established taxonomy, two reviewers described the methods of included studies and extracted data that were summarised in tables and then analysed. Differences were resolved by consensus.

RESULTS

Of articles initially identified, we selected unique published studies describing at least one facilitator to the uptake of evidence from systematic reviews. The 15 unique studies reported 10 surveys, three qualitative investigations and two mixed studies that addressed potential facilitators. Five studies were from Canada, four from the UK, three from Australia, one from Iran and one from South-east Asia (Indonesia, Malaysia, Thailand and the Philippines), with one study covering both Canada and UK. In total, the 15 studies covered eight countries from four continents. Of 2495 participants in the 15 studies, at least 1343 (53.8%) were physicians. Perceived facilitators to the use of evidence from systematic reviews varied. The 15 studies yielded 54 potential facilitators to systematic review uptake. The five most commonly reported perceived facilitators to uptake of evidence from systematic reviews were the following: the perception that systematic reviews have multiple uses for improving knowledge, research, clinical protocols and evidence-based medicine skills (6/15); a content that included benefits, harms and costs and is current, transparent and timely (6/15); a format with a 1:3:25 staged access and executive summary (5/15); training in use (4/15); and peer-group support (4/15).

CONCLUSION

The results expand our understanding of how multiple factors act as facilitators to optimal clinical practice. This systematic review reveals that interventions to foster uptake of evidence from systematic reviews, meta-analyses and The Cochrane Library can build on a broad range of facilitators.

Communicating clinical trial outcomes: Effects of presentation method on physicians’ evaluations of new treatments

Physicians expect a treatment to be more effective when its clinical outcomes are described as relative rather than as absolute risk reductions. We examined whether effects of presentation method (relative vs. absolute risk reduction) remain when physicians are provided the baseline risk information, a vital piece of statistical information omitted in previous studies. Using a between-subjects design, ninety five physicians were presented the risk reduction associated with a fictitious treatment for hypertension either as an absolute risk reduction or as a relative risk reduction, with or without including baseline risk information. Physicians reported that the treatment would be more effective and that they would be more willing to prescribe it when its risk reduction was presented to them in relative rather than in absolute terms. The relative risk reduction was perceived as more effective than absolute risk reduction even when the baseline risk information was explicitly reported. We recommend that information about absolute risk reduction be made available to physicians in the reporting of clinical outcomes. Moreover, health professionals should be cognizant of the potential biasing effects of risk information presented in relative risk terms.

A proposal for an additional clinical trial outcome measure assessing preventive effect as delay of events

Many effect measures used in clinical trials are problematic because they are differentially understood by patients and physicians. The emergence of novel methods such as accelerated failure-time models and quantile regression has shifted the focus of effect measurement from probability measures to time-to-event measures. Such modeling techniques are rapidly evolving, but matching non-parametric descriptive measures are lacking. We propose such a measure, the delay of events, demonstrating treatment effect as a gain in event-free time. We believe this measure to be of value for shared clinical decision-making. The rationale behind the measure is given, and it is conceptually explained using the Kaplan–Meier estimate and the quantile regression framework. A formula for calculation of the delay of events is given. Hypothetical and empirical examples are used to demonstrate the measure. The measure is discussed in relation to other measures highlighting the time effects of preventive treatments. There is a need to further investigate the properties of the measure as well as its role in clinical decision-making.

Communicating treatment effectiveness in the context of chronic disease processes

A better understanding of the risk concept in relation to treatment of chronic disease processes is warranted. Consequences of therapies may be described in terms of risk reductions or postponement of adverse outcomes. This review argues that for most interventions, neither of these measures adequately characterize the distribution of treatment benefit across patients. More focus should be placed on choosing the context-relevant outcome measure and on developing techniques for extracting and communicating 2D outcome measures where the distribution of outcomes across patients is explicitly reflected. Since the distribution of outcomes affects patients' valuation of interventions, more research into risk communication is likely to lead to improved decision making.

Meta-analyses of placebo-controlled trials of acamprosate for the treatment of alcohol dependence: impact of the combined pharmacotherapies and behavior interventions study

OBJECTIVES

: The Combined Pharmacotherapies and Behavior Interventions Study (COMBINE) reported no significant difference between acamprosate and placebo in the treatment of alcohol dependence. To evaluate the impact of COMBINE, we performed a meta-analysis of acamprosate placebo-controlled trials with the inclusion of data from COMBINE. As a secondary analysis, we added the COMBINE data to a recently published meta-analysis of naltrexone placebo-controlled trials.

METHODS

: A structured literature search of major databases was performed from January 1990 to August 2007 for acamprosate placebo-controlled randomized trials. Mean differences in cumulative abstinent days (CAD) and abstinence rates (AR) from eligible studies were statistically combined to calculate point estimates and 95% CI for differences in CAD and AR.

RESULTS

: Ten and 16 studies evaluating CAD and AR, respectively were suitable for statistical pooling. The findings revealed that acamprosate was superior to placebo in the mean number of CAD (P < 0.001) and AR (pooled AR = 1.58; P < 0.001). The pooled AR for naltrexone was also significant indicating a relative benefit over placebo (AR = 1.27; P < 0.001). The COMBINE trial results contributed a weight of less than 15% to the final pooled statistical outcomes for both agents.

CONCLUSIONS

: The current study confirmed that acamprosate and naltrexone are both effective agents for the treatment of patients with alcohol dependence. Systematic reviews with meta-analyses of randomized controlled trials and randomized controlled trials with adequate sample sizes are in the same (highest) level of evidence. Therefore, clinicians should use both these sources of information as their foundation for selecting optimal therapy for patients with alcohol dependence.

The Right Tool Is What They Need, Not What We Have

While patients often receive risk information, exactly what constitutes being “informed” about health risks is often unclear. Patients have specific needs, such as avoiding being surprised by a possible outcome and making complex risk trade-off decisions. Yet all risk information is not equally informative for those needs. In this article, I present a taxonomy of seven risk concepts that vary in their inherent precision and evaluability. Congruent with the “less is more” concept, I argue that risk communications should use formats that are tailored to message recipients’ specific informational needs. Simpler formats can be used when patients only need to order risks, while more complex numerical probability statements will be necessary when patients need to assess differences in risk magnitude and put those differences into meaningful context. Selecting need-congruent formats when designing communications about risks to patients is a novel approach that may better support patients’ health care decision making.

Cardiovascular risk

Cardiovascular disease is a major, growing, worldwide problem. It is important that individuals at risk of developing cardiovascular disease can be effectively identified and appropriately stratified according to risk. This review examines what we understand by the term risk, traditional and novel risk factors, clinical scoring systems, and the use of risk for informing prescribing decisions. Many different cardiovascular risk factors have been identified. Established, traditional factors such as ageing are powerful predictors of adverse outcome, and in the case of hypertension and dyslipidaemia are the major targets for therapeutic intervention. Numerous novel biomarkers have also been described, such as inflammatory and genetic markers. These have yet to be shown to be of value in improving risk prediction, but may represent potential therapeutic targets and facilitate more targeted use of existing therapies. Risk factors have been incorporated into several cardiovascular disease prediction algorithms, such as the Framingham equation, SCORE and QRISK. These have relatively poor predictive power, and uncertainties remain with regards to aspects such as choice of equation, different risk thresholds and the roles of relative risk, lifetime risk and reversible factors in identifying and treating at-risk individuals. Nonetheless, such scores provide objective and transparent means of quantifying risk and their integration into therapeutic guidelines enables equitable and cost-effective distribution of health service resources and improves the consistency and quality of clinical decision making.

Animated graphics for comparing two risks: a cautionary tale

Background

The increasing use of computer-administered risk communications affords the potential to replace static risk graphics with animations that use motion cues to reinforce key risk messages. Research on the use of animated graphics, however, has yielded mixed findings, and little research exists to identify the specific animations that might improve risk knowledge and patients’ decision making.

Objective

To test whether viewing animated forms of standard pictograph (icon array) risk graphics displaying risks of side effects would improve people’s ability to select the treatment with the lowest risk profile, as compared with viewing static images of the same risks.

Methods

A total of 4198 members of a demographically diverse Internet panel read a scenario about two hypothetical treatments for thyroid cancer. Each treatment was described as equally effective but varied in side effects (with one option slightly better than the other). Participants were randomly assigned to receive all risk information in 1 of 10 pictograph formats in a quasi-factorial design. We compared a control condition of static grouped icons with a static scattered icon display and with 8 Flash-based animated versions that incorporated different combinations of (1) building the risk 1 icon at a time, (2) having scattered risk icons settle into a group, or (3) having scattered risk icons shuffle themselves (either automatically or by user control). We assessed participants’ ability to choose the better treatment (choice accuracy), their gist knowledge of side effects (knowledge accuracy), and their graph evaluation ratings, controlling for subjective numeracy and need for cognition.

Results

When compared against static grouped-icon arrays, no animations significantly improved any outcomes, and most showed significant performance degradations. However, participants who received animations of grouped icons in which at-risk icons appeared 1 at a time performed as well on all outcomes as the static grouped-icon control group. Displays with scattered icons (static or animated) performed particularly poorly unless they included the settle animation that allowed users to view event icons grouped.

Conclusions

Many combinations of animation, especially those with scattered icons that shuffle randomly, appear to inhibit knowledge accuracy in this context. Static pictographs that group risk icons, however, perform very well on measures of knowledge and choice accuracy. These findings parallel recent evidence in other data communication contexts that less can be more—that is, that simpler, more focused information presentation can result in improved understanding. Decision aid designers and health educators should proceed with caution when considering the use of animated risk graphics to compare two risks, given that evidence-based, static risk graphics appear optimal.

How do prescribing doctors anticipate the effect of statins?

OBJECTIVES

Statins are a lipid-lowering treatment, prescribed frequently to prevent cardiovascular events. The objective of this study was to explore how doctors anticipate the effect of statins and what factors are associated with their willingness to initiate treatment.

METHODS

A total of 330 Swedish cardiologists, internists and general practitioners were asked to consider two hypothetical patient cases, one with and one without previous coronary heart disease. Based on these cases, the respondents answered questions about their willingness to initiate treatment and what effects they might expect. The expectation of effect was assessed in two ways: (1) the absolute risk reduction of myocardial infarction in 1000 patients treated with statins for 5 years; and (2) statins' average effect on increased life expectancy. The doctors' beliefs about absolute risk reduction were compared with results from clinical trials.

RESULTS

Most doctors had a suboptimal expectation about absolute risk reduction; only about one-third had expectations in the range supported by evidence-based data. There were different views about statins' ability to prolong life: that is, average gain in life expectancy due to treatment was believed to be 2 years in the primary patient case, and 3 years in the second patient case. The doctors' beliefs about statins' ability to prolong life were associated significantly with their willingness to initiate treatment.

CONCLUSION

The overall results imply that doctors have varying and suboptimal understanding of the effect of statins. This may inhibit the goal of integrating clinical research into clinical practice.

Relationship of persistent manic symptoms to the diagnosis of pediatric bipolar spectrum disorders

OBJECTIVE

The diagnosis of bipolar spectrum disorders (BPSDs [bipolar I and II disorders, cyclothymic disorder, and bipolar disorder not otherwise specified]) in youth remains controversial. The present study evaluated the possibility that the presence of persistent manic symptoms over a relatively short interval may increase the probability of a BPSD DSM diagnosis.

METHOD

Data were obtained from the screening and baseline assessments collected from 2005 through 2008 of an ongoing prospective, longitudinal study (Longitudinal Assessment of Manic Symptoms) examining the diagnosis and phenomenology of youth (N = 692) presenting to outpatient centers at ages 6-12 years. Youth were assessed for elevated symptoms of mania (ESM) with the Parent General Behavior Inventory-10-Item Mania Scale (PGBI-10M), the primary outcome measure. Screening and baseline scores separated individuals into those with ESM (ESM+; PGBI-10M score ≥ 12) and a control group of youth without ESM (ESM-; PGBI-10M score < 12). Youth were classified into 4 groups: persistent ESM+, remitted ESM+, persistent ESM-, and progressed to ESM+.

RESULTS

Individuals with persistent ESM+ were more likely to have a BPSD (relative risk = 3.04; 95% CI, 2.15-4.30). Using 2 administrations of the PGBI-10M spaced over a relatively brief interval (median = 4.0, mean = 6.1, SD = 5.9 weeks) improved the prediction of BPSD over using only the first administration (ΔR(2) = 0.10, Δχ(2)(1) = 50.06, P < .001). Likelihood ratios indicated that persistent ESM- substantially decreased the probability of BPSD. While high levels of persistent ESM+ increased the probability of a BPSD diagnosis, the final positive predictive value was only sufficient to signify the need for more thorough clinical evaluation.

CONCLUSIONS

In many cases, obtaining repeated parent report of mania symptoms substantially altered the probability of a BPSD diagnosis and may be a useful adjunct to a careful clinical evaluation. Future waves of data collection from this longitudinal study will be crucial for devising clinically useful methods for identifying or ruling out pediatric BPSD.

Using alternative statistical formats for presenting risks and risk reductions

BACKGROUND

The success of evidence-based practice depends on the clear and effective communication of statistical information.

OBJECTIVES

To evaluate the effects of using alternative statistical presentations of the same risks and risk reductions on understanding, perception, persuasiveness and behaviour of health professionals, policy makers, and consumers.

SEARCH STRATEGY

We searched Ovid MEDLINE (1966 to October 2007), EMBASE (1980 to October 2007), PsycLIT (1887 to October 2007), and the Cochrane Central Register of Controlled Trials (The Cochrane Library, 2007, Issue 3). We reviewed the reference lists of relevant articles, and contacted experts in the field.

SELECTION CRITERIA

We included randomized and non-randomized controlled parallel and cross-over studies. We focused on four comparisons: a comparison of statistical presentations of a risk (eg frequencies versus probabilities) and three comparisons of statistical presentation of risk reduction: relative risk reduction (RRR) versus absolute risk reduction (ARR), RRR versus number needed to treat (NNT), and ARR versus NNT.

DATA COLLECTION AND ANALYSIS

Two authors independently selected studies for inclusion, extracted data, and assessed risk of bias. We contacted investigators to obtain missing information. We graded the quality of evidence for each outcome using the GRADE approach. We standardized the outcome effects using adjusted standardized mean difference (SMD).

MAIN RESULTS

We included 35 studies reporting 83 comparisons. None of the studies involved policy makers. Participants (health professionals and consumers) understood natural frequencies better than probabilities (SMD 0.69 (95% confidence interval (CI) 0.45 to 0.93)). Compared with ARR, RRR had little or no difference in understanding (SMD 0.02 (95% CI -0.39 to 0.43)) but was perceived to be larger (SMD 0.41 (95% CI 0.03 to 0.79)) and more persuasive (SMD 0.66 (95% CI 0.51 to 0.81)). Compared with NNT, RRR was better understood (SMD 0.73 (95% CI 0.43 to 1.04)), was perceived to be larger (SMD 1.15 (95% CI 0.80 to 1.50)) and was more persuasive (SMD 0.65 (95% CI 0.51 to 0.80)). Compared with NNT, ARR was better understood (SMD 0.42 (95% CI 0.12 to 0.71)), was perceived to be larger (SMD 0.79 (95% CI 0.43 to 1.15)).There was little or no difference for persuasiveness (SMD 0.05 (95% CI -0.04 to 0.15)). The sensitivity analyses including only high quality comparisons showed consistent results for persuasiveness for all three comparisons. Overall there were no differences between health professionals and consumers. The overall quality of evidence was rated down to moderate because of the use of surrogate outcomes and/or heterogeneity. None of the comparisons assessed behaviourbehaviour.

AUTHORS' CONCLUSIONS

Natural frequencies are probably better understood than probabilities. Relative risk reduction (RRR), compared with absolute risk reduction (ARR) and number needed to treat (NNT), may be perceived to be larger and is more likely to be persuasive. However, it is uncertain whether presenting RRR is likely to help people make decisions most consistent with their own values and, in fact, it could lead to misinterpretation. More research is needed to further explore this question.

Deceiving Numbers

Background. Increased 5-y survival for screened patients is often inferred to mean that fewer patients die of cancer. However, due to several biases, the 5-y survival rate is a misleading metric for evaluating a screening’s effectiveness. If physicians are not aware of these issues, informed screening counseling cannot take place. Methods. Two questionnaire versions (“group” and “time”) presented 4 conditions: 5-y survival (5Y), 5-y survival and annual disease-specific mortality (5YM), annual disease-specific mortality (M), and 5-y survival, annual disease-specific mortality, and incidence (5YMI). Questionnaire version “time” presented data as a comparison between 2 time points and version “group” as a comparison between a screened and an unscreened group. All data were based on statistics for the same cancer site (prostate). Outcome variables were the recommendation of screening, reasoning behind recommendation, judgment of the screening’s effectiveness, and, if judged effective, a numerical estimate of how many fewer people out of 1000 would die if screened regularly. After randomized allocation, 65 German physicians in internal medicine and its subspecialities completed either of the 2 questionnaire versions. Results. Across both versions, 66% of the physicians recommended screening when presented with 5Y, but only 8% of the same physicians made the recommendation when presented with M (5YM: 31%; 5YMI: 55%). Also, 5Y made considerably more physicians (78%) judge the screening to be effective than any other condition (5YM: 31%; M: 5%; 5YMI: 49%) and led to the highest overestimations of benefit. Conclusion. A large number of physicians erroneously based their screening recommendation and judgment of screening’s effectiveness on the 5-y survival rate. Results show that reporting disease-specificmortality rates can offer a simple solution to physicians’ confusion about the real effect of screening.

What is the point: will screening mammography save my life?

BACKGROUND

We analyzed the claim "mammography saves lives" by calculating the life-saving absolute benefit of screening mammography in reducing breast cancer mortality in women ages 40 to 65.

METHODS

To calculate the absolute benefit, we first estimated the screen-free absolute death risk from breast cancer by adjusting the Surveillance, Epidemiology and End Results Program 15-year cumulative breast cancer mortality to account for the separate effects of screening mammography and improved therapy. We calculated the absolute risk reduction (reduction in absolute death risk), the number needed to screen assuming repeated screening, and the survival percentages without and with screening. We varied the relative risk reduction from 10%-30% based on the randomized trials of screening mammography. We developed additional variations of the absolute risk reduction for a screening intervention, including the average benefit of a single screen, as well as the life-saving proportion among patients with earlier cancer detection.

RESULTS

Because the screen-free absolute death risk is approximately 1% overall but rises with age, the relative risk reduction from repeated screening mammography is about 100 times the absolute risk reduction between the starting ages of 50 and 60. Assuming a base case 20% relative risk reduction, repeated screening starting at age 50 saves about 1.8 (overall range, 0.9-2.7) lives over 15 years for every 1000 women screened. The number needed to screen repeatedly is 1000/1.8, or 570. The survival percentage is 99.12% without and 99.29% with screening. The average benefit of a single screening mammogram is 0.034%, or 2970 women must be screened once to save one life. Mammography saves 4.3% of screen-detectable cancer patients' lives starting at age 50. This means 23 cancers must be found starting at age 50, or 27 cancers at age 40 and 21 cancers at age 65, to save one life.

CONCLUSION

The life-saving absolute benefit of screening mammography increases with age as the absolute death risk increases. The number of events needed to save one life varies depending on the prospective screening subset or reference class. Less than 5% of women with screen-detectable cancers have their lives saved.

Cancer risk communication in mainstream and ethnic newspapers

INTRODUCTION

We wanted to understand how cancer risks are communicated in mainstream and ethnic newspapers, to determine whether the 2 kinds of newspapers differ and to examine features of news stories and sources that might predict optimal risk communication.

METHODS

Optimal risk communication was defined as presenting the combination of absolute risk, relative risk, and prevention response efficacy information. We collected data by conducting a content analysis of cancer news coverage from 2003 (5,327 stories in major newspapers, 565 stories in ethnic newspapers). Comparisons of mainstream and ethnic newspapers were conducted by using cross-tabulations and Pearson chi2 tests for significance. Logistic regression equations were computed to calculate odds ratios and 95% confidence intervals for optimal risk communication.

RESULTS

In both kinds of newspapers, cancer risks were rarely communicated numerically. When numeric presentations of cancer risks were used, only 26.2% of mainstream and 29.5% of ethnic newspaper stories provided estimates of both absolute and relative risk. For both kinds of papers, only 19% of news stories presented risk communication optimally. Cancer risks were more likely to be communicated optimally if they focused on prostate cancer, were reports of new research, or discussed medical or demographic risks.

CONCLUSION

Research is needed to understand how these nonnumeric and decontextualized presentations of risk might contribute to inaccurate risk perceptions among news consumers.

Diabetes experts' reasoning about diabetes prevention studies: a questionnaire survey

BACKGROUND

Presentation of results of diabetes prevention studies as relative risk reductions and the use of diagnostic categories instead of metabolic parameters leads to overestimation of effects on diabetes risk. This survey examines to what extent overestimation of diabetes prevention is related to overestimation of prevention of late complications.

METHODS

Participants of two postgraduate courses in clinical diabetology in Austria (n = 69) and Germany (n = 31) were presented a questionnaire with 8 items at the beginning of the meetings. All 100 questionnaires were returned with 92 filled in completely. Participants were asked 1) to rate the importance of differently framed results of prevention studies and, for comparison, of the United Kingdom Prospective Diabetes Study (UKPDS), 2) to estimate to what extent late complications could be prevented by the achieved reductions in diabetes risk or HbA1c values, respectively.

RESULTS

Prevention of diabetes by 60% was considered important by 84% of participants and 35% thought that complications could be prevented by >/= 55%. However, if corresponding HbA1c values were presented (6.0% versus 6.1%) only 19% rated this effect important, and 12% thought that late complications could be prevented by >/= 55%. The difference in HbA1c of 0.9% over 10 years in the UKPDS was considered important by 75% of participants and 16% thought that complications ('any diabetes related endpoint') were reduced by >/= 55% (correct answer <15% by 20% participants).

CONCLUSION

The novel key message of this study is that the misleading reporting of diabetes prevention studies results in overestimation of effects on late complications.

Laypersons' understanding of relative risk reductions: randomised cross-sectional study

Background

Despite increasing recognition of the importance of involving patients in decisions on preventive healthcare interventions, little is known about how well patients understand and utilise information provided on the relative benefits from these interventions. The aim of this study was to explore whether lay people can discriminate between preventive interventions when effectiveness is presented in terms of relative risk reduction (RRR), and whether such discrimination is influenced by presentation of baseline risk.

Methods

The study was a randomised cross-sectional interview survey of a representative sample (n = 1,519) of lay people with mean age 59 (range 40–98) years in Denmark. In addition to demographic information, respondents were asked to consider a hypothetical drug treatment to prevent heart attack. Its effectiveness was randomly presented as RRR of 10, 20, 30, 40, 50 or 60 percent, and half of the respondents were presented with quantitative information on the baseline risk of heart attack. The respondents had also been asked whether they were diagnosed with hypercholesterolemia or had experienced a heart attack.

Results

In total, 873 (58%) of the respondents consented to the hypothetical treatment. While 49% accepted the treatment when RRR = 10%, the acceptance rate was 58–60% for RRR>10. There was no significant difference in acceptance rates across respondents irrespective of whether they had been presented with quantitative information on baseline risk or not.

Conclusion

In this study, lay people's decisions about therapy were only slightly influenced by the magnitude of the effect when it was presented in terms of RRR. The results may indicate that lay people have difficulties in discriminating between levels of effectiveness when they are presented in terms of RRR.

The number needed to treat for adalimumab, etanercept, and infliximab based on ACR50 response in three randomized controlled trials on established rheumatoid arthritis: a systematic literature review

OBJECTIVE

To compare the efficacy of adalimumab, etanercept, and infliximab in patients with established rheumatoid arthritis (RA) taking concomitant methotrexate (MTX) by calculating the number needed to treat (NNT) using three different methods.

METHODS

A systematic literature search of the Cochrane Library, MEDLINE, and EMBASE was conducted from inception to 30 June 2006. Two pairs of investigators, a Danish and a Swedish pair, independently conducted a structured literature review. The reviewers selected any published randomized, double-blind, MTX controlled study of adalimumab, etanercept, and infliximab, presenting the American College of Rheumatology 50% response (ACR50) after 12 months in RA patients with a mean disease duration of at least 5 years. The two review groups independently extracted the estimates necessary to calculate the NNT.

RESULTS

The reviewers consistently selected the same three randomized, controlled trials (RCTs), one for each of the drugs, and extracted equal data for the number of patients completing the 12-month intervention, and the corresponding number of ACR50 responding patients after therapy. Some baseline differences were noted: patients in the etanercept trial had a shorter disease duration and did not receive MTX prior to inclusion; patients in the adalimumab study had lower Health Assessment Questionnaire (HAQ) scores. The calculated NNTs varied slightly depending on the method used. The fully adjusted NNTs (95% confidence intervals) for adalimumab, etanercept, infliximab standard dosage and infliximab double dosage were 4 (3-6), 4 (3-6), 8 (4-66), and 4 (3-11) patients, respectively.

CONCLUSION

This study indicates equal efficacy of the three anti-tumour necrosis factor (TNF) therapies.

Alternate methods of framing information about medication side effects: incremental risk versus total risk of occurrence

Communications of treatment risk, such as medication package inserts, commonly report total rates of adverse reactions (e.g., 4% get heartburn with placebo, 9% with medication). This approach, however, requires mental arithmetic to distinguish the incremental risk caused by medication (here, 5%) from the total post-treatment risk. In two Internet-administered survey experiments (N = 2,012 and 1,393), we tested whether explicitly reporting the incremental risk and framing it as the "additional risk" of complications influenced people's impressions of adverse event risks. Study 1 compared side-by-side displays of total risks against sequential presentations that highlighted the incremental risk, using both text and graphical formats. Results showed that incremental risk formats significantly lowered participants' worry about complications and reduced biases caused by varying the risk denominator. Study 2 unpacked this factor and showed that its effect on both perceived likelihood and worry derives primarily from the incremental risk framing rather than from sequential presentation. Explicitly reporting incremental risk statistics appears to facilitate recognition of how much risk already exists at baseline. Presenting adverse reaction risks in this manner may improve patient comprehension of the effects of treatment decisions and support effective risk communication.

What do we know about communicating risk? A brief review and suggestion for contextualising serious, but rare, risk, and the example of cox-2 selective and non-selective NSAIDs

BACKGROUND

Communicating risk is difficult. Although different methods have been proposed - using numbers, words, pictures or combinations - none has been extensively tested. We used electronic and bibliographic searches to review evidence concerning risk perception and presentation. People tend to underestimate common risk and overestimate rare risk; they respond to risks primarily on the basis of emotion rather than facts, seem to be risk averse when faced with medical interventions, and want information on even the rarest of adverse events.

METHODS

We identified observational studies (primarily in the form of meta-analyses) with information on individual non-steroidal anti-inflammatory drug (NSAID) or selective cyclooxygenase-2 inhibitor (coxib) use and relative risk of gastrointestinal bleed or cardiovascular event, the background rate of events in the absence of NSAID or coxib, and the likelihood of death from an event. Using this information we present the outcome of additional risk of death from gastrointestinal bleed and cardiovascular event for individual NSAIDs and coxibs alongside information about death from other causes in a series of perspective scales.

RESULTS

The literature on communicating risk to patients is limited. There are problems with literacy, numeracy and the human tendency to overestimate rare risk and underestimate common risk. There is inconsistency in how people translate between numbers and words. We present a method of communicating information about serious risks using the common outcome of death, using pictures, numbers and words, and contextualising the information. The use of this method for gastrointestinal and cardiovascular harm with NSAIDs and coxibs shows differences between individual NSAIDs and coxibs.

CONCLUSION

Although contextualised risk information can be provided on two possible adverse events, many other possible adverse events with potential serious consequences were omitted. Patients and professionals want much information about risks of medical interventions but we do not know how best to meet expectations. The impact of contextualised information remains to be tested.

[Critical appraisal of articles about clinical audit]

OBJECTIVE

To summarize the most important criteria used in critical appraisal of publications about clinical audit by the review of the relevant English language literature.

METHODS

Electronic databases, including Medline, Science Direct, and Ingenta Select, and Internet search were used to find relevant English language publication between 1985 and 2005. Hand search and the reference lists search for publications were also applied. In addition, researchers were also contacted for publications.

RESULTS

The literature is surprisingly scarce about the critical appraisal of clinical audit publications. As a result hand search was as important as electronic search. The following three screening criteria can be used in order to critically appraise the scientific literature: (a) whether or not valid and relevant criteria and standards were used to evaluate clinical practice, (b) the criteria were used systematically on a representative sample of patients, and (c) the results are important and applicable (generalizable) in our practice. The publication is useful if it helps to improve the quality of our own clinical practice.

CONCLUSION

Critical appraisal criteria can be used to improve the quality of clinical audit, and disseminate the results of audit, as well as find high quality evidence for designing and implementing quality improvement initiatives.