Emerging Methods in Comparative Effectiveness and Safety

Comparing effectiveness and outcomes in asthma and cystic fibrosis

As technology yields new treatments, pediatric pulmonologists need determine how best to use them and how to decide which ones are best for any specific group or individual patient. Physicians have always customized therapies based upon patient response, but the new concept of "Personalized (or precision) medicine" focuses attention to a greater degree on the individual needs of patients based on their genetic, biomarker, phenotypic, or psychosocial characteristics. The newly developed biologics for treatment of asthma and CFTR modulators for treatment of cystic fibrosis (CF) highlight this newer approach. As we have more treatments available, new approaches to testing efficacy and effectiveness of these new therapies is necessary in order to efficiently bring them to market and compare their benefits in real world practice. While comparative effectiveness can be tested in pragmatic clinic trials, the most common approaches make use of observational data such as administrative databases and patient registries but their use for this is fraught with pitfalls that may or may not be methodologically surmountable. Once new therapies have been shown to be efficacious and effective, it is important to be cognizant of methods for ensuring that all patients actually receive the treatments that will be best for them. Comparisons of the effectiveness of clinical practice in the form of benchmarking is helpful for this, and consideration of costs and cost-effectiveness is essential to judging the best treatment for patients in a real-world setting.

Creating a Common Data Model for Comparative Effectiveness with the Observational Medical Outcomes Partnership

BACKGROUND

Adoption of a common data model across health systems is a key infrastructure requirement to allow large scale distributed comparative effectiveness analyses. There are a growing number of common data models (CDM), such as Mini-Sentinel, and the Observational Medical Outcomes Partnership (OMOP) CDMs.

OBJECTIVES

In this case study, we describe the challenges and opportunities of a study specific use of the OMOP CDM by two health systems and describe three comparative effectiveness use cases developed from the CDM.

METHODS

The project transformed two health system databases (using crosswalks provided) into the OMOP CDM. Cohorts were developed from the transformed CDMs for three comparative effectiveness use case examples. Administrative/billing, demographic, order history, medication, and laboratory were included in the CDM transformation and cohort development rules.

RESULTS

Record counts per person month are presented for the eligible cohorts, highlighting differences between the civilian and federal datasets, e.g. the federal data set had more outpatient visits per person month (6.44 vs. 2.05 per person month). The count of medications per person month reflected the fact that one system's medications were extracted from orders while the other system had pharmacy fills and medication administration records. The federal system also had a higher prevalence of the conditions in all three use cases. Both systems required manual coding of some types of data to convert to the CDM.

CONCLUSIONS

The data transformation to the CDM was time consuming and resources required were substantial, beyond requirements for collecting native source data. The need to manually code subsets of data limited the conversion. However, once the native data was converted to the CDM, both systems were then able to use the same queries to identify cohorts. Thus, the CDM minimized the effort to develop cohorts and analyze the results across the sites.

Lessons from comparative effectiveness research methods development projects funded under the Recovery Act

BACKGROUND

The American Recovery and Reinvestment Act of 2009 (ARRA) directed nearly US$29.2 million to comparative effectiveness research (CER) methods development.

AIM

To help inform future CER methods investments, we describe the ARRA CER methods projects, identify barriers to this research and discuss the alignment of topics with published methods development priorities.

METHODS

We used several existing resources and held discussions with ARRA CER methods investigators.

RESULTS & CONCLUSION

Although funded projects explored many identified priority topics, investigators noted that much work remains. For example, given the considerable investments in CER data infrastructure, the methods development field can benefit from additional efforts to educate researchers about the availability of new data sources and about how best to apply methods to match their research questions and data.

Improved prognosis of septic shock in patients with cirrhosis: a multicenter study*

OBJECTIVE

To determine the evolution of the outcome of patients with cirrhosis and septic shock.

DESIGN

A 13-year (1998-2010) multicenter retrospective cohort study of prospectively collected data.

SETTING

The Collège des Utilisateurs des Bases des données en Réanimation (CUB-Réa) database recording data related to admissions in 32 ICUs in Paris area.

PATIENTS

Thirty-one thousand two hundred fifty-one patients with septic shock were analyzed; 2,383 (7.6%) had cirrhosis.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Compared with noncirrhotic patients, patients with cirrhosis had higher Simplified Acute Physiology Score II (63.1 ± 22.7 vs 58.5 ± 22.8, p < 0.0001) and higher prevalence of renal (71.5% vs 54.8%, p < 0.0001) and neurological (26.1% vs 19.5%, p < 0.0001) dysfunctions. Over the study period, in-ICU and in-hospital mortality was higher in patients with cirrhosis (70.1% and 74.5%) compared with noncirrhotic patients (48.3% and 51.7%, p < 0.0001 for both comparisons). Cirrhosis was independently associated with an increased risk of death in ICU (adjusted odds ratio = 2.524 [2.279-2.795]). In patients with cirrhosis, factors independently associated with in-ICU mortality were as follows: admission for a medical reason, Simplified Acute Physiology Score II, mechanical ventilation, renal replacement therapy, spontaneous bacterial peritonitis, positive blood culture, and infection by fungus, whereas direct admission and admission during the most recent midterm period (2004-2010) were associated with a decreased risk of death. From 1998 to 2010, prevalence of septic shock in patients with cirrhosis increased from 8.64 to 15.67 per 1,000 admissions to ICU (p < 0.0001) and their in-ICU mortality decreased from 73.8% to 65.5% (p = 0.01) despite increasing Simplified Acute Physiology Score II. In-ICU mortality decreased from 84.7% to 68.5% for those patients placed under mechanical ventilation (p = 0.004) and from 91.2% to 78.4% for those who received renal replacement therapy (p = 0.04).

CONCLUSIONS

The outcome of patients with cirrhosis and septic shock has markedly improved over time, akin to the noncirrhotic population. In 2010, the in-ICU survival rate was 35%, which now fully justifies to admit these patients to ICU.

Development of the RTI item bank on risk of bias and precision of observational studies

OBJECTIVE

To create a practical and validated item bank for evaluating the risk of bias and precision of observational studies of interventions or exposures included in systematic evidence reviews.

STUDY DESIGN AND SETTING

The item bank, developed at RTI International, was created based on 1,492 questions included in earlier instruments, organized by the quality domains identified by Deeks et al. Items were eliminated and refined through face validity, cognitive, content validity, and interrater reliability testing.

RESULTS

The resulting item bank consisting of 29 questions for evaluating the risk of bias and precision of observational studies of interventions or exposures (1) captures all of the domains critical for evaluating this type of research, (2) is comprehensive and can be easily lifted "off the shelf" by different researchers, (3) can be adapted to different topic areas and study types (e.g., cohort, case-control, cross-sectional, and case series studies), and (4) provides sufficient instruction to apply the tool to varied topics.

CONCLUSION

One bank of items, with specific instructions for focusing abstractor evaluations, can be created to judge the risk of bias and precision of the variety of observational studies that may be used in systematic and comparative effectiveness reviews.

Risk of serious cardiac events in older adults using antipsychotic agents

BACKGROUND

Antipsychotic agents can lead to severe cardiovascular adverse events due to multiple mechanisms involving electrophysiologic and metabolic effects. Few epidemiologic studies have evaluated the risk of serious cardiovascular-related events in typical and atypical antipsychotic users.

OBJECTIVE

The purpose of this study was to compare the risk of serious cardiac events in older adults taking typical antipsychotics with those taking atypical antipsychotics.

METHODS

Prescription and medical information were derived from the IMS LifeLink Health Plan Claims database. The study involved a retrospective cohort of older adults (≥50 years) taking atypical or typical antipsychotics from July 1, 2000, to December 31, 2007. The primary outcome measure was hospitalization or emergency room visit due to serious cardiac events, including thromboembolism, myocardial infarction, cardiac arrest, and ventricular arrhythmias within 1 year after the index date. The 2 groups were matched on a propensity score to minimize the baseline differences between the groups. Survival analysis was conducted on the matched cohort to assess the risk of serious cardiovascular events in typical versus atypical users.

RESULTS

A total of 5580 patients were selected in each antipsychotic users group after propensity score matching. Serious cardiac events were found in 666 (11.9 %) atypical antipsychotic users and 698 (12.4%) typical antipsychotic users. Survival analysis revealed that typical antipsychotic users were at increased risk of serious cardiovascular events compared with atypical antipsychotic users (hazard ratio = 1.21; 95% CI, 1.04-1.40) after controlling for other factors.

CONCLUSIONS

Moderate increases in risk of serious cardiac events are associated with older adults using typical antipsychotic agents compared with atypical users. Health care professionals should carefully evaluate the benefit/risk ratio of antipsychotic agents before prescribing these agents to a vulnerable population.

Improving state Medicaid policies with comparative effectiveness research: a key role for academic health centers

After the Patient Protection and Affordable Care Act is fully implemented, Medicaid will be the largest single health care payer in the United States. Each U.S. state controls the size and scope of the medicine benefit beyond the federally mandated minimum; however, regulations that require balanced budgets and prohibit deficit spending limit each state's control. In a recessionary environment with reduced revenue, state Medicaid programs operate under a fixed or shrinking budget. Thus, the state Medicaid experience of providing high-quality care under explicit financial limits can inform Medicare and private payers of measures that control per-capita costs without adversely affecting health outcomes. The academic medicine community must play an expanded role in filling evidence gaps in order to continuously improve health policy making among U.S. states. The Drug Effectiveness Review Project and the Medicaid Evidence-based Decisions Project are two multistate Medicaid collaborations that leverage academic health center researchers' comparative effectiveness research (CER) projects to answer policy-relevant research questions. The authors of this article highlight how academic medicine can support states' health policies through CER and how CER-driven benefit-design choices can help states meet their cost and quality needs.

Implications for Nursing Research and Generation of Evidence

A sound informatics infrastructure is essential to optimise the application of evidence in nursing practice. A comprehensive review of the infrastructure and associated research methods is supported by an extensive resource of references to point the interested reader to further resources for more in depth study. Information and communication technology (ICT) has been recognized as a fundamental component of applying evidence to practice for several decades. Although the role of ICT in generating knowledge from practice was formally identified as a nursing informatics research priority in the early 1990s (NINR Priority Expert Panel on Nursing Informatics, 1993), it has received heightened interest recently. In this chapter, the authors summarize some important trends in research that motivate increased attention to practice-based generation of evidence. These include an increased emphasis on interdisciplinary, translational, and comparative effectiveness research; novel research designs; frameworks and models that inform generation of evidence from practice; and creation of data sets that include not only variables related to biological and genetic measures, but also social and behavioral variables. The chapter also includes an overview of the ICT infrastructure and informatics processes required to facilitate generation of evidence from practice and across research studies: (1) information structures (e.g., re-usable concept representations, tailored templates for data acquisition), (2) processes (e.g., data mining algorithms, natural language processing), and (3) technologies (e.g., data repositories, visualization tools that optimize cognitive support). In addition, the authors identify key knowledge gaps related to informatics support for nursing research and generation of evidence from practice.

Comparative effectiveness research: evidence-based medicine meets health care reform in the USA

Rationale Comparative effectiveness research (CER) is the study of two or more approaches to a health problem to determine which one results in better health outcomes. It is viewed by some in the USA as a promising strategy for health care reform. Aims and Objectives In this paper, nascent US CER policy will be described and analysed in order to determine its similarities and differences with EBM and its chances of success. Methods Document review and process tracing Results CER shares the logic of policies promoting evidence-based medicine, but invites greater methodological flexibility to ensure external validity across a range of health care topics. Conclusions This may narrow the inferential distance from knowledge to action, but efforts to change the US health care system through CER will face familiar epistemological quandaries and 'patient-centred' politics on the left and right.

The policy debate over public investment in comparative effectiveness research

BACKGROUND

Policy makers across the political spectrum, as well as many clinicians and physician professional associations, have proposed that better information on comparative clinical effectiveness should be a key element of any solution to the US health-care cost crisis. This superficial consensus hides intense disagreements over critical issues essential to any new public effort to promote more comparative effectiveness research (CER).

METHODS AND RESULTS

This article reviews the background for these disputes, summarizes the different perspectives represented by policy makers and advocates, and offers a framework to aid both practicing and academic internists in understanding the key elements of the emerging debate. Regarding the fundamental question of "what is CER," disagreements rage over whether value or cost effectiveness should be a consideration, and how specific patient perspectives should be reflected in the development and the use of such research. The question of how to pay for CER invokes controversies over the role of the market in producing such information and the private (e.g., insurers and employers) versus public responsibility for its production. The financing debate further highlights the high stakes of comparative effectiveness research, and the risks of stakeholder interests subverting any public process. Accordingly there are a range of proposals for the federal government's role in prioritization, development, and dissemination of CER.

CONCLUSION

The internal medicine community, with its long history of commitment to scientific medical practice and its leadership in evidence-based medicine, should have a strong interest and play an active role in this debate.

A nursing informatics research agenda for 2008-18: contextual influences and key components

The context for nursing informatics research has changed significantly since the National Institute of Nursing Research-funded Nursing Informatics Research Agenda was published in 1993 and the Delphi study of nursing informatics research priorities reported a decade ago. The authors focus on 3 specific aspects of context--genomic health care, shifting research paradigms, and social (Web 2.0) technologies--that must be considered in formulating a nursing informatics research agenda. These influences are illustrated using the significant issue of healthcare associated infections (HAI). A nursing informatics research agenda for 2008-18 must expand users of interest to include interdisciplinary researchers; build upon the knowledge gained in nursing concept representation to address genomic and environmental data; guide the reengineering of nursing practice; harness new technologies to empower patients and their caregivers for collaborative knowledge development; develop user-configurable software approaches that support complex data visualization, analysis, and predictive modeling; facilitate the development of middle-range nursing informatics theories; and encourage innovative evaluation methodologies that attend to human-computer interface factors and organizational context.

AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program

Comparative effectiveness reviews (CERs) are systematic reviews that evaluate evidence on alternative interventions to help clinicians, policy makers, and patients make informed treatment choices. Reviews should assess harms and benefits to provide balanced assessments of alternative interventions. Identifying important harms of treatment and quantifying the magnitude of any risks require CER authors to consider a broad range of data sources, including randomized controlled trials (RCTs) and observational studies. This may require evaluation of unpublished data in addition to published reports. Appropriate synthesis of harms data must also consider issues related to evaluation of rare or uncommon events, assessments of equivalence or noninferiority, and use of indirect comparisons. This article presents guidance for evaluating harms when conducting and reporting CERs. We include suggestions for prioritizing harms to be evaluated, use of terminology related to reporting of harms, selection of sources of evidence on harms, assessment of risk of bias (quality) of harms reporting, synthesis of evidence on harms, and reporting of evidence on harms.