Effectiveness of anti-TNFα for Crohn disease: research in a pediatric learning health system

Exploring Physicians' Dual Perspectives on the Transition From Free Text to Structured and Standardized Documentation Practices: Interview and Participant Observational Study

BACKGROUND

Clinical documentation plays a crucial role in providing and coordinating care. Despite the widespread adoption of electronic health record (EHR) systems, many end users still document clinical data in a manner similar to traditional paper-based records. To fully leverage the benefits of EHR systems, it is necessary to adopt new documentation approaches that facilitate easy access to information at the point of care and seamless exchange of information across health care facilities.

OBJECTIVE

We aimed to evaluate how the transition from an older EHR system to a cross-institutional EHR system impacts physicians' documentation practices and gain a deeper understanding of the factors influencing their choice between free text and structured and standardized documentation methods.

METHODS

A qualitative study was conducted between September 2023 and January 2024. It involved participant observations and individual semistructured interviews with physicians at a university hospital in Norway. Data were analyzed using reflexive thematic analysis.

RESULTS

The analysis revealed 3 main themes. First, physicians encountered challenges during the implementation phase of the new EHR system due to its complexity and their unfamiliarity with its use. However, with time, physicians gradually adopted new documentation processes. This integration or adoption primarily occurred by learning through practical experience and collaborative knowledge exchange with their peers. Second, although the implementation of the new EHR system had increased structured and standardized clinical documentation, free text remained the preferred method, with some exceptions. In addition, the fact that many physicians still relied on free-text documentation created a sense of distrust among them toward some of the standardized clinical data. Finally, the informants had mixed perceptions of Systematized Nomenclature of Medicine-Clinical Terms. Some viewed it as a more nuanced terminology system, while others found it more complex. Most informants found using templates for routine procedures beneficial as it saved time in the documentation process and ensured that all necessary parameters and documentation requirements were met.

CONCLUSIONS

The study findings revealed that physicians' acceptance of new documentation processes is influenced by various social and technological factors. These factors include previous documentation experiences, perceived benefits, familiarity with the EHR system, time constraints, and user-friendliness of the system. While physicians generally have a positive attitude toward using templates for routine procedures, they often create their own templates, and data within these templates are documented in a free-text format. To address this, health care organizations should consider implementing common standardized or semistandardized templates to reduce disparities in documentation, enhance data recording, and ensure adherence to guidelines. Furthermore, to facilitate the transition to the new documentation processes, we recommend providing physicians with customized training programs and platforms for tacit knowledge exchange.

A systematic review and proposed framework for sustainable learning healthcare systems

BACKGROUND

The healthcare sector is a complex domain that faces challenges in effectively learning from practices and outcome data. The Learning Health System (LHS) has emerged as a potential framework to improve healthcare by promoting continuous learning. However, its adoption remains limited, often involving only a single clinical department or a part of the LHS cycle. There is a need to gain a better understanding of implementing LHS on a larger scale.

AIM

To identify complete implementations of the LHS for providing recommendations into their implementation strategies, success factors, barriers, and outcomes.

METHODS

A systematic review was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines using PubMed and Scopus databases. Data from the included papers were thematically categorized into four primary areas: (1) Scale of LHS Implementation; (2) Implementation strategies and the factors that facilitated the implementation of LHS; (3) LHS outcomes; and (4) Barriers /challenges related to the LHS implementation.

RESULTS

We identified 1,279 papers, of which 37 were included in the final analysis. Barriers to implementing LHS included interoperability, data integration, electronic health records (EHRs) challenges, organizational culture, leadership, and regulatory hurdles. Most LHS initiatives lacked discussion on long-term economic sustainability models, and only 16 papers provided objective measurements of performance changes. Drawing from the findings of the included studies, this paper offers recommendations for the effective implementation of the LHS.

CONCLUSION

The establishment of sustainable LHS necessitates several key components. First, there is a need to develop long-term economic sustainability models. Secondly, governance at the national level should promote common Application Programming Interfaces (APIs) across LHS implementations, communication channels to share tacit knowledge, efficient Institutional Review Board, ethical approval processes, and connect various initiatives currently operating independently. Lastly, the success of LHS relies not only on technological infrastructure but also on the active participation of multidisciplinary teams in decision-making and sharing of tacit knowledge.

Using multiple imputation of real-world data to estimate clinical remission in pediatric inflammatory bowel disease

Aim: To evaluate the performance of the multiple imputation (MI) method for estimating clinical effectiveness in pediatric Crohn's disease in the ImproveCareNow registry; to address the analytical challenge of missing data. Materials & methods: Simulation studies were performed by creating missing datasets based on fully observed data from patients with moderate-to-severe Crohn's disease treated with non-ustekinumab biologics. MI was used to impute sPCDAI remission statuses in each simulated dataset. Results: The true remission rate (75.1% [95% CI: 72.6%, 77.5%]) was underestimated without imputation (72.6% [71.8%, 73.3%]). With MI, the estimate was 74.8% (74.4%, 75.2%). Conclusion: MI reduced nonresponse bias and improved the validity, reliability, and efficiency of real-world registry data to estimate remission rate in pediatric patients with Crohn's disease.

Quantification of Mucosal Activity from Colonoscopy Reports via the Simplified Endoscopic Mucosal Assessment for Crohn's Disease

BACKGROUND

Endoscopic mucosal healing is the gold standard for evaluating Crohn's disease (CD) treatment efficacy. Standard endoscopic indices are not routinely used in clinical practice, limiting the quality of retrospective research. A method for retrospectively quantifying mucosal activity from documentation is needed. We evaluated the simplified endoscopic mucosal assessment for CD (SEMA-CD) to determine if it can accurately quantify mucosal severity recorded in colonoscopy reports.

METHODS

Pediatric patients with CD underwent colonoscopy that was video recorded and evaluated via Simple Endoscopic Score for CD (SES-CD) and SEMA-CD by central readers. Corresponding colonoscopy reports were de-identified. Central readers blinded to clinical history and video scoring were randomly assigned colonoscopy reports with and without images. The SEMA-CD was scored for each report. Correlation with video SES-CD and SEMA-CD were assessed with Spearman rho, inter-rater, and intrarater reliability with kappa statistics.

RESULTS

Fifty-seven colonoscopy reports were read a total of 347 times. The simplified endoscopic mucosal assessment for CD without images correlated with both SES-CD and SEMA-CD from videos (rho = 0.82, P < .0001 for each). The addition of images provided similar correlation. Inter-rater and intrarater reliability were 0.93 and 0.92, respectively.

CONCLUSIONS

The SEMA-CD applied to retrospective evaluation of colonoscopy reports accurately and reproducibly correlates with SES-CD and SEMA-CD of colonoscopy videos. The SEMA-CD for evaluating colonoscopy reports will enable quantifying mucosal healing in retrospective research. Having objective outcome data will enable higher-quality research to be conducted across multicenter collaboratives and in clinical registries. External validation is needed.

The SHOnet learning health system: Infrastructure for continuous learning in pediatric rehabilitation

Introduction

To describe the development and implementation of learning health system (LHS) infrastructure for a pediatric specialty care health system to support LHS research in pediatric rehabilitation settings.

Methods

An existing pediatric common data model (eg, PEDSnet) of standardized medical terminologies for research was expanded and leveraged for this stud, and applied to SHOnet, a clinical research data resource consisting of deidentified data extracted from the electronic health record (EHR) from the Shriners Hospitals for Children speacialty pediatric health care system. We mapped EHR data for laboratory, procedures, drugs, and conditions to standardized vocabularies including ICD-10, CPT, RxNorm, and LOINC to the common data model using an established extraction-transformation-loading process. Rigorous quality checks were conducted to ensure a high degree of data conformance, completeness, and plausibility. SHOnet data elements from all sources are de-identified and the server is managed by the SHC Information Systems Department. SHOnet data are refreshed monthly and data elements are continually expanded based on new research endeavors.

Interventions

Not applicable.

Results

The Shriners Health Outcomes Network (SHOnet) includes data for over 10 000 distinct observational data elements based on over two million patient encounters between 2011 and present.

Conclusion

The systematic process to develop SHOnet is replicable and flexible for other pediatric rehabilitation research settings interested in building out their LHS capabilities. Challenges and facilitators may arise for building such LHS infrastructure for rehabilitation in areas of (a) data capture, curation, query, and governance, (b) generating knowledge from data, and (c) dissemination and implementation of new institutional knowledge. Further research studies are needed to evaluate these data resources for scalable system-learning endeavors.SHOnet is an exemplar of an LHS for rehabilitation and specialty care settings. The success of an LHS is dependent on engagement of multiple stakeholders, shared governance, effective knowledge translation, and deep commitment to long-term strategies for engaging clinicians, administration, and families in leveraging knowledge to improve clinical outcomes.

Clinical Outcome Assessments in Pediatric Patients With Ulcerative Colitis and Crohn's Disease Receiving Biologics: A Retrospective Cohort Study

Background

To assess disease activity, steroid-free remission, and other clinical outcome assessments among pediatric patients with ulcerative colitis (UC) and Crohn's disease (CD) in the ImproveCareNow (ICN) registry.

Methods

Patients aged 2-17 years diagnosed with UC or CD between June 1, 2013 and December 31, 2019 were enrolled if they initiated a biologic after enrollment in the ICN registry and completed at least 12 months follow-up after first maintenance dose. Baseline (at biologic initiation) demographics were summarized using descriptive statistics. Pediatric UC Activity Index (PUCAI), partial Mayo score, and Physician Global Assessment (PGA) were assessed for UC; and the Short Pediatric Crohn's Disease Activity Index (sPCDAI) and PGA were assessed for CD at first maintenance dose, 1- and 3-year time points. Kappa coefficients were used to assess the level of agreement between the outcome measures.

Results

A total of 1887 patients (UC = 350; CD = 1537) were included. Baseline demographics were similar across groups. For UC patients, mean PUCAI scores decreased and the proportion of patients in steroid-free remission, quiescent state based on PGA, and remission based on partial Mayo score increased from first maintenance dose to 1 and 3 years. For CD patients, mean sPCDAI score of CD patients decreased and the proportion of patients in steroid-free remission by sPCDAI and in quiescent state based on PGA increased from first maintenance dose to 1 and 3 years. Kappa coefficients showed only modest correlation between disease activity assessments.

Conclusions

Disease activity scores improved over time, with more pediatric patients with UC and CD achieving steroid-free remission at 1 and 3 years after first biologic maintenance dose.

Development and Testing of a New Simplified Endoscopic Mucosal Assessment for Crohn's Disease: The SEMA-CD

OBJECTIVES

Endoscopic mucosal improvement is the gold standard for assessing treatment efficacy in clinical trials of Crohn's disease. Current endoscopic indices are not routinely used in clinical practice. The lack of endoscopic information in large clinical registries limits their use for research. A quick, easy, and accurate method is needed for assessing mucosal improvement for clinicians in real-world practice. We developed and tested a novel simplified endoscopic mucosal assessment for Crohn's disease (SEMA-CD).

METHODS

We developed a 5-point scale for ranking endoscopic severity of ileum and colon based on Simple Endoscopic Score for Crohn's disease (SES-CD). Central readers were trained to perform SES-CD and SEMA-CD. Pediatric patients with Crohn's disease undergoing colonoscopy were enrolled. Video recordings of colonoscopies were de-identified and randomly assigned to blinded central readers. The SES-CD and SEMA-CD were scored for each video. The SES-CD was considered the validated standard for comparison. Correlation was assessed with Spearman rho, inter- and intrarater reliability with kappa statistics.

RESULTS

Fifty-seven colonoscopies were read a total of 212 times. Correlation between SEMA-CD and SES-CD was strong (rho = 0.98, P < 0.0001). Inter-rater reliability for SEMA-CD was 0.80, and intrarater reliability was 0.83. Central readers rated SEMA-CD as easier than SES-CD.

CONCLUSION

The SEMA-CD accurately and reproducibly correlates with the standard SES-CD. Central readers viewed SEMA-CD as easier than SES-CD. Use of SEMA-CD in practice should enable collecting mucosal improvement information in large populations of patients. This will improve the quality of research that can be conducted in clinical registries. External validation is needed.

Clinical data for paediatric research: the Swiss approach : Proceedings of the National Symposium in Bern, Switzerland, Dec 5-6, 2019

BACKGROUND AND PURPOSE

Continuous improvement of health and healthcare system is hampered by inefficient processes of generating new evidence, particularly in the case of rare diseases and paediatrics. Currently, most evidence is generated through specific research projects, which typically require extra encounters with patients, are costly and entail long delays between the recognition of specific needs in healthcare and the generation of necessary evidence to address those needs. The Swiss Personalised Health Network (SPHN) aims to improve the use of data obtained during routine healthcare encounters by harmonizing data across Switzerland and facilitating accessibility for research. The project "Harmonising the collection of health-related data and biospecimens in paediatric hospitals throughout Switzerland (SwissPedData)" was an infrastructure development project funded by the SPHN, which aimed to identify and describe available data on child health in Switzerland and to agree on a standardised core dataset for electronic health records across all paediatric teaching hospitals. Here, we describe the results of a two-day symposium that aimed to summarise what had been achieved in the SwissPedData project, to put it in an international context, and to discuss the next steps for a sustainable future. The target audience included clinicians and researchers who produce and use health-related data on children in Switzerland.

KEY HIGHLIGHTS

The symposium consisted of state-of-the-art lectures from national and international keynote speakers, workshops and plenary discussions. This manuscript summarises the talks and discussions in four sections: (I) a description of the Swiss Personalized Health Network and the results of the SwissPedData project; (II) examples of similar initiatives from other countries; (III) an overview of existing health-related datasets and projects in Switzerland; and (IV) a summary of the lessons learned and future prospective from workshops and plenary discussions.

IMPLICATIONS

Streamlined processes linking initial collection of information during routine healthcare encounters, standardised recording of this information in electronic health records and fast accessibility for research are essential to accelerate research in child health and make it affordable. Ongoing projects prove that this is feasible in Switzerland and elsewhere. International collaboration is vital to success. The next steps include the implementation of the SwissPedData core dataset in the clinical information systems of Swiss hospitals, the use of this data to address priority research questions, and the acquisition of sustainable funding to support a slim central infrastructure and local support in each hospital. This will lay the foundation for a national paediatric learning health system in Switzerland.

Measuring engagement in a collaborative learning health system: The case of ImproveCareNow

BACKGROUND

Collaborative learning health systems have demonstrated improved outcomes for a range of different chronic conditions. Patient and healthcare provider engagement in these systems is thought to be associated with improved outcomes. We have adapted an observational framework to measure, and track over time, engagement in ImproveCareNow, a collaborative learning health system for children with inflammatory bowel disease.

INTRODUCTION

We developed a categorical classification scheme for engagement in ImproveCareNow. Each tier is defined in terms of observable individual behaviors. When an individual completes one or more qualifying behavior, s/he is classified as engaged at that tier. Individuals are entered into a database, which is accessible to care centers throughout the ImproveCareNow network. Database records include fields for individual name, behavior type, time, place, and level of engagement.

RESULTS

The resulting system is employed at 79 ImproveCareNow care centers in the United States. The system recognizes four levels of engagement. Behaviors are recorded in a managed vocabulary and recorded in an online database. The database is queried weekly for individual engagement behaviors, which are tracked longitudinally. Center- and network-level statistics are generated and disseminated to stakeholders.

CONCLUSION

It is possible to monitor longitudinal engagement in a collaborative learning health system, thereby charting progress toward engagement goals and enabling quantitative evaluation of interventions aimed at increasing engagement.

Knee Joint Swelling at Presentation: A Case of Pediatric Crohn Disease With a TNFAIP3 Mutation

Crohn disease (CD) is a chronic inflammatory disease, and its incidence in children is rising. Despite extensive reports and investigations, the pathogenesis of CD has not been clearly elucidated, particularly in regard to triggering factors. A genetic predisposition is considered important when investigating the mechanism leading to CD, and the discovery of new CD-associated genes has increased our understanding of its immunopathogenesis and improved the efficacy of its treatment of CD. Early detection and treatment (eg, as children) with gene-based precision therapy can effectively prevent complications related to CD. In this case, a Chinese Han boy with CD associated with a mutation of tumor necrosis factor α-induced protein 3 was treated with recombinant human tumor necrosis factor-a receptor II:IgG Fc fusion protein. We suspected the boy had CD because of chronic abdominal pain, aphthous stomatitis, moderate anemia, a high erythrocyte sedimentation rate (36-79 mm/h), multiple intestinal ulcers, knee joint swelling, and a tumor necrosis factor α-induced protein 3 mutation. After total enteral nutrition and hormone therapy for 5 months, his abdominal pain and joint symptoms did not improve, so we started gene-based precision therapy with recombinant human tumor necrosis factor-a receptor II: IgG Fc fusion protein, which may play an important role in restricting TNF-α-induced NF-κB signaling. After 3 weeks, inflammation indicators were within the normal range, and multiple ulcers and joint symptoms were relieved. The present case demonstrates a safe therapeutic schedule that leads to rapid improvements in the clinical and biochemical status of patients with CD.

An Analysis of the Learning Health System in Its First Decade in Practice: Scoping Review

BACKGROUND

In the past decade, Lynn Etheredge presented a vision for the Learning Health System (LHS) as an opportunity for increasing the value of health care via rapid learning from data and immediate translation to practice and policy. An LHS is defined in the literature as a system that seeks to continuously generate and apply evidence, innovation, quality, and value in health care.

OBJECTIVE

This review aimed to examine themes in the literature and rhetoric on the LHS in the past decade to understand efforts to realize the LHS in practice and to identify gaps and opportunities to continue to take the LHS forward.

METHODS

We conducted a thematic analysis in 2018 to analyze progress and opportunities over time as compared with the initial Knowledge Gaps and Uncertainties proposed in 2007.

RESULTS

We found that the literature on the LHS has increased over the past decade, with most articles focused on theory and implementation; articles have been increasingly concerned with policy.

CONCLUSIONS

There is a need for attention to understanding the ethical and social implications of the LHS and for exploring opportunities to ensure that these implications are salient in implementation, practice, and policy efforts.

Robust-ODAL: Learning from heterogeneous health systems without sharing patient-level data

Electronic Health Records (EHR) contain extensive patient data on various health outcomes and risk predictors, providing an efficient and wide-reaching source for health research. Integrated EHR data can provide a larger sample size of the population to improve estimation and prediction accuracy. To overcome the obstacle of sharing patient-level data, distributed algorithms were developed to conduct statistical analyses across multiple clinical sites through sharing only aggregated information. However, the heterogeneity of data across sites is often ignored by existing distributed algorithms, which leads to substantial bias when studying the association between the outcomes and exposures. In this study, we propose a privacy-preserving and communication-efficient distributed algorithm which accounts for the heterogeneity caused by a small number of the clinical sites. We evaluated our algorithm through a systematic simulation study motivated by real-world scenarios and applied our algorithm to multiple claims datasets from the Observational Health Data Sciences and Informatics (OHDSI) network. The results showed that the proposed method performed better than the existing distributed algorithm ODAL and a meta-analysis method.

Identifying the "Blip on the Radar Screen": Leveraging Big Data in Defining Drug Safety and Efficacy in Pediatric Practice

The immense amount of electronic health data (pharmacy and administrative claims, electronic health records, and clinical registries) that is being generated every day in the care of patients has the potential to be leveraged for improving clinical decisions at the point of care, uncovering or validating drug efficacy and drug safety. The potential use of big data for improving safe and effective use of medications is especially important in children because of their low drug exposure relative to adults. Electronic health data is collected primarily for clinical or billing purposes and not for research purposes. The major steps involved in data acquisition, extraction, aggregation, analysis, modeling, and interpretation are discussed. It is important to understand the limitation of big data and utilize appropriate study design and statistical methods. Possible applications are presented along with specific examples of how big data has been used in drug research to find that blip on the radar screen that may give an efficacy or safety signal that can lead to further investigation.

Shared decision making in IBD: A novel approach to trial consent and timing

Background

Shared decision making (SDM) between families and physicians may facilitate informed, timely decisions to proceed with biologic therapy in children with inflammatory bowel disease (IBD). Our team previously developed an SDM tool to aid communication between physicians and families when considering biologic therapy for children with IBD.

Objective

We are conducting a prospective, pre-post pilot trial of a new SDM tool. The primary aim of the study is to assess feasibility of both the intervention and trial procedures for a future large-scale trial.

Methods

We are enrolling physicians with experience prescribing biologic therapy in the past year and families of children with IBD. Families in the intervention arm receive a 3-step intervention including a letter sent before trial consent or clinic appointment, an in-clinic decision tool and a follow-up phone call. Our primary trial outcome is a measure of feasibility, with measures of clinical and decision outcomes secondary. We seek to enroll 27 families in each of 2 arms (usual-care and intervention) and plan data collection at the time of the initial visit or hospital stay, and at 1 week, 3 months, and 6 months after the initial visit.

Conclusion

This study protocol is designed to demonstrate that integrating novel consent procedures, including timing and multiple versions of written consent, may increase trial feasibility while maintaining scientific rigor and full protection of study participants.

Shared Decision Making About Starting Anti-TNFs: A Pediatric Perspective

Shared decision making (SDM) is central to patient-centered medicine and has the potential to improve outcomes for pediatric patients with inflammatory bowel diseases. We surveyed specialists about their use of SDM in the decision to start a tumor necrosis factor-α inhibitor in pediatric patients. Results were compared between those who reported using SDM and those who did not. Of 209 respondents, 157 (75%) reported using SDM. Physician/practice characteristics were similar between users and nonusers. There were no statistically significant differences between groups in the components deemed important to the decision-making process nor the number of barriers or facilitators to SDM. Exploratory analyses suggested that physicians using SDM were more accepting of adolescent involvement in the decision-making process. Our results question the effectiveness of using reported barriers and facilitators to guide interventions to improve use of SDM, and suggest further work is needed to understand the adolescent role in decision making.

Learning health systems

Healthcare organizations have invested significant resources into integrating comprehensive electronic health record (EHR) systems into clinical care. EHRs digitize healthcare in ways that allow for repurposing of clinical information to support quality improvement, research, population health, and health system analytics. This has facilitated the development of Learning Health Systems. Learning health systems (LHS) merge healthcare delivery with research, data science, and quality improvement processes. The LHS cycle begins and ends with the clinician-patient interaction, and aspires to provide continuous improvements in quality, outcomes, and health care efficiency. Although, the health sector has been slow to embrace the LHS concept, innovative approaches for improving healthcare, such as a LHS, have shown that better outcomes can be achieved by engaging patients and physicians in communities committed to a common purpose. Here, we explore the mission of a pediatric LHS, such as PEDSnet, which is driven by the distinctive goals of a child's well-being. Its vision is to create a national LHS architecture in which all pediatric institutions can participate. While challenges still exist in the development and adoption of LHS, these challenges are being met with innovative strategies and strong collaborative relationships to reduce system uncertainty while improving patient outcomes.

Secondary Loss of Response to Infliximab in Pediatric Crohn Disease: Does It Matter How and When We Start?

OBJECTIVES

A significant proportion of children with Crohn disease develop a secondary loss of response (LOR) to infliximab. Our aim was to study the impact of initial treatment strategies on secondary LOR.

METHODS

We reviewed the medical records of children with Crohn disease who received scheduled maintenance infliximab therapy for at least 12 months. We compared children who developed LOR with those who did not; with regards to their clinical and laboratory parameters, disease phenotype, and treatment strategy before developing LOR.

RESULTS

A total of 73 children (median age at diagnosis 11 (2-16) years, 41 boys) who had received a median duration of 33 (13-110) months of infliximab therapy were included in the final analysis. LOR was seen in 25(34.2%). Demographic variables, disease phenotype (age, disease location, and behavior), inflammatory parameters, and pediatric Crohn disease activity index at induction with infliximab were similar between both groups. Children with LOR had a significantly greater number of flares of the disease when compared to those who did not have LOR (4 [1-8] vs 2 [1-5] P = 0.03). The choice of the concomitant immunomodulator-methotrexate (11/29 [37.9%]) versus azathioprine (11/36 [30.5%]) (P = 0.6) did not affect LOR rates. The median time-lag between diagnosis and induction with infliximab was significantly longer in children with LOR as compared to those who did not have an LOR (28 [4-90] months vs 12.5 [1-121] months, P = 0.004).

CONCLUSION

Early use of infliximab in pediatric Crohn disease is associated with a decrease in secondary LOR. The type of concomitant immunomodulator used does not make a difference to LOR rates.

Analytical Methods for a Learning Health System: 3. Analysis of Observational Studies

The third paper in a series on how learning health systems can use routinely collected electronic health data (EHD) to advance knowledge and support continuous learning, this review describes how analytical methods for individual-level electronic health data EHD, including regression approaches, interrupted time series (ITS) analyses, instrumental variables, and propensity score methods, can also be used to address the question of whether the intervention “works.”

The two major potential sources of bias in non-experimental studies of health care interventions are that the treatment groups compared do not have the same probability of treatment or exposure and the potential for confounding by unmeasured covariates. Although very different, the approaches presented in this chapter are all based on assumptions about data, causal relationships, and biases. For instance, regression approaches assume that the relationship between the treatment, outcome, and other variables is properly specified, all of the variables are available for analysis (i.e., no unobserved confounders) and measured without error, and that the error term is independent and identically distributed. The instrumental variables approach requires identifying an instrument that is related to the assignment of treatment but otherwise has no direct on the outcome. Propensity score methods approaches, on the other hand, assume that there are no unobserved confounders. The epidemiological designs discussed also make assumptions, for instance that individuals can serve as their own control.

To properly address these assumptions, analysts should conduct sensitivity analyses within the assumptions of each method to assess the potential impact of what cannot be observed. Researchers also should analyze the same data with different analytical approaches that make alternative assumptions, and to apply the same methods to different data sets. Finally, different analytical methods, each subject to different biases, should be used in combination and together with different designs, to limit the potential for bias in the final results.

Architectural frameworks: defining the structures for implementing learning health systems

Background

The vision of transforming health systems into learning health systems (LHSs) that rapidly and continuously transform knowledge into improved health outcomes at lower cost is generating increased interest in government agencies, health organizations, and health research communities. While existing initiatives demonstrate that different approaches can succeed in making the LHS vision a reality, they are too varied in their goals, focus, and scale to be reproduced without undue effort. Indeed, the structures necessary to effectively design and implement LHSs on a larger scale are lacking. In this paper, we propose the use of architectural frameworks to develop LHSs that adhere to a recognized vision while being adapted to their specific organizational context. Architectural frameworks are high-level descriptions of an organization as a system; they capture the structure of its main components at varied levels, the interrelationships among these components, and the principles that guide their evolution. Because these frameworks support the analysis of LHSs and allow their outcomes to be simulated, they act as pre-implementation decision-support tools that identify potential barriers and enablers of system development. They thus increase the chances of successful LHS deployment.

Discussion

We present an architectural framework for LHSs that incorporates five dimensions—goals, scientific, social, technical, and ethical—commonly found in the LHS literature. The proposed architectural framework is comprised of six decision layers that model these dimensions. The performance layer models goals, the scientific layer models the scientific dimension, the organizational layer models the social dimension, the data layer and information technology layer model the technical dimension, and the ethics and security layer models the ethical dimension. We describe the types of decisions that must be made within each layer and identify methods to support decision-making.

Conclusion

In this paper, we outline a high-level architectural framework grounded in conceptual and empirical LHS literature. Applying this architectural framework can guide the development and implementation of new LHSs and the evolution of existing ones, as it allows for clear and critical understanding of the types of decisions that underlie LHS operations. Further research is required to assess and refine its generalizability and methods.

Human Microbiome and Learning Healthcare Systems: Integrating Research and Precision Medicine for Inflammatory Bowel Disease

Healthcare institutions face widespread challenges of delivering high-quality and cost-effective care, while keeping up with rapid advances in biomedical knowledge and technologies. Moreover, there is increased emphasis on developing personalized or precision medicine targeted to individuals or groups of patients who share a certain biomarker signature. Learning healthcare systems (LHS) have been proposed for integration of research and clinical practice to fill major knowledge gaps, improve care, reduce healthcare costs, and provide precision care. To date, much discussion in this context has focused on the potential of human genomic data, and not yet on human microbiome data. Rapid advances in human microbiome research suggest that profiling of, and interventions on, the human microbiome can provide substantial opportunity for improved diagnosis, therapeutics, risk management, and risk stratification. In this study, we discuss a potential role for microbiome science in LHSs. We first review the key elements of LHSs, and discuss possibilities of Big Data and patient engagement. We then consider potentials and challenges of integrating human microbiome research into clinical practice as part of an LHS. With rapid growth in human microbiome research, patient-specific microbial data will begin to contribute in important ways to precision medicine. Hence, we discuss how patient-specific microbial data can help guide therapeutic decisions and identify novel effective approaches for precision care of inflammatory bowel disease. To the best of our knowledge, this expert analysis makes an original contribution with new insights poised at the emerging intersection of LHSs, microbiome science, and postgenomics medicine.

High Levels of Decisional Conflict and Decision Regret When Making Decisions About Biologics

OBJECTIVES

The aim of the study was to understand the association between parents' perceptions of the decision process and the decision outcomes in decisions about the use of biologics in pediatric chronic conditions.

METHODS

We mailed surveys to parents of children with inflammatory bowel disease or juvenile idiopathic arthritis who had started treatment with biologics in the prior 2 years and were treated at either of 2 children's hospitals. The survey included measures of the decision process, including decision control and physician engagement, and decision outcomes, including conflict and regret. We used means and frequencies to assess the response distributions. General linear models were used to test the associations between decision process and decision outcomes.

RESULTS

We had 201 respondents (response rate 54.9%). Approximately 47.0% reported using shared decision making. Each physician engagement behavior was experienced by the majority of parents, with the highest percentage reporting that their child's physician used language they understood and listened to them. Approximately 48.5% of parents had decisional conflict scores of 25 or greater, indicating high levels of conflict. Approximately 28.2% had no regret, 31.8% had mild regret, and the remaining 40.0% had moderate to severe regret. Shared decision making was not associated with improved decisional conflict, but physician engagement behaviors were associated with both decisional conflict and regret.

CONCLUSIONS

Improving decision outcomes will require more than just focusing on who parents perceive as controlling the final decision. Developing interventions that facilitate specific physician engagement behaviors may decrease parents' distress around decision making and improve decision outcomes.

Development and Refinement of a Learning Health Systems Training Program

CONTEXT

In the emerging Learning Health System (LHS), the application and generation of medical knowledge are a natural outgrowth of patient care. Achieving this ideal requires a physician workforce adept in information systems, quality improvement methods, and systems-based practice to be able to use existing data to inform future care. These skills are not currently taught in medical school or graduate medical education.

CASE DESCRIPTION

We initiated a first-ever Learning Health Systems Training Program (LHSTP) for resident physicians. The curriculum builds analytical, informatics and systems engineering skills through an active-learning project utilizing health system data that culminates in a final presentation to health system leadership.

FINDINGS

LHSTP has been in place for two years, with 14 participants from multiple medical disciplines. Challenges included scheduling, mentoring, data standardization, and iterative optimization of the curriculum for real-time instruction. Satisfaction surveys and feedback were solicited mid-year in year 2. Most respondents were satisfied with the program, and several participants wished to continue in the program in various capacities after their official completion.

MAJOR THEMES

We adapted our curriculum to successes and challenges encountered in the first two years. Modifications include a revised approach to teaching statistics, smaller cohorts, and more intensive mentorship. We continue to explore ways for our graduates to remain involved in the LHSTP and to disseminate this program to other institutions.

CONCLUSION

The LHSTP is a novel curriculum that trains physicians to lead towards the LHS. Successful methods have included diverse multidisciplinary educators, just in time instruction, tailored content, and mentored projects with local health system impact.

Parents' information needs and influential factors when making decisions about TNF-α inhibitors

BACKGROUND

Parents struggle when making treatment decisions for children with arthritis or other chronic conditions. Understanding their decision-making process is an essential step towards improving the decision-making experience. The objective of this study was to describe parents' information needs and the influences on their decision making about treatment with TNF-α inhibitors.

METHODS

Survey domains were developed based on qualitative data and cognitive interviewing. We mailed the survey to parents of children with juvenile idiopathic arthritis or inflammatory bowel disease who had initiated treatment with TNF-α inhibitors in the prior 2 years. Data were analyzed using descriptive and non-parametric statistics.

RESULTS

Survey response rate was 54.9 %. Each item had <2 % missing responses. Parents used an array of information sources when deciding about treatment with TNF-α inhibitors. Resources other than their child's specialist were most often used to increase confidence in parents' decisions or because they wanted to know more about other people's experiences being treated with TNF-α inhibitors, rather than due to a lack of understanding. All but two (cost and route of administration) of the influential decision factors were very or extremely important to the majority of participants with factors related to long-term side effects, treatment efficacy, and disease impact being most important.

CONCLUSIONS

This study describes parents' information needs and influential factors in treatment decision making. Results suggest that future work should be aimed at helping families weigh risks and benefits, such as through decision support interventions, as well as developing opportunities to include people beyond the family and physician in the decision-making process.

Effectiveness Research in Inflammatory Bowel Disease: A Necessity and a Methodological Challenge

Efficacy, safety and economic issues are the main factors influencing the use of inflammatory bowel disease [IBD]-related medications. The best level of evidence comes from randomised clinical trials. The benefit of the intervention observed in a clinical trial could be reduced once it is implemented in clinical practice: its real-life efficacy, known as effectiveness, could be questioned. That is why effectiveness research based on observational studies is required to obtain-long term data on natural history, including surgery or hospitalisation, and safety. Before starting these real-life studies, it is crucial to be aware of the inherent risks of bias and confounding, to develop a good study plan, and to select the optimal design. Even if the choice of the design is optimal and if the risks of bias and confounding are minimised, the implementation of robust statistical methodology is necessary to increase the validity of the results and allow their dissemination into clinical practice. The objective of this paper is to highlight some inherent methodological problems in effectiveness research and to review some statistical tools with a focus on IBD studies and trials.

PEDSnet: how a prototype pediatric learning health system is being expanded into a national network

Except for a few conditions, pediatric disorders are rare diseases. Because of this, no single institution has enough patients to generate adequate sample sizes to produce generalizable knowledge. Aggregating electronic clinical data from millions of children across many pediatric institutions holds the promise of producing sufficiently large data sets to accelerate knowledge discovery. However, without deliberately embedding these data in a pediatric learning health system (defined as a health care organization that is purposefully designed to produce research in routine care settings and implement evidence at the point of care), efforts to act on this new knowledge, reducing the distress and suffering that children experience when sick, will be ineffective. In this article we discuss a prototype pediatric learning health system, ImproveCareNow, for children with inflammatory bowel disease. This prototype is being scaled up to create PEDSnet, a national network that will support the efficient conduct of clinical trials, observational research, and quality improvement across diseases, specialties, and institutions.

A Digital Architecture for a Network-Based Learning Health System: Integrating Chronic Care Management, Quality Improvement, and Research

INTRODUCTION

We collaborated with the ImproveCareNow Network to create a proof-of-concept architecture for a network-based Learning Health System. This collaboration involved transitioning an existing registry to one that is linked to the electronic health record (EHR), enabling a "data in once" strategy. We sought to automate a series of reports that support care improvement while also demonstrating the use of observational registry data for comparative effectiveness research.

DESCRIPTION OF ARCHITECTURE

We worked with three leading EHR vendors to create EHR-based data collection forms. We automated many of ImproveCareNow's analytic reports and developed an application for storing protected health information and tracking patient consent. Finally, we deployed a cohort identification tool to support feasibility studies and hypothesis generation. There is ongoing uptake of the system. To date, 31 centers have adopted the EHR-based forms and 21 centers are uploading data to the registry. Usage of the automated reports remains high and investigators have used the cohort identification tools to respond to several clinical trial requests.

SUGGESTIONS FOR FUTURE USE

The current process for creating EHR-based data collection forms requires groups to work individually with each vendor. A vendor-agnostic model would allow for more rapid uptake. We believe that interfacing network-based registries with the EHR would allow them to serve as a source of decision support. Additional standards are needed in order for this vision to be achieved, however.

CONCLUSIONS

We have successfully implemented a proof-of-concept Learning Health System while providing a foundation on which others can build. We have also highlighted opportunities where sponsors could help accelerate progress.

Building electronic data infrastructure for comparative effectiveness research: accomplishments, lessons learned and future steps

There are large gaps in our knowledge on the potential impact of diagnostics and therapeutics on outcomes of patients treated in the real world. Comparative effectiveness research aims to fill these gaps to maximize effectiveness of these interventions. Health information technology has the potential to dramatically improve the practice of medicine and of research. This is an overview of about US$100 million of American Recovery and Reinvestment Act investment in 12 projects managed by the Agency for Healthcare Research and Quality to build an electronic clinical data infrastructure that connects research with healthcare delivery. The achievements and lessons learned from these projects provided a foundation for the National Patient-Centered Clinical Research Network (PCORnet)and will help to guide future infrastructure development needed to build an efficient, scalable and sustainable learning health system.