The Learning Healthcare System: Where are we now? A systematic review

Which heart failure patients benefit most from non-invasive telemedicine? An overview of current evidence and future directions

Telemedicine in heart failure (HF) management may positively impact health outcomes, but varied effects in studies hinder guidance in HF guidelines. Evidence on the effectiveness of telemedicine in HF subpopulations is limited. We conducted a scoping review to evaluate and synthesise evidence on the effectiveness of telemedicine across HF subpopulations that could guide telemedicine strategies in routine practice. Meta-analyses concerning randomised controlled trials (RCTs) with subgroup analyses on telemedicine effectives were identified in PubMed. We identified 15 RCTs, encompassing 21 different subgroups based on characteristics of HF patients. Findings varied across studies and no definite evidence was found about which patients benefit most from telemedicine. Subgroup definitions were inconsistent, not always a priori defined and subgroups contained few patients. Some studies found heterogeneous effects of telemedicine on mortality and hospitalisation across subgroups defined by: New York Heart Association (NYHA) classification, previous HF decompensation, implantable device, concurrent depression, time since hospital discharge and duration of HF. Patients represented in the RCTs were mostly male, aged 65-75 years, with HF with reduced ejection fraction and NYHA class II/III. Traditional RCTs have not been able to provide clinicians with guidance; continuous real-world evidence generation could enhance monitoring and identify who benefits from telemedicine.

Exploring the barriers, facilitators and needs to use patient outcomes in district nursing care: A multi-method qualitative study

AIM AND OBJECTIVES

To provide an in-depth insight into the barriers, facilitators and needs of district nurses and nurse assistants on using patient outcomes in district nursing care.

BACKGROUND

As healthcare demands grow, particularly in district nursing, there is a significant need to understand how to systematically measure and improve patient outcomes in this setting. Further investigation is needed to identify the barriers and facilitators for effective implementation.

DESIGN

A multi-method qualitative study.

METHODS

Open-ended questions of a survey study (N = 132) were supplemented with in-depth online focus group interviews involving district nurses and nurse assistants (N = 26) in the Netherlands. Data were analysed using thematic analysis.

RESULTS

Different barriers, facilitators and needs were identified and compiled into 16 preconditions for using outcomes in district nursing care. These preconditions were summarised into six overarching themes: follow the steps of a learning healthcare system; provide patient-centred care; promote the professional's autonomy, attitude, knowledge and skills; enhance shared responsibility and collaborations within and outside organisational boundaries; prioritise and invest in the use of outcomes; and boost the unity and appreciation for district nursing care.

CONCLUSIONS

The preconditions identified in this study are crucial for nurses, care providers, policymakers and payers in implementing the use of patient outcomes in district nursing practice. Further exploration of appropriate strategies is necessary for a successful implementation.

RELEVANCE TO CLINICAL PRACTICE

This study represents a significant step towards implementing the use of patient outcomes in district nursing care. While most research has focused on hospitals and general practitioner settings, this study focuses on the needs for district nursing care. By identifying 16 key preconditions across themes such as patient-centred care, professional autonomy and unity, the findings offer valuable guidance for integrating a learning healthcare system that prioritises the measurement and continuous improvement of patient outcomes in district nursing.

REPORTING METHOD

Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines.

PATIENT OF PUBLIC CONTRIBUTION

No patient or public contribution.

Promoting Diversity, Equity, Inclusion, and Justice in Grantmaking for Health Care Research: A Pragmatic Review and Framework

Funders of research have an opportunity to advance health equity and social justice by incorporating principles of diversity, equity, inclusion, and justice (DEIJ) in their approach to grantmaking. We conducted a pragmatic review to identify opportunities for grantmakers in the health care sector to integrate DEIJ in their funding activities. The resulting framework discusses recommendations within three phases as follows: (1) Organizational Context (i.e., initiate DEIJ efforts within the grantmaking organization, invest in community partnerships, and establish DEIJ goals), (2) Grantmaking Process (i.e., DEIJ-specific practices related to grant design, application, proposal review processes, and support for grantees), and (3) Assessment of Process and Outcomes (i.e., measurement, evaluation, and dissemination to maximize impact of DEIJ efforts). Throughout all grantmaking phases, it is critical to partner with and engage individuals and communities that have been historically marginalized in health care and research. In this article, we describe how adoption of framework practices can leverage grantmaking to advance DEIJ for communities, researchers, and projects.

Leveraging data to support health equity in an integrated delivery and finance system

Introduction

To accelerate healthcare transformation and advance health equity, scientists in learning health systems (LHSs) require ready access to integrated, comprehensive data that includes information on social determinants of health (SDOH).

Methods

We describe how an integrated delivery and finance system leveraged its learning ecosystem to advance health equity through (a) a cross-sector initiative to integrate healthcare and human services data for better meeting clients' holistic needs and (b) a system-level initiative to collect and use patient-reported SDOH data for connecting patients to needed resources.

Results

Through these initiatives, we strengthened our health system's capacity to meet diverse patient needs, address health disparities, and improve health outcomes. By sharing and integrating healthcare and human services data, we identified 281 000 Shared Services Clients and enhanced care management for 100 adult Medicaid/Special Needs Plan members. Over a 1-year period, we screened 9173 (37%) patients across UPMC's Women's Health Services Line and connected over 700 individuals to social services and supports.

Conclusions

Opportunities exist for LHSs to improve, expand, and sustain their innovative data practices. As learnings continue to emerge, LHSs will be well positioned to accelerate healthcare transformation and advance health equity.

Using implementation science to operate as a learning health system to improve outcomes in early psychosis

AIM

Early interventions are well understood to improve psychosis outcomes, but their successful implementation remains limited. This article introduces a three-step roadmap for advancing the implementation of evidence-based practices to operate as a learning health system, which can be applied to early interventions for psychosis and is intended for an audience that is relatively new to systematic approaches to implementation.

METHODS

The roadmap is grounded in implementation science, which specializes in methods to promote routine use of evidence-based innovations. The roadmap draws on learning health system principles that call for commitment of leadership, application of evidence, examination of care experiences, and study of health outcomes. Examples are discussed for each roadmap step, emphasizing both data- and stakeholder-related considerations applicable throughout the roadmap.

CONCLUSIONS

Early psychosis care is a promising topic through which to discuss the critical need to move evidence into practice. Despite remarkable advances in early psychosis interventions, population-level impact of those interventions is yet to be realized. By providing an introduction to how implementation science principles can be operationalized in a learning health system and sharing examples from early psychosis care, this article prompts inclusion of a wider audience in essential discourse on the role that implementation science can play for moving evidence into practice for other realms of psychiatric care as well. To this end, the proposed roadmap can serve as a conceptual guiding template and framework through which various psychiatric services can methodically pursue timely implementation of evidence-based interventions for higher quality care and improved outcomes.

Building Learning Healthcare Systems for Critical Care Medicine

Learning healthcare systems are an evolving way of integrating informatics, analytics, and continuous improvement into daily practice in healthcare. This article discusses strategies to build learning healthcare systems for critical care medicine.

Building ontology-based temporal databases for data reuse: An applied example on hospital organizational structures

Keeping track of data semantics and data changes in the databases is essential to support retrospective studies and the reproducibility of longitudinal clinical analysis by preventing false conclusions from being drawn from outdated data. A knowledge model combined with a temporal model plays an essential role in organizing the data and improving query expressiveness across time and multiple institutions. This paper presents a modelling framework for temporal relational databases using an ontology to derive a shareable and interoperable data model. The framework is based on: OntoRela an ontology-driven database modelling approach and Unified Historicization Framework a temporal database modelling approach. The method was applied to hospital organizational structures to show the impact of tracking organizational changes on data quality assessment, healthcare activities and data access rights. The paper demonstrated the usefulness of an ontology to provide a formal, interoperable, and reusable definition of entities and their relationships, as well as the adequacy of the temporal database to store, trace, and query data over time.

Progress with the Learning Health System 2.0: a rapid review of Learning Health Systems' responses to pandemics and climate change

BACKGROUND

Pandemics and climate change each challenge health systems through increasing numbers and new types of patients. To adapt to these challenges, leading health systems have embraced a Learning Health System (LHS) approach, aiming to increase the efficiency with which data is translated into actionable knowledge. This rapid review sought to determine how these health systems have used LHS frameworks to both address the challenges posed by the COVID-19 pandemic and climate change, and to prepare for future disturbances, and thus transition towards the LHS2.0.

METHODS

Three databases (Embase, Scopus, and PubMed) were searched for peer-reviewed literature published in English in the five years to March 2023. Publications were included if they described a real-world LHS's response to one or more of the following: the COVID-19 pandemic, future pandemics, current climate events, future climate change events. Data were extracted and thematically analyzed using the five dimensions of the Institute of Medicine/Zurynski-Braithwaite's LHS framework: Science and Informatics, Patient-Clinician Partnerships, Continuous Learning Culture, Incentives, and Structure and Governance.

RESULTS

The search yielded 182 unique publications, four of which reported on LHSs and climate change. Backward citation tracking yielded 13 additional pandemic-related publications. None of the climate change-related papers met the inclusion criteria. Thirty-two publications were included after full-text review. Most were case studies (n = 12, 38%), narrative descriptions (n = 9, 28%) or empirical studies (n = 9, 28%). Science and Informatics (n = 31, 97%), Continuous Learning Culture (n = 26, 81%), Structure and Governance (n = 23, 72%) were the most frequently discussed LHS dimensions. Incentives (n = 21, 66%) and Patient-Clinician Partnerships (n = 18, 56%) received less attention. Twenty-nine papers (91%) discussed benefits or opportunities created by pandemics to furthering the development of an LHS, compared to 22 papers (69%) that discussed challenges.

CONCLUSIONS

An LHS 2.0 approach appears well-suited to responding to the rapidly changing and uncertain conditions of a pandemic, and, by extension, to preparing health systems for the effects of climate change. LHSs that embrace a continuous learning culture can inform patient care, public policy, and public messaging, and those that wisely use IT systems for decision-making can more readily enact surveillance systems for future pandemics and climate change-related events.

TRIAL REGISTRATION

PROSPERO pre-registration: CRD42023408896.

Seamless EMR data access: Integrated governance, digital health and the OMOP-CDM

Objectives In this overview, we describe theObservational Medical Outcomes Partnership Common Data Model (OMOP-CDM), the established governance processes employed in EMR data repositories, and demonstrate how OMOP transformed data provides a lever for more efficient and secure access to electronic medical record (EMR) data by health service providers and researchers.Methods Through pseudonymisation and common data quality assessments, the OMOP-CDM provides a robust framework for converting complex EMR data into a standardised format. This allows for the creation of shared end-to-end analysis packages without the need for direct data exchange, thereby enhancing data security and privacy. By securely sharing de-identified and aggregated data and conducting analyses across multiple OMOP-converted databases, patient-level data is securely firewalled within its respective local site.Results By simplifying data management processes and governance, and through the promotion of interoperability, the OMOP-CDM supports a wide range of clinical, epidemiological, and translational research projects, as well as health service operational reporting.Discussion Adoption of the OMOP-CDM internationally and locally enables conversion of vast amounts of complex, and heterogeneous EMR data into a standardised structured data model, simplifies governance processes, and facilitates rapid repeatable cross-institution analysis through shared end-to-end analysis packages, without the sharing of data.Conclusion The adoption of the OMOP-CDM has the potential to transform health data analytics by providing a common platform for analysing EMR data across diverse healthcare settings.

From digital health to learning health systems: four approaches to using data for digital health design

Digital health technologies, powered by digital data, provide an opportunity to improve the efficacy and efficiency of health systems at large. However, little is known about different approaches to the use of data for digital health design, or about their possible relations to system-level dynamics. In this contribution, we identify four existing approaches to the use of data for digital health design, namely the silent, the overt, the data-enabled, and the convergent. After characterising the approaches, we provide real-life examples of each. Furthermore, we compare the approaches in terms of selected desirable characteristics of the design process, highlighting relative advantages and disadvantages. Finally, we reflect on the system-level relevance of the differentiation between the approaches and point towards future research directions. Overall, the contribution provides researchers and practitioners with a broad conceptual framework to examine data-related challenges and opportunities in digital health design.

Frameworks, guidelines, and tools to develop a learning health system for Indigenous health: An environmental scan for Canada

Introduction

First Nations, Inuit, and Métis (FNIM) peoples experience systemic health disparities within Ontario's healthcare system. Learning health systems (LHS) is a rapidly growing interdisciplinary area with the potential to address these inequitable health outcomes through a comprehensive health system that draws on science, informatics, incentives, and culture for ongoing innovation and improvement. However, global literature is in its infancy with grounding theories and principles still emerging. In addition, there is inadequate information on LHS within Ontario's health care context.

Methods

We conducted an environmental scan between January and April 2021 and again in June 2022 to identify existing frameworks, guidelines, and tools for designing, developing, implementing, and evaluating an LHS.

Results

We found 37 relevant sources. This paper maps the literature and identifies gaps in knowledge based on five key pillars: (a) data and evidence-driven, (b) patient-centeredness, (c) system-supported, (d) cultural competencies enabled, and (e) the learning health system.

Conclusion

We provide recommendations for implementation accordingly. The literature on LHS provides a starting point to address the health disparities of FNIM peoples within the healthcare system but Indigenous community partnerships in LHS development and operation will be key to success.

Deep medicine … Navigating the intersection of technology, cognition and ethics in the digital age of medicine

The digital expansion in medicine and healthcare has been immense and extremely valuable. The biggest concern in the face of this inevitable growth is how we manage to keep contact with our patients and preserve the human touch so essential in healing. Digital healthcare should not be about technology replacing clinicians. Instead, it should be about augmenting and supplementing healthcare providers to improve the ways in which we deliver personalised healthcare. It is vital that we focus on how we can revitalise the patient-clinician relationship in this digital age.

External validity in distributed data networks

PURPOSE

While much has been written about how distributed networks address internal validity, external validity is rarely discussed. We aimed to define key terms related to external validity, discuss how they relate to distributed networks, and identify how three networks (the US Food and Drug Administration's Sentinel System, the Canadian Network for Observational Drug Effect Studies [CNODES], and the National Patient Centered Clinical Research Network [PCORnet]) deal with external validity.

METHODS

We define external validity, target populations, target validity, generalizability, and transportability and describe how each relates to distributed networks. We then describe Sentinel, CNODES, and PCORnet and how each approaches these concepts, including a sample case study.

RESULTS

Each network approaches external validity differently. As its target population is US citizens and it includes only US data, Sentinel primarily worries about lack of external validity by not including some segments of the population. The fact that CNODES includes Canadian, United States, and United Kingdom data forces them to seriously consider whether the United States and United Kingdom data will be transportable to Canadian citizens when they meta-analyze database-specific estimates. PCORnet, with its focus on study-specific cohorts and pragmatic trials, conducts more case-by-case explorations of external validity for each new analytic data set it generates.

CONCLUSIONS

There is no one-size-fits-all approach to external validity within distributed networks. With these networks and comparisons between their findings becoming a key part of pharmacoepidemiology, there is a need to adapt tools for improving external validity to the distributed network setting.

Implementation model for a national learning health system (IMPLEMENT-National LHS): a concept analysis and systematic review protocol

INTRODUCTION

Despite efforts and repeated calls to improve the organisation and quality of healthcare and services, and in view of the many challenges facing health systems, the results and capacity to adapt and integrate innovations and new knowledge remain suboptimal. Learning health systems (LHS) may be an effective model to accelerate the application of research for real quality improvement in healthcare. However, while recognising the enormous potential of LHS, the literature suggests the model remains more of an aspiration than a reality.

METHODS AND ANALYSIS

To reach a fine understanding of the implementation of the concepts involved in LHS, we will use a hybrid method which combines concept analyses with systematic review methodology. We will use a two-step analysis, a content analysis to analyse the definitions, uses and attributes of the concept and a systematic review to analyse the concept's implementation mechanisms. We will search eight databases and grey literature and present a broad synthesis of the available evidence regarding design, implementation and evaluation of LHS in a multilevel perspective. We will follow the latest Preferred Reporting Items for Systematic Review and Meta-Analysis statement for conducting and reporting a systematic review. Two reviewers will independently screen the titles and abstracts against the eligibility criteria followed by full-text screening of potentially relevant articles for final inclusion decision. Conflicts will be resolved with a senior author. We will include published primary studies that use qualitative, quantitative or mixed methods. The assessment of risk of bias will be made using the Mixed-Methods Appraisal Tool.

ETHICS AND DISSEMINATION

This systematic review is exempt from ethics approval. The results formulated will highlight evidence-based interventions that support the implementation of a national LHS. They will be of particular interest to decision makers, researchers, managers, clinicians and patients allowing finally to implement the promising proposal of LHSs at national scale.

PROSPERO REGISTRATION NUMBER

CRD42023393565.

Infrastructure tools to support an effective Radiation Oncology Learning Health System

PURPOSE

Radiation Oncology Learning Health System (RO-LHS) is a promising approach to improve the quality of care by integrating clinical, dosimetry, treatment delivery, research data in real-time. This paper describes a novel set of tools to support the development of a RO-LHS and the current challenges they can address.

METHODS

We present a knowledge graph-based approach to map radiotherapy data from clinical databases to an ontology-based data repository using FAIR concepts. This strategy ensures that the data are easily discoverable, accessible, and can be used by other clinical decision support systems. It allows for visualization, presentation, and data analyses of valuable information to identify trends and patterns in patient outcomes. We designed a search engine that utilizes ontology-based keyword searching, synonym-based term matching that leverages the hierarchical nature of ontologies to retrieve patient records based on parent and children classes, connects to the Bioportal database for relevant clinical attributes retrieval. To identify similar patients, a method involving text corpus creation and vector embedding models (Word2Vec, Doc2Vec, GloVe, and FastText) are employed, using cosine similarity and distance metrics.

RESULTS

The data pipeline and tool were tested with 1660 patient clinical and dosimetry records resulting in 504 180 RDF (Resource Description Framework) tuples and visualized data relationships using graph-based representations. Patient similarity analysis using embedding models showed that the Word2Vec model had the highest mean cosine similarity, while the GloVe model exhibited more compact embeddings with lower Euclidean and Manhattan distances.

CONCLUSIONS

The framework and tools described support the development of a RO-LHS. By integrating diverse data sources and facilitating data discovery and analysis, they contribute to continuous learning and improvement in patient care. The tools enhance the quality of care by enabling the identification of cohorts, clinical decision support, and the development of clinical studies and machine learning programs in radiation oncology.

The use and role of digital technology in learning health systems: A scoping review

OBJECTIVE

The review aimed to identify which digital technologies are proposed or used within learning health systems (LHS) and to analyze the extent to which they support learning processes in LHS.

MATERIALS AND METHODS

Multiple databases and grey literature were searched with terms related to LHS. Manual searches and backward searches of reference lists were also undertaken. The review considered publications from 2007 to 2022. Records focusing on LHS, referring to one or more digital technologies, and describing how at least one digital technology could be used in LHS were included.

RESULTS

2046 records were screened for inclusion and 154 records were included in the analysis. Twenty categories of digital technology were identified. The two most common ones across records were data recording and processing and electronic health records. Digital technology was primarily leveraged to support data access and aggregation and data analysis, two of the seven recognized learning processes within LHS learning cycles.

DISCUSSION

The results of the review show that a wide array of digital technologies is being leveraged to support learning cycles within LHS. Nevertheless, an over-reliance on a narrow set of technologies supporting knowledge discovery, a lack of direct evaluation of digital technologies and ambiguity in technology descriptions are hindering the realization of the LHS vision.

CONCLUSION

Future LHS research and initiatives should aim to integrate digital technology to support practice change and impact evaluation. The use of recognized evaluation methods for health information technology and more detailed descriptions of proposed technologies are also recommended.

A Systematic Review on Navigation Programs for Persons Living With Dementia and Their Caregivers

BACKGROUND AND OBJECTIVES

System navigation programs are becoming more available to meet the needs of patients with complex care needs. The aim of this review was to systematically assess the outcomes of navigation programs for persons with dementia and their family caregivers.

RESEARCH DESIGN AND METHODS

A systematic review methodology was employed. Ten databases were searched for all relevant articles published until October 30, 2021. English-language full-text articles were included if they focused on implemented navigation program(s) that primarily supported persons with dementia who were aged 50 or older. Methodological quality was assessed by 2 independent raters using the Physiotherapy Evidence Database Scale, the STrengthening the Reporting of OBservational studies in Epidemiology checklist, and the Mixed Methods Appraisal Tool.

RESULTS

Fourteen articles were included in the review. There was Level 1 evidence for the benefits of system navigation programs on delaying institutionalization, wherein benefits appeared to be specific to interventions that had an in-person component. There was Level 1 (n = 4) and Level 3 (n = 1) evidence on service use from time of diagnosis to continued management of dementia. Finally, Level 1 to Level 5 evidence indicated a number of benefits on caregiver outcomes.

DISCUSSION AND IMPLICATIONS

There is strong evidence on the benefits of system navigation for people with dementia on delaying institutionalization and caregiver outcomes, but outcomes across other domains (i.e., functional independence) are less clear, which may be due to the varied approaches within system navigation models of care.

FeBRILe3: Safety Evaluation of Febrile Infant Guidelines Through Prospective Bayesian Monitoring

OBJECTIVES

Despite evidence supporting earlier discharge of well-appearing febrile infants at low risk of serious bacterial infection (SBI), admissions for ≥48 hours remain common. Prospective safety monitoring may support broader guideline implementation.

METHODS

A sequential Bayesian safety monitoring framework was used to evaluate a new hospital guideline recommending early discharge of low-risk infants. Hospital readmissions within 7 days of discharge were regularly assessed against safety thresholds, derived from historic rates and expert opinion, and specified a priori (8 per 100 infants). Infants aged under 3 months admitted to 2 Western Australian metropolitan hospitals for management of fever without source were enrolled (August 2019-December 2021), to a prespecified maximum 500 enrolments.

RESULTS

Readmission rates remained below the prespecified threshold at all scheduled analyses. Median corrected age was 34 days, and 14% met low-risk criteria (n = 71). SBI was diagnosed in 159 infants (32%), including urinary tract infection (n = 140) and bacteraemia (n = 18). Discharge occurred before 48 hours for 192 infants (38%), including 52% deemed low-risk. At study completion, 1 of 37 low-risk infants discharged before 48 hours had been readmitted (3%), for issues unrelated to SBI diagnosis. In total, 20 readmissions were identified (4 per 100 infants; 95% credible interval 3, 6), with >0.99 posterior probability of being below the prespecified noninferiority threshold, indicating acceptable safety.

CONCLUSIONS

A Bayesian monitoring approach supported safe early discharge for many infants, without increased risk of readmission. This framework may be used to embed safety evaluations within future guideline implementation programs to further reduce low-value care.

A New Treatment Program: Focused Integrated Team-based Treatment Program for Bipolar Disorder (FITT-BD)

OBJECTIVE

Bipolar disorder (BD) is complicated by a dynamic, chronic course along with multiple comorbid psychiatric and medical conditions, making it challenging for clinicians to treat and patients to thrive. To efficiently manage the complexity of BD and help patients recover, we developed a Focused Integrated Team-based Treatment Program for Bipolar Disorder (FITT-BD). The purpose of this paper is to describe how we developed this clinic and the lessons we learned.

METHODS

We developed FITT-BD by integrating strategies from stepped care, collaborative care, and learning health care systems. We describe the rationale, details, and lessons learned in developing FITT-BD.

RESULTS

By integrating stepped care, collaborative care, and a learning health care system approach, FITT-BD aims to reduce barriers to care, leverage the expertise of a multidisciplinary treatment team, ensure patient-centeredness, and use assessments to inform and continuously improve outcomes in real time. We learned that there are challenges in the creation of a web-based application that tracks the treatment of patients within a network of hospitals.

CONCLUSIONS

The success of FITT-BD will be determined by the degree to which it can increase treatment access, improve treatment adherence, and help individuals with BD achieve their treatment goals. We expect that FITT-BD will improve outcomes in the context of ongoing clinical care.

PUBLIC HEALTH SIGNIFICANCE

The treatment of BD is challenging and complex. We propose a new treatment model for BD: FITT-BD. We expect that this program will be a patient-centered approach that improves outcomes in the context of ongoing clinical care for patients with BD.

Climate Change, Environmental Health, and Challenges for Nursing Discipline

Current data and scientific predictions about the consequences of climate change are accurate in suggesting disaster. Since 2019, climate change has become a threat to human health, and major consequences on health and health systems are already observed. Climate change is a central concern for the nursing discipline, even though nursing theorists' understanding of the environment has led to problematic gaps that impact the current context. Today, nursing discipline is facing new challenges. Nurses are strategically placed to respond to the impacts of climate change through their practice, research, and training in developing, implementing, and sustaining innovation towards climate change mitigation and adaptation. It is urgent for them to adapt their practice to this reality to become agents of change.

Optimizing cardiovascular risk assessment and registration in a developing cardiovascular learning health care system: Women benefit most

Since 2015 we organized a uniform, structured collection of a fixed set of cardiovascular risk factors according the (inter)national guidelines on cardiovascular risk management. We evaluated the current state of a developing cardiovascular towards learning healthcare system-the Utrecht Cardiovascular Cohort Cardiovascular Risk Management (UCC-CVRM)-and its potential effect on guideline adherence in cardiovascular risk management. We conducted a before-after study comparing data from patients included in UCC-CVRM (2015-2018) and patients treated in our center before UCC-CVRM (2013-2015) who would have been eligible for UCC-CVRM using the Utrecht Patient Oriented Database (UPOD). Proportions of cardiovascular risk factor measurement before and after UCC-CVRM initiation were compared, as were proportions of patients that required (change of) blood pressure, lipid, or blood glucose lowering treatment. We estimated the likelihood to miss patients with hypertension, dyslipidemia, and elevated HbA1c before UCC-CVRM for the whole cohort and stratified for sex. In the present study, patients included up to October 2018 (n = 1904) were matched with 7195 UPOD patients with similar age, sex, department of referral and diagnose description. Completeness of risk factor measurement increased, ranging from 0% -77% before to 82%-94% after UCC-CVRM initiation. Before UCC-CVRM, we found more unmeasured risk factors in women compared to men. This sex-gap resolved in UCC-CVRM. The likelihood to miss hypertension, dyslipidemia, and elevated HbA1c was reduced by 67%, 75% and 90%, respectively, after UCC-CVRM initiation. A finding more pronounced in women compared to men. In conclusion, a systematic registration of the cardiovascular risk profile substantially improves guideline adherent assessment and decreases the risk of missing patients with elevated levels with an indication for treatment. The sex-gap disappeared after UCC-CVRM initiation. Thus, an LHS approach contributes to a more inclusive insight into quality of care and prevention of cardiovascular disease (progression).

Building a National Clinical Trials Network in child and youth mental health: Growing Minds Australia

Many fields of medicine have benefitted from the formation of clinical trials networks, whereby researchers come together on a large scale to identify high-priority questions and implement coordinated clinical trials. Clinical trials networks in the field of mental health, however, have been rare and largely absent from the Australian context. Here, we present an overview of the newly formed Growing Minds Australia Clinical Trials Network, which represents the first comprehensive clinical trials network in child and youth mental health in Australia. The 60 principal members of the Growing Minds Australia Clinical Trials Network represent teams across 19 diverse areas related to specific forms of psychopathology (e.g. internalising, externalising, neurodevelopmental disorders, early psychosis, substance use), specific research methods and processes (e.g. health economics, eHealth, implementation science) and specialised areas of practice (e.g. school-based systems, parenting interventions, Indigenous mental health, refugee families). Core functions of the Growing Minds Australia Clinical Trials Network include collaborative trial protocol development; peer review, prioritisation and endorsement of proposed trials; training; development of clinical guidelines; and consumer representation. The research by the clinical trials network will encompass the populations typically accessing youth mental health services, while placing a key emphasis on the early periods of life, and the role of parents and caregivers as critical partners in the co-design of research and the delivery of intervention and prevention strategies. The structures and processes built into the network are designed to coordinate collaboration between diverse stakeholders and ensure that provisions for translation are integrated into research from the outset. In this paper, we examine the potential for a dedicated clinical trials network to initiate fundamental improvement in child and youth mental health systems, and discuss the unique and complex challenges associated with establishing such an initiative.

Conceptualizing learning health systems: A mapping review

Introduction

Health systems worldwide face the challenge of increasing population health with high-quality care and reducing health care expenditure growth. In pursuit for a solution, regional cross-sectoral partnerships aim to reorganize and integrate services across public health, health care and social care. Although the complexity of regional partnerships demands an incremental strategy, it is yet not known how learning works within these partnerships. To understand learning in regional cross-sectoral partnerships for health, this study aims to map the concept Learning Health System (LHS).

Methods

This mapping review used a qualitative text analysis approach. A literature search was conducted in Embase and was limited to English-language papers published in the period 2015-2020. Title-abstract screening was performed using established exclusion criteria. During full-text screening, we combined deductive and inductive coding. The concept LHS was disentangled into aims, design elements, and process of learning. Data extraction and analysis were performed in MAX QDA 2020.

Results

In total, 155 articles were included. All articles used the LHS definition of the Institute of Medicine. The interpretation of the concept LHS varied widely. The description of LHS contained 25 highly connected aims. In addition, we identified nine design elements. Most elements were described similarly, only the interpretation of stakeholders, data infrastructure and data varied. Furthermore, we identified three types of learning: learning as 1) interaction between clinical practice and research; 2) a circular process of converting routine care data to knowledge, knowledge to performance; and performance to data; and 3) recurrent interaction between stakeholders to identify opportunities for change, to reveal underlying values, and to evaluate processes. Typology 3 was underrepresented, and the three types of learning rarely occurred simultaneously.

Conclusion

To understand learning within regional cross-sectoral partnerships for health, we suggest to specify LHS-aim(s), operationalize design elements, and choose deliberately appropriate learning type(s).

The Alliance for Healthier Communities as a Learning Health System for primary care: A qualitative analysis in Ontario, Canada

RATIONALE, AIMS AND OBJECTIVES

A learning health system model can be used to efficiently evaluate and incorporate evidence-based care into practice. However, there is a paucity of evidence describing key organizational attributes needed to ensure a successful learning health system within primary care. We interviewed stakeholders for a primary care learning health system in Ontario, Canada (the Alliance for Healthier Communities) to identify strengths and areas for improvement.

METHOD

We conducted a qualitative descriptive study using individual semistructured interviews with Alliance stakeholders between December 2019 and March 2020. The Alliance delivers community-governed primary healthcare through 109 organizations including Community Health Centres (CHCs). All CHC staff within the Alliance were invited to participate. Interviews were audio-recorded and transcribed verbatim. We performed a thematic analysis using a team approach.

RESULTS

We interviewed 29 participants across six CHCs, including Executive Directors, managers, healthcare providers and data support staff. We observed three foundational elements necessary for a successful learning health system within primary care: shared organizational goals and culture, data quality and resources. Building on this foundation, people are needed to drive the learning health system, and this is conditional on their level of engagement. The main factors motivating staff member's engagement with the learning health system included their drive to help improve patient care, focusing on initiatives of personal interest and understanding the purpose of different initiatives. Areas for improvement were identified such as the ability to extract and use data to inform changes in real-time, better engagement and protected time for providers to do improvement work, and more staff dedicated to data extraction and analysis.

CONCLUSIONS

We identified key components needed to establish a learning health system in primary care. Similar primary care organizations in Canada and elsewhere can use these insights to guide their development as learning health systems.

“We cobble together a storyline of system performance using a diversity of things”: a qualitative study of perspectives on public health performance measurement in Canada

Background

There have been longstanding calls for public health systems transformations in many countries, including Canada. Core to these calls has been strengthening performance measurement. While advancements have been made in performance measurement for certain sectors of the health care system (primarily focused on acute and primary health care), effective use of indicators for measuring public health systems performance are lacking. This study describes the current state, anticipated challenges, and future directions in the development and implementation of a public health performance measurement system for Canada.

Methods

We conducted a qualitative study using semi-structured interviews with public health leaders (n = 9) between July and August 2021. Public health leaders included researchers, government staff, and former medical officers of health who were purposively selected due to their expertise and experience with performance measurement with relevance to public health systems in Canada. Thematic analysis included both a deductive approach for themes consistent with the conceptual framework and an inductive approach to allow new themes to emerge from the data.

Results

Conceptual, methodological, contextual, and infrastructure challenges were highlighted by participants in designing a performance measurement system for public health. Specifically, six major themes evolved that encompass 1) the mission and purpose of public health systems, including challenges inherent in measuring the functions and services of public health; 2) the macro context, including the impacts of chronic underinvestment and one-time funding injections on the ability to sustain a measurement system; 3) the organizational structure/governance of public health systems including multiple forms across Canada and underdevelopment of information technology systems; 4) accountability approaches to performance measurement and management; and 5) timing and unobservability in public health indicators. These challenges require dedicated investment, strong leadership, and political will from the federal and provincial/territorial governments.

Conclusion

Unprecedented attention on public health due to the coronavirus disease 2019 pandemic has highlighted opportunities for system improvements, such as addressing the lack of a performance measurement system. This study provides actionable knowledge on conceptual, methodological, contextual, and infrastructure challenges needed to design and build a pan-Canadian performance measurement system for public health.

A conceptual framework to describe and evaluate a socially accountable learning health system: Development and application in a northern, rural, and remote setting

Health care and academic institutions are increasingly committing to social accountability, a strategic shift that requires priorities, activities, and evaluations to be co-determined with all relevant partners. Consequently, governments, accreditors, funders, and communities are calling for these institutions to communicate their progress towards social accountability. The purpose of this study was to develop a conceptual framework around a socially accountable learning health system. This article presents an integrated analysis of two studies: (i) a narrative review of 11 prominent social accountability and health services conceptual frameworks and (ii) a reflexive thematic analysis of 18 key informant interviews. Using a systematic conceptual framework development and integrated theory of change/realist evaluation methodologies, we describe a synthesis of these findings to develop a conceptual framework for describing and evaluating socially accountable health professional education. The resulting framework describes assessment phases of social accountability, transitions between phases, learning cycles, and the actors and systems that collectively mobilise social accountability at multiple levels in health and education systems. The framework can be used to evaluate interventions or characterise progress towards social accountability in different settings, as illustrated in the example at the end of the paper. The framework emphasises the significance of designing, mobilising, and evaluating social accountability as part of a contextualised learning health system.

Completion of an Enhanced Recovery Program in a Pediatric Ambulatory Surgery Center: A Quality Improvement Initiative

BACKGROUND

Enhanced Recovery After Surgery (ERAS) was first established in 2001 focusing on recovery from complex surgical procedures in adults and recently expanded to ambulatory surgery. The evidence for ERAS in children is limited. In 2018, recognized experts began developing needed pediatric evidence. Center-wide efforts involving all ambulatory surgical patients and procedures have not previously been described.

METHODS

A comprehensive assessment and gap analysis of ERAS elements in our ambulatory center identified 11 of 19 existing elements. The leadership committed to implementing an Enhanced Recovery Program (ERP) to improve existing elements and close as many remaining gaps as possible. A quality improvement (QI) team was launched to improve 5 existing ERP elements and to introduce 6 new elements (target 17/19 ERP elements). The project plan was broken into 1 preparation phase to collect baseline data and 3 implementation phases to enhance existing and implement new elements. Statistical process control methodology was used. Team countermeasures were based on available evidence. A consensus process was used to resolve disagreement. Monthly meetings were held to share real-time data, gather new feedback, and modify countermeasure plans as needed. The primary outcome measure selected was mean postanesthesia care unit (PACU) length of stay (LOS). Secondary outcomes measures were mean maximum pain score in PACU and patient/family satisfaction scores.

RESULTS

The team had expanded the pool of active ERP elements from 11 to 16 of 19. The mean PACU LOS demonstrated significant reduction (early in phase 1 and again in phase 3). No change was seen for the mean maximum pain score in PACU or surgical complication rates. Patient/family satisfaction scores were high and sustained throughout the period of study (91.1% ± 5.7%). Patient/family and provider engagement/compliance were high.

CONCLUSIONS

This QI project demonstrated the feasibility of pediatric ERP in an ambulatory surgical setting. Furthermore, a center-wide approach was shown to be possible. Additional studies are needed to determine the relevance of this project to other institutions.

Pharmacy Workplace Wellbeing and Resilience: Themes Identified from a Hermeneutic Phenomenological Analysis with Future Recommendations

This study applied a hermeneutic phenomenological approach to better understand pharmacy workplace wellbeing and resilience using respondents' written comments along with a blend of the researchers' understanding of the phenomenon and the published literature. Our goal was to apply this understanding to recommendations for the pharmacy workforce and corresponding future research. Data were obtained from the 2021 APhA/NASPA National State-Based Pharmacy Workplace Survey, launched in the United States in April 2021. Promotion of the online survey to pharmacy personnel was accomplished through social media, email, and online periodicals. Responses continued to be received through the end of 2021. A data file containing 6973 responses was downloaded on 7 January 2022 for analysis. Usable responses were from those who wrote an in-depth comment detailing stories and experiences related to pharmacy workplace and resilience. There were 614 respondents who wrote such comments. The findings revealed that business models driven by mechanized assembly line processes, business metrics that supersede patient outcomes, and reduction of pharmacy personnel's professional judgement have contributed to the decline in the experience of providing patient care in today's health systems. The portrait of respondents' lived experiences regarding pharmacy workplace wellbeing and resilience was beyond the individual level and revealed the need for systems change. We propose several areas for expanded inquiry in this domain: (1) shared trauma, (2) professional responsibility and autonomy, (3) learned subjection, (4) moral injury and moral distress, (5) sociocultural effects, and (6) health systems change.

Which strategies support the effective use of clinical practice guidelines and clinical quality registry data to inform health service delivery? A systematic review

BACKGROUND

Empirical evidence suggests data and insights from the clinical practice guidelines and clinical quality registries are not being fully utilised, leaving health service managers, clinicians and providers without clear guidance on how best to improve healthcare delivery. This lack of uptake of existing research knowledge represents low value to the healthcare system and needs to change.

METHODS

Five electronic databases (MEDLINE, Embase, CINAHL, Cochrane Central and Cochrane Database of Systematic Reviews) were systematically searched. Included studies were published between 2000 and 2020 reporting on the attributes, evidence usage and impact of clinical practice guidelines and clinical quality registries on health service delivery.

RESULTS

Twenty-six articles including one randomised controlled trial, eight before-and-after studies, eight case studies/reviews, five surveys and four interview studies, covering a wide range of medical conditions and conducted in the USA, Australia and Europe, were identified. Five complementary strategies were derived to maximise the likelihood of best practice health service delivery: (1) feedback and transparency, (2) intervention sustainability, (3) clinical practice guideline adherence, (4) productive partnerships and (5) whole-of-team approach.

CONCLUSION

These five strategies, used in context-relevant combinations, are most likely to support the application of existing high-quality data, adding value to health service delivery. The review highlighted the limitations of study design in opportunistic registry studies that do not produce clear, usable evidence to guide changes to health service implementation practices. Recommendations include exploration of innovative methodologies, improved coordination of national registries and the use of incentives to encourage guideline adherence and wider dissemination of strategies used by successful registries.

Conceptualising learning healthcare systems and organisations in the context of rehabilitation: a scoping review protocol

Background: Transformative system wide action is needed for healthcare systems to meet the needs of an increasing aging population and changing health needs. One idea is that health systems can become “learning organisations” (LO) or “learning healthcare systems” (LHS) that continuously generate and apply evidence, innovation, quality, and value to provide better care. This is of value to non-acute healthcare settings such as rehabilitation, which are complex, multi-dimensional and multi-disciplinary in nature. Little is known about how these frameworks have been applied to rehabilitation settings. Objective and inclusion criteria: The aim of this scoping review is to systematically map and summarise the literature conceptualising and operationalising LHS and LO in rehabilitation settings. Studies will be included which define a LO or LHS; or describe an operating LHS/LO; or include the translation of research evidence generated from LHS/LO data into healthcare improvement within a rehabilitation context will be included. Study designs such as quantitative, qualitative, mixed method studies, and case studies will be included. Methods: The guidelines from the Joanna Briggs institute methodology for scoping reviews will be used for this review. The literature search will be performed using a three-step search strategy: an initial limited search of two databases has been performed to identify relevant key words and index terms. The developed search string will be adapted and applied across the following databases: OVID MEDLINE, EMBASE, CINAHL Plus, APA PsycINFO and COCHRANE Database of Systematic Reviews. This will be followed by search of the reference lists of selected sources and relevant data-hubs. A draft data extraction framework will be used and updated iteratively to extract data. Frequency counts and qualitative content analysis will be employed to address the research question of how LHS and LO have been conceptualised and operationalised in the context of rehabilitation.

Evaluating a learning health system initiative: Lessons learned during COVID-19 in Saskatchewan, Canada

Introduction

Evaluating a learning health system (LHS) encourages continuous system improvement and collaboration within the healthcare system. Although LHS is a widely accepted concept, there is little knowledge about evaluating an LHS. To explore the outputs and outcomes of an LHS model, we evaluated the COVID-19 Evidence Support Team (CEST) in Saskatchewan, Canada, an initiative to rapidly review scientific evidence about COVID-19 for decision-making. By evaluating this program during its formation, we explored how and to what extent the CEST initiative was used by stakeholders. An additional study aim was to understand how CEST could be applied as a functional LHS and the value of similar knowledge-to-action cycles.

Methods

Using a formative evaluation design, we conducted qualitative interviews with key informants (KIs) who were involved with COVID-19 response strategies in Saskatchewan. Transcripts were analyzed using reflexive thematic analysis to identify key themes. A program logic model was created to represent the inputs, activities, outputs, and outcomes of the CEST initiative.

Results

Interview data from 11 KIs were collated under three overarching categories: (1) outputs, (2) short-term outcomes, and (3) long-term outcomes from the CEST initiative. Overall, participants found the CEST initiative improved speed and access to reliable information, supported and influenced decision-making and public health strategies, leveraged partnerships, increased confidence and reassurance, and challenged misinformation. Themes relating to the long-term outcomes of the initiative included improving coordination, awareness, and using good judgment and planning to integrate CEST sustainably into the health system.

Conclusion

This formative evaluation demonstrated that CEST was a valued program and a promising LHS model for Saskatchewan. The future direction involves addressing program recommendations to implement this model as a functional LHS in Saskatchewan.

Mapping continuous learning using social network research: a social network study of Australian Genomics as a Learning Health System

OBJECTIVES

To explore a macrolevel Learning Health System (LHS) and examine if an intentionally designed network can foster a collaborative learning community over time. The secondary aim was to demonstrate the application of social network research to the field of LHS.

DESIGN

Two longitudinal online questionnaires of the Australian Genomics learning community considering relationships between network members at three time points: 2016, 2018, 2019. The questionnaire included closed Likert response questions on collaborative learning patterns and open-response questions to capture general perceptions of the community. Social network data were analysed and visually constructed using Gephi V.0.9.2 software, Likert questions were analysed using SPSS, and open responses were analysed thematically using NVivo.

SETTING

Australian Genomic Health Alliance.

PARTICIPANTS

Clinicians, scientists, researchers and community representatives.

RESULTS

Australian Genomics members highlighted the collaborative benefits of the network as a learning community to foster continuous learning in the ever-evolving field of clinical genomics. The learning community grew from 186 members (2016), to 384 (2018), to 439 (2019). Network density increased (2016=0.023, 2018=0.043), then decreased (2019=0.036). Key players remained consistent with potential for new members to achieve focal positions in the network. Informal learning was identified as the most influential learning method for genomic practice.

CONCLUSIONS

This study shows that intentionally building a network provides a platform for continuous learning-a fundamental component for establishing an LHS. The Australian Genomics learning community shows evidence of maturity and sustainability in supporting the continuous learning culture of clinical genomics. The network provides a practical means to spread new knowledge and best practice across the entire field. We show that intentionally designed networks provide the opportunity and means for interdisciplinary learning between diverse agents over time and demonstrate the application of social network research to the LHS field.

A replicable, open-source, data integration method to support national practice-based research & quality improvement systems

OBJECTIVES

The Epilepsy Learning Healthcare System (ELHS) was created in 2018 to address measurable improvements in outcomes for people with epilepsy. However, fragmentation of data systems has been a major barrier for reporting and participation. In this study, we aimed to test the feasibility of an open-source Data Integration (DI) method that connects real-life clinical data to national research and quality improvement (QI) systems.

METHODS

The ELHS case report forms were programmed as EPIC SmartPhrases at Mass General Brigham (MGB) in December 2018 and subsequently as EPIC SmartForms in June 2021 to collect actionable, standardized, structured epilepsy data in the electronic health record (EHR) for subsequent pull into the external national registry of the ELHS. Following the QI methodology in the Chronic Care Model, 39 providers, epileptologists and neurologists, incorporated the ELHS SmartPhrase into their clinical workflow, focusing on collecting diagnosis of epilepsy, seizure type according to the International League Against Epilepsy, seizure frequency, date of last seizure, medication adherence and side effects. The collected data was stored in the Enterprise Data Warehouse (EDW) without integration with external systems. We developed and validated a DI method that extracted the data from EDW using structured query language and later preprocessed using text mining. We used the ELHS data dictionary to match fields in the preprocessed notes to obtain the final structured dataset with seizure control information. For illustration, we described the data curated from the care period of 12/2018-12/2021.

RESULTS

The cohort comprised a total of 1806 patients with a mean age of 43 years old (SD: 17.0), where 57% were female, 80% were white, and 84% were non-Hispanic/Latino. Using our DI method, we automated the data mining, preprocessing, and exporting of the structured dataset into a local database, to be weekly accessible to clinicians and quality improvers. During the period of SmartPhrase implementation, there were 5168 clinic visits logged by providers documenting each patient's seizure type and frequency. During this period, providers documented 59% patients having focal seizures, 35% having generalized seizures and 6% patients having another type. Of the cohort, 45% patients had private insurance. The resulting structured dataset was bulk uploaded via web interface into the external national registry of the ELHS.

CONCLUSIONS

Structured data can be feasibly extracted from text notes of epilepsy patients for weekly reporting to a national learning healthcare system.

Moving towards vertically integrated artificial intelligence development

Substantial interest and investment in clinical artificial intelligence (AI) research has not resulted in widespread translation to deployed AI solutions. Current attention has focused on bias and explainability in AI algorithm development, external validity and model generalisability, and lack of equity and representation in existing data. While of great importance, these considerations also reflect a model-centric approach seen in published clinical AI research, which focuses on optimising architecture and performance of an AI model on best available datasets. However, even robustly built models using state-of-the-art algorithms may fail once tested in realistic environments due to unpredictability of real-world conditions, out-of-dataset scenarios, characteristics of deployment infrastructure, and lack of added value to clinical workflows relative to cost and potential clinical risks. In this perspective, we define a vertically integrated approach to AI development that incorporates early, cross-disciplinary, consideration of impact evaluation, data lifecycles, and AI production, and explore its implementation in two contrasting AI development pipelines: a scalable "AI factory" (Mayo Clinic, Rochester, United States), and an end-to-end cervical cancer screening platform for resource poor settings (Paps AI, Mbarara, Uganda). We provide practical recommendations for implementers, and discuss future challenges and novel approaches (including a decentralised federated architecture being developed in the NHS (AI4VBH, London, UK)). Growth in global clinical AI research continues unabated, and introduction of vertically integrated teams and development practices can increase the translational potential of future clinical AI projects.

Learning health systems in low-income and middle-income countries: exploring evidence and expert insights

Introduction

Learning health systems (LHS) is a multifaceted subject. This paper reviewed current concepts as well as real-world experiences of LHS, drawing on published and unpublished knowledge in order to identify and describe important principles and practices that characterise LHS in low/middle-income country (LMIC) settings.

Methods

We adopted an exploratory approach to the literature review, recognising there are limited studies that focus specifically on system-wide learning in LMICs, but a vast set of connected bodies of literature. 116 studies were included, drawn from an electronic literature search of published and grey literature. In addition, 17 interviews were conducted with health policy and research experts to gain experiential knowledge.

Results

The findings were structured by eight domains on learning enablers. All of these interact with one another and influence actors from community to international levels.

We found that learning comes from the connection between information, deliberation, and action. Moreover, these processes occur at different levels. It is therefore important to consider experiential knowledge from multiple levels and experiences. Creating spaces and providing resources for communities, staff and managers to deliberate on their challenges and find solutions has political implications, however, and is challenging, particularly when resources are constrained, funding and accountability are fragmented and the focus is short-term and narrow. Nevertheless, we can learn from countries that have managed to develop institutional mechanisms and human capacities which help health systems respond to changing environments with ‘best fit’ solutions.

Conclusion

Health systems are knowledge producers, but learning is not automatic. It needs to be valued and facilitated. Everyday governance of health systems can create spaces for reflective practice and learning within routine processes at different levels. This article highlights important enablers, but there remains much work to be done on developing this field of knowledge.

Identifying transient ischemic attack (TIA) patients at high-risk of adverse outcomes: development and validation of an approach using electronic health record data

Background

Risk-stratification tools that have been developed to identify transient ischemic attack (TIA) patients at risk of recurrent vascular events typically include factors which are not readily available in electronic health record systems. Our objective was to evaluate two TIA risk stratification approaches using electronic health record data.

Methods

Patients with TIA who were cared for in Department of Veterans Affairs hospitals (October 2015—September 2018) were included. The six outcomes were mortality, recurrent ischemic stroke, and the combined endpoint of stroke or death at 90-days and 1-year post-index TIA event. The cohort was split into development and validation samples. We examined the risk stratification of two scores constructed using electronic health record data. The Clinical Assessment Needs (CAN) score is a validated measure of risk of hospitalization or death. The PREVENT score was developed specifically for TIA risk stratification.

Results

A total of N = 5250 TIA patients were included in the derivation sample and N = 4248 in the validation sample. The PREVENT score had higher c-statistics than the CAN score across all outcomes in both samples. Within the validation sample the c-statistics for the PREVENT score were: 0.847 for 90-day mortality, 0.814 for 1-year mortality, 0.665 for 90-day stroke, and 0.653 for 1-year stroke, 0.699 for 90-day stroke or death, and 0.744 for 1-year stroke or death. The PREVENT score classified patients into categories with extreme nadir and zenith outcome rates. The observed 1-year mortality rate among validation patients was 7.1%; the PREVENT score lowest decile of patients had 0% mortality and the highest decile group had 30.4% mortality.

Conclusions

The PREVENT score had strong c-statistics for the mortality outcomes and classified patients into distinct risk categories. Learning healthcare systems could implement TIA risk stratification tools within electronic health records to support ongoing quality improvement.

Registration

ClinicalTrials.gov Identifier: NCT02769338.

The Perils of a "My Work Here is Done" perspective: a mixed methods evaluation of sustainment of an evidence-based intervention for transient ischemic attack

BACKGROUND

To evaluate quality improvement sustainment for Transient Ischemic Attack (TIA) and identify factors influencing sustainment, which is a challenge for Learning Healthcare Systems.

METHODS

Mixed methods were used to assess changes in care quality across periods (baseline, implementation, sustainment) and identify factors promoting or hindering sustainment of care quality. PREVENT was a stepped-wedge trial at six US Department of Veterans Affairs implementation sites and 36 control sites (August 2015-September 2019). Quality of care was measured by the without-fail rate: proportion of TIA patients who received all of the care for which they were eligible among brain imaging, carotid artery imaging, neurology consultation, hypertension control, anticoagulation for atrial fibrillation, antithrombotics, and high/moderate potency statins. Key informant interviews were used to identify factors associated with sustainment.

RESULTS

The without-fail rate at PREVENT sites improved from 36.7% (baseline, 58/158) to 54.0% (implementation, 95/176) and settled at 48.3% (sustainment, 56/116). At control sites, the without-fail rate improved from 38.6% (baseline, 345/893) to 41.8% (implementation, 363/869) and remained at 43.0% (sustainment, 293/681). After adjustment, no statistically significant difference in sustainment quality between intervention and control sites was identified. Among PREVENT facilities, the without-fail rate improved ≥2% at 3 sites, declined ≥2% at two sites, and remained unchanged at one site during sustainment. Factors promoting sustainment were planning, motivation to sustain, integration of processes into routine practice, leadership engagement, and establishing systems for reflecting and evaluating on performance data. The only factor that was sufficient for improving quality of care during sustainment was the presence of a champion with plans for sustainment. Challenges during sustainment included competing demands, low volume, and potential problems with medical coding impairing use of performance data. Four factors were sufficient for declining quality of care during sustainment: low motivation, champion inactivity, no reflecting and evaluating on performance data, and absence of leadership engagement.

CONCLUSIONS

Although the intervention improved care quality during implementation; performance during sustainment was heterogeneous across intervention sites and not different from control sites. Learning Healthcare Systems seeking to sustain evidence-based practices should embed processes within routine care and establish systems for reviewing and reflecting upon performance.

TRIAL REGISTRATION

Clinicaltrials.gov ( NCT02769338 ).

From "Invented here" to "Use it everywhere!": A Learning health system from bottom and/or top?

Introduction

Departing from a practical problem of how to use digitalization to improve care quality and efficiency, this paper investigates how the concept of Learning Health Systems (LHSs) can be applied to an existing organization. LHSs offer a vision for how healthcare can accelerate both scale-up of innovations and quality improvements at all levels. However, aligning stakeholders at different levels to convergent development is challenging and translation and adaptation of the LHS concept to fit with the existing organization is essential.

Methods

A one-year longitudinal action research (AR) study was conducted within five psychiatric departments at the Sahlgrenska University Hospital in Gothenburg, Sweden. Translation of the LHS concept to the local circumstances within the organization was set as the aim, to both improve practice and further scientific understanding. An AR group led the practical and scholarly work and holistic data were collected, including field notes, documents, recordings, and workshops. Data were analyzed by an insider-outsider approach.

Results

The one-year study is described to provide insights into the process of designing a locally adapted LHS using an AR approach. Practical needs were identified and iteratively matched with theory to form a local LHS model. A conflict between top-down and bottom-up views on development emerged, where higher-level management tended to prioritize uniform solutions and developers local learning. An adapted solution to balance these approaches was negotiated, consisting of a technical and an organizational part.

Conclusions

The conflict between top-down and bottom-up approaches for how to implement LHSs needs to be considered both in practical work to transform care organizations and in scientific studies of LHSs. The approach to translate, rather than instrumentally implement, LHSs to real-world settings is suggested as advantageous. Furthermore, designing such endeavors as AR projects can provide excellent conditions to create LHSs that work in practice.

The SHOnet learning health system: Infrastructure for continuous learning in pediatric rehabilitation

Introduction

To describe the development and implementation of learning health system (LHS) infrastructure for a pediatric specialty care health system to support LHS research in pediatric rehabilitation settings.

Methods

An existing pediatric common data model (eg, PEDSnet) of standardized medical terminologies for research was expanded and leveraged for this stud, and applied to SHOnet, a clinical research data resource consisting of deidentified data extracted from the electronic health record (EHR) from the Shriners Hospitals for Children speacialty pediatric health care system. We mapped EHR data for laboratory, procedures, drugs, and conditions to standardized vocabularies including ICD-10, CPT, RxNorm, and LOINC to the common data model using an established extraction-transformation-loading process. Rigorous quality checks were conducted to ensure a high degree of data conformance, completeness, and plausibility. SHOnet data elements from all sources are de-identified and the server is managed by the SHC Information Systems Department. SHOnet data are refreshed monthly and data elements are continually expanded based on new research endeavors.

Interventions

Not applicable.

Results

The Shriners Health Outcomes Network (SHOnet) includes data for over 10 000 distinct observational data elements based on over two million patient encounters between 2011 and present.

Conclusion

The systematic process to develop SHOnet is replicable and flexible for other pediatric rehabilitation research settings interested in building out their LHS capabilities. Challenges and facilitators may arise for building such LHS infrastructure for rehabilitation in areas of (a) data capture, curation, query, and governance, (b) generating knowledge from data, and (c) dissemination and implementation of new institutional knowledge. Further research studies are needed to evaluate these data resources for scalable system-learning endeavors.SHOnet is an exemplar of an LHS for rehabilitation and specialty care settings. The success of an LHS is dependent on engagement of multiple stakeholders, shared governance, effective knowledge translation, and deep commitment to long-term strategies for engaging clinicians, administration, and families in leveraging knowledge to improve clinical outcomes.

Health system learning with Indigenous communities: a study protocol for a two-eyed seeing review and multiple case study

Background

It is well documented that Canadian healthcare does not fully meet the health needs of First Nations, Inuit or Métis peoples. In 1996, the Royal Commission on Aboriginal Peoples concluded that Indigenous peoples’ healthcare needs had to be met by strategies and systems that emerged from Indigenous worldviews and cultures. In 2015, the Truth and Reconciliation Commission also called on health organizations to learn from Indigenous “knowledges” and integrate Indigenous worldviews alongside biomedicine and other western ways of knowing. These calls have not yet been met. Meanwhile, the dynamic of organizational learning from knowledges and evidence within communities is poorly understood—particularly when learning is from communities whose ways of knowing differ from those of the organization. Through an exploration of organizational and health system learning, this study will explore how organizations learn from the Indigenous communities they serve and contribute to (re-)conceptualizing the learning organization and learning health system in a way that privileges Indigenous knowledges and ways of knowing.

Methods

This study will employ a two-eyed seeing literature review and embedded multiple case study. The review, based on Indigenous and western approaches to reviewing and synthesizing knowledges, will inform understanding of health system learning from different ways of knowing. The multiple case study will examine learning by three distinct government organizations in Northwest Territories, a jurisdiction in northern Canada, that have roles to support community health and wellness: Tłı̨chǫ Government, Gwich’in Tribal Council, and Government of Northwest Territories. Case study data will be collected via interviews, talking circles, and document analysis. A steering group, comprising Tłı̨chǫ and Gwich’in Elders and representatives from each of the three partner organizations, will guide all aspects of the project.

Discussion

Examining systems that create health disparities is an imperative for Canadian healthcare. In response, this study will help to identify and understand ways for organizations to learn from and respectfully apply knowledges and evidence held within Indigenous communities so that their health and wellness are supported. In this way, this study will help to guide health organizations in the listening and learning that is required to contribute to reconciliation in healthcare.

Learning and Predicting from Dynamic Models for COVID-19 Patient Monitoring

COVID-19 has challenged health systems to learn how to learn. This paper describes the context, methods and challenges for learning to improve COVID-19 care at one academic health center. Challenges to learning include: (1) choosing a right clinical target; (2) designing methods for accurate predictions by borrowing strength from prior patients' experiences; (3) communicating the methodology to clinicians so they understand and trust it; (4) communicating the predictions to the patient at the moment of clinical decision; and (5) continuously evaluating and revising the methods so they adapt to changing patients and clinical demands. To illustrate these challenges, this paper contrasts two statistical modeling approaches - prospective longitudinal models in common use and retrospective analogues complementary in the COVID-19 context - for predicting future biomarker trajectories and major clinical events. The methods are applied to and validated on a cohort of 1,678 patients who were hospitalized with COVID-19 during the early months of the pandemic. We emphasize graphical tools to promote physician learning and inform clinical decision making.

Clarifying the concept of a learning health system for healthcare delivery organizations: Implications from a qualitative analysis of the scientific literature

The "learning health system" (LHS) concept has been defined in broad terms, which makes it challenging for health system leaders to determine exactly what is required to transform their organization into an LHS. This study provides a conceptual map of the LHS landscape by identifying the activities, principles, tools, and conditions that LHS researchers have associated with the concept. Through a multi-step screening process, two researchers identified 79 publications from PubMed (published before January 2020) that contained information relevant to the question, "What work is required of a healthcare organization that is operating as an LHS?" Those publications were coded as to whether or not they referenced each of 94 LHS elements in the taxonomy developed by the study team. This taxonomy, named the Learning Health Systems Consolidated Framework (LHS-CF), organizes the elements into five "bodies of work" (organizational learning, translation of evidence into practice, building knowledge, analyzing clinical data, and engaging stakeholders) and four "enabling conditions" (workforce skilled for LHS work, data systems and informatics technology in place, organization invests resources in LHS work, and supportive organizational culture). We report the frequency that each of the 94 elements was referenced across the 79 publications. The four most referenced elements were: "organization builds knowledge or evidence," "quality improvement practices are standard practice," "patients and family members are actively engaged," and "organizational culture emphasizes and supports learning." By dissecting the LHS construct into its component elements, the LHS-CF taxonomy can serve as a useful tool for LHS researchers and practitioners in defining the aspects of LHS they are addressing. By assessing how often each element is referenced in the literature, the study provides guidance to health system leaders as to how their organization needs to evolve in order to become an LHS - while also recognizing that each organization should emphasize elements that are most aligned with their mission and goals.

Use of Clinical Data Interchange Standards Consortium (CDISC) Standards for Real-world Data: Expert Perspectives From a Qualitative Delphi Survey

Background

Real-world data (RWD) and real-world evidence (RWE) are playing increasingly important roles in clinical research and health care decision-making. To leverage RWD and generate reliable RWE, data should be well defined and structured in a way that is semantically interoperable and consistent across stakeholders. The adoption of data standards is one of the cornerstones supporting high-quality evidence for the development of clinical medicine and therapeutics. Clinical Data Interchange Standards Consortium (CDISC) data standards are mature, globally recognized, and heavily used by the pharmaceutical industry for regulatory submissions. The CDISC RWD Connect Initiative aims to better understand the barriers to implementing CDISC standards for RWD and to identify the tools and guidance needed to more easily implement them.

Objective

The aim of this study is to understand the barriers to implementing CDISC standards for RWD and to identify the tools and guidance that may be needed to implement CDISC standards more easily for this purpose.

Methods

We conducted a qualitative Delphi survey involving an expert advisory board with multiple key stakeholders, with 3 rounds of input and review.

Results

Overall, 66 experts participated in round 1, 56 in round 2, and 49 in round 3 of the Delphi survey. Their inputs were collected and analyzed, culminating in group statements. It was widely agreed that the standardization of RWD is highly necessary, and the primary focus should be on its ability to improve data sharing and the quality of RWE. The priorities for RWD standardization included electronic health records, such as data shared using Health Level 7 Fast Health care Interoperability Resources (FHIR), and the data stemming from observational studies. With different standardization efforts already underway in these areas, a gap analysis should be performed to identify the areas where synergies and efficiencies are possible and then collaborate with stakeholders to create or extend existing mappings between CDISC and other standards, controlled terminologies, and models to represent data originating across different sources.

Conclusions

There are many ongoing data standardization efforts around human health data–related activities, each with different definitions, levels of granularity, and purpose. Among these, CDISC has been successful in standardizing clinical trial-based data for regulation worldwide. However, the complexity of the CDISC standards and the fact that they were developed for different purposes, combined with the lack of awareness and incentives to use a new standard and insufficient training and implementation support, are significant barriers to setting up the use of CDISC standards for RWD. The collection and dissemination of use cases, development of tools and support systems for the RWD community, and collaboration with other standards development organizations are potential steps forward. Using CDISC will help link clinical trial data and RWD and promote innovation in health data science.

Best practices in the real-world data life cycle

With increasing digitization of healthcare, real-world data (RWD) are available in greater quantity and scope than ever before. Since the 2016 United States 21st Century Cures Act, innovations in the RWD life cycle have taken tremendous strides forward, largely driven by demand for regulatory-grade real-world evidence from the biopharmaceutical sector. However, use cases for RWD continue to grow in number, moving beyond drug development, to population health and direct clinical applications pertinent to payors, providers, and health systems. Effective RWD utilization requires disparate data sources to be turned into high-quality datasets. To harness the potential of RWD for emerging use cases, providers and organizations must accelerate life cycle improvements that support this process. We build on examples obtained from the academic literature and author experience of data curation practices across a diverse range of sectors to describe a standardized RWD life cycle containing key steps in production of useful data for analysis and insights. We delineate best practices that will add value to current data pipelines. Seven themes are highlighted that ensure sustainability and scalability for RWD life cycles: data standards adherence, tailored quality assurance, data entry incentivization, deploying natural language processing, data platform solutions, RWD governance, and ensuring equity and representation in data.

"Another Tool in Your Toolkit": Pediatric Occupational and Physical Therapists' Perspectives of Initiating Telehealth during the COVID-19 Pandemic

AIMS

Pediatric occupational and physical therapy service delivery via telehealth increased during the COVID-19 pandemic. Real-world experience can guide service improvement. This study explored experiences, barriers, and facilitators of initial telehealth implementation from the therapist's perspective.

METHODS

Qualitative descriptive approach. Semi-structured interviews were conducted with occupational therapists (n = 4) and physical therapists (n = 4) between May-June 2020. Interviews were recorded, and transcribed verbatim. Data were coded inductively to generate themes, then re-coded deductively to classify barriers and facilitators to telehealth acceptance and use using the Unified Technology Acceptance Theory.

RESULTS

Participants had 16.5 [(2-35); median (range)] years of experience (3 months with telehealth) and predominantly worked with preschool children. Three themes about telehealth were identified: a practical option; requires skill development and refinement; beneficial in perpetuity. Most frequently cited barriers were the lack of opportunity for 'hands-on' assessment/intervention and the learning curve required. Most frequently cited facilitators included seeing a child in their own environment, attendance may be easier for some families, and families' perception that telehealth was useful.

CONCLUSION

Despite rapid implementation, therapists largely described telehealth as a positive experience. Telehealth facilitated continued service provision and was perceived as relevant post-pandemic. Additional training and ensuring equitable access to services are priorities as telehealth delivery evolves.