Describing a complex primary health care population to support future decision support initiatives

Introduction

Developing decision support tools using data from a health care organization, to support care within that organization, is a promising paradigm to improve care delivery and population health. Descriptive epidemiology may be a valuable supplement to stakeholder input towards selection of potential initiatives and to inform methodological decisions throughout tool development. We additionally propose that to properly characterize complex populations in large-scale descriptive studies, both simple statistical and machine learning techniques can be useful.

Objective

To describe sociodemographic, clinical, and health care use characteristics of primary care clients served by the Alliance for Healthier Communities, which provides team-based primary health care through Community Health Centres (CHCs) across Ontario, Canada.

Methods

We used electronic health record data from adult ongoing primary care clients served by CHCs in 2009-2019. We performed traditional table-based summaries for each characteristic; and applied three unsupervised learning techniques to explore patterns of common condition co-occurrence, care provider teams, and care frequency.

Results

There were 221,047 eligible clients. Sociodemographics: We described 13 characteristics, stratified by CHC type and client multimorbidity status. Clinical characteristics: Eleven-year prevalence of 24 investigated conditions ranged from 1% (Hepatitis C) to 63% (chronic musculoskeletal problem) with non-uniform risk across the care history; multimorbidity was common (81%) with variable co-occurrence patterns. Health care use characteristics: Most care was provided by physician and nursing providers, with heterogeneous combinations of other provider types. A subset of clients had many issues addressed within single-visits and there was within- and between-client variability in care frequency. In addition to substantive findings, we discuss methodological considerations for future decision support initiatives.

Conclusions

We demonstrated the use of methods from statistics and machine learning, applied with an epidemiological lens, to provide an overview of a complex primary care population and lay a foundation for stakeholder engagement and decision support tool development.

Patient-reported outcomes via electronic health record portal versus telephone: a pragmatic randomized pilot trial of anxiety or depression symptoms in epilepsy

Objective

To close gaps between research and clinical practice, tools are needed for efficient pragmatic trial recruitment and patient-reported outcome collection. The objective was to assess feasibility and process measures for patient-reported outcome collection in a randomized trial comparing electronic health record (EHR) patient portal questionnaires to telephone interview among adults with epilepsy and anxiety or depression symptoms.

Materials and Methods

Recruitment for the randomized trial began at an epilepsy clinic visit, with EHR-embedded validated anxiety and depression instruments, followed by automated EHR-based research screening consent and eligibility assessment. Fully eligible individuals later completed telephone consent, enrollment, and randomization. Participants were randomized 1:1 to EHR portal versus telephone outcome assessment, and patient-reported and process outcomes were collected at 3 and 6 months, with primary outcome 6-month retention in EHR arm (feasibility target: ≥11 participants retained).

Results

Participants (N = 30) were 60% women, 77% White/non-Hispanic, with mean age 42.5 years. Among 15 individuals randomized to EHR portal, 10 (67%, CI 41.7%-84.8%) met the 6-month retention endpoint, versus 100% (CI 79.6%-100%) in the telephone group (P = 0.04). EHR outcome collection at 6 months required 11.8 min less research staff time per participant than telephone (5.9, CI 3.3-7.7 vs 17.7, CI 14.1-20.2). Subsequent telephone contact after unsuccessful EHR attempts enabled near complete data collection and still saved staff time.

Discussion

In this randomized study, EHR portal outcome assessment did not meet the retention feasibility target, but EHR method saved research staff time compared to telephone.

Conclusion

While EHR portal outcome assessment was not feasible, hybrid EHR/telephone method was feasible and saved staff time.

Evaluating a learning health system initiative: Lessons learned during COVID-19 in Saskatchewan, Canada

Introduction

Evaluating a learning health system (LHS) encourages continuous system improvement and collaboration within the healthcare system. Although LHS is a widely accepted concept, there is little knowledge about evaluating an LHS. To explore the outputs and outcomes of an LHS model, we evaluated the COVID-19 Evidence Support Team (CEST) in Saskatchewan, Canada, an initiative to rapidly review scientific evidence about COVID-19 for decision-making. By evaluating this program during its formation, we explored how and to what extent the CEST initiative was used by stakeholders. An additional study aim was to understand how CEST could be applied as a functional LHS and the value of similar knowledge-to-action cycles.

Methods

Using a formative evaluation design, we conducted qualitative interviews with key informants (KIs) who were involved with COVID-19 response strategies in Saskatchewan. Transcripts were analyzed using reflexive thematic analysis to identify key themes. A program logic model was created to represent the inputs, activities, outputs, and outcomes of the CEST initiative.

Results

Interview data from 11 KIs were collated under three overarching categories: (1) outputs, (2) short-term outcomes, and (3) long-term outcomes from the CEST initiative. Overall, participants found the CEST initiative improved speed and access to reliable information, supported and influenced decision-making and public health strategies, leveraged partnerships, increased confidence and reassurance, and challenged misinformation. Themes relating to the long-term outcomes of the initiative included improving coordination, awareness, and using good judgment and planning to integrate CEST sustainably into the health system.

Conclusion

This formative evaluation demonstrated that CEST was a valued program and a promising LHS model for Saskatchewan. The future direction involves addressing program recommendations to implement this model as a functional LHS in Saskatchewan.

A call to integrate health equity into learning health system research training

In 2016, the Agency for Healthcare Research and Quality (AHRQ) recommended seven domains for training and mentoring researchers in learning health systems (LHS) science. Health equity was not included as a competency domain. This commentary from scholars in the Consortium for Applied Training to Advance the Learning health system with Scholars/Trainees (CATALyST) K12 program recommends that competency domains be extended to reflect growing demands for evidence on health inequities and interventions to alleviate them. We present real-life case studies from scholars in an LHS research training program that illustrate facilitators, challenges, and potential solutions at the program, funder, and research community-level to receiving training and mentorship in health equity-focused LHS science. We recommend actions in four areas for LHS research training programs: (a) integrate health equity throughout the current LHS domains; (b) develop training and mentoring in health equity; (c) establish program evaluation standards for consideration of health equity; and (d) bring forth relevant, extant expertise from the areas of health disparities research, community-based participatory research, and community-engaged health services research. We emphasize that LHS research must acknowledge and build on the substantial existing contributions, mainly by scholars of color, in the health equity field.

Patient-Reported Outcome Dashboards Within the Electronic Health Record to Support Shared Decision-making: Protocol for Co-design and Clinical Evaluation With Patients With Advanced Cancer and Chronic Kidney Disease

BACKGROUND

Patient-reported outcomes-symptoms, treatment side effects, and health-related quality of life-are important to consider in chronic illness care. The increasing availability of health IT to collect patient-reported outcomes and integrate results within the electronic health record provides an unprecedented opportunity to support patients' symptom monitoring, shared decision-making, and effective use of the health care system.

OBJECTIVE

The objectives of this study are to co-design a dashboard that displays patient-reported outcomes along with other clinical data (eg, laboratory tests, medications, and appointments) within an electronic health record and conduct a longitudinal demonstration trial to evaluate whether the dashboard is associated with improved shared decision-making and disease management outcomes.

METHODS

Co-design teams comprising study investigators, patients with advanced cancer or chronic kidney disease, their care partners, and their clinicians will collaborate to develop the dashboard. Investigators will work with clinic staff to implement the co-designed dashboard for clinical testing during a demonstration trial. The primary outcome of the demonstration trial is whether the quality of shared decision-making increases from baseline to the 3-month follow-up. Secondary outcomes include longitudinal changes in satisfaction with care, self-efficacy in managing treatments and symptoms, health-related quality of life, and use of costly and potentially avoidable health care services. Implementation outcomes (ie, fidelity, appropriateness, acceptability, feasibility, reach, adoption, and sustainability) during the co-design process and demonstration trial will also be collected and summarized.

RESULTS

The dashboard co-design process was completed in May 2020, and data collection for the demonstration trial is anticipated to be completed by the end of July 2022. The results will be disseminated in at least one manuscript per study objective.

CONCLUSIONS

This protocol combines stakeholder engagement, health care coproduction frameworks, and health IT to develop a clinically feasible model of person-centered care delivery. The results will inform our current understanding of how best to integrate patient-reported outcome measures into clinical workflows to improve outcomes and reduce the burden of chronic disease on patients and health care systems.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/38461.

The Texas Health Resources Clinical Scholars Program: Learning healthcare system workforce development through embedded translational research

Introduction

Texas Health Resources (THR), a large, nonprofit health care system based in the Dallas-Fort Worth area, has collaborated with the University of Texas Southwestern Medical Center (UTSW) to develop and operate a unique, integrated approach for Learning Health System (LHS) workforce development. This training model centers on academic health system faculty members conducting later-stage translational research within a partnering regional care delivery organization.

Methods

The THR Clinical Scholars Program engages early career UTSW faculty members to conduct studies that are likely to have an impact on care delivery at the health system level. Interested candidates submit formal applications to the program. A joint committee comprised of senior research faculty from UTSW and THR clinical leadership reviews proposals with a focus on the shared LHS needs of both institutions-developing high quality research output that can be applied to enhance care delivery. A key prioritization criterion for funding is the degree to which the research addresses a question relevant to THR as a high-volume network with multiple channels for consumers to access care. The program emphasis is on supporting embedded research initiatives using health system data to generate knowledge that will improve the quality and efficiency of care for the patient populations served by the participant organizations.

Results

We discuss specific strategic and tactical components of the THR Clinical Scholars Program including an overview of the academic affiliation agreement between the collaborating organizations, criteria for successful program applications, data sharing, and funding. We also share project summaries from selected clinical scholars as examples of the LHS research done in the program to date.

Conclusion

This experience report provides an implementation framework for other academic health systems interested in adopting similar LHS workforce training models with community partners.

Eliciting patients' healthcare goals and concerns: Do questions influence responses?

There is increasing interest in asking patients questions before their visits to elicit goals and concerns, which is part of the move to support the concept of coproducing care. The phrasing and delivery of such questions differs across settings and is likely to influence responses. This report describes a study that (i) used a three-level model to categorize the goals and concerns elicited by two different pre-visit questions, and (ii) describes associations between responses elicited and the phrasing and delivery of the two questions. The questions were administered to patients with rheumatic disease, and patients with inflammatory bowel disease (IBD). Paper-based responses from 150 patients with rheumatic disease and 338 patients with IBD were analyzed (163 paper, 175 electronic). The goals and concerns elicited were primarily disease or symptom-specific. The specific goal and concern examples featured in one pre-visit question were more commonly reported in responses to that question, compared to the question without examples. Questions completed electronically before the visit were associated with longer responses than those completed on paper in the waiting room. In conclusion, how and when patients' goals and concerns are elicited appears to have an impact on responses and warrants further investigation.

Beyond prediction: Off-target uses of artificial intelligence-based predictive analytics in a learning health system

Introduction

Artificial-intelligence (AI)-based predictive analytics provide new opportunities to leverage rich sources of continuous data to improve patient care through early warning of the risk of clinical deterioration and improved situational awareness.Part of the success of predictive analytic implementation relies on integration of the analytic within complex clinical workflows. Pharmaceutical interventions have off-target uses where a drug indication has not been formally studied for a different indication but has potential for clinical benefit. An analog has not been described in the context of AI-based predictive analytics, that is, when a predictive analytic has been trained on one outcome of interest but is used for additional applications in clinical practice.

Methods

In this manuscript we present three clinical vignettes describing off-target use of AI-based predictive analytics that evolved organically through real-world practice.

Results

Off-target uses included:real-time feedback about treatment effectiveness, indication of readiness to discharge, and indication of the acuity of a hospital unit.

Conclusion

Such practice fits well with the learning health system goals to continuously integrate data and experience to provide.

Conceptualizing learning health systems: A mapping review

Introduction

Health systems worldwide face the challenge of increasing population health with high-quality care and reducing health care expenditure growth. In pursuit for a solution, regional cross-sectoral partnerships aim to reorganize and integrate services across public health, health care and social care. Although the complexity of regional partnerships demands an incremental strategy, it is yet not known how learning works within these partnerships. To understand learning in regional cross-sectoral partnerships for health, this study aims to map the concept Learning Health System (LHS).

Methods

This mapping review used a qualitative text analysis approach. A literature search was conducted in Embase and was limited to English-language papers published in the period 2015-2020. Title-abstract screening was performed using established exclusion criteria. During full-text screening, we combined deductive and inductive coding. The concept LHS was disentangled into aims, design elements, and process of learning. Data extraction and analysis were performed in MAX QDA 2020.

Results

In total, 155 articles were included. All articles used the LHS definition of the Institute of Medicine. The interpretation of the concept LHS varied widely. The description of LHS contained 25 highly connected aims. In addition, we identified nine design elements. Most elements were described similarly, only the interpretation of stakeholders, data infrastructure and data varied. Furthermore, we identified three types of learning: learning as 1) interaction between clinical practice and research; 2) a circular process of converting routine care data to knowledge, knowledge to performance; and performance to data; and 3) recurrent interaction between stakeholders to identify opportunities for change, to reveal underlying values, and to evaluate processes. Typology 3 was underrepresented, and the three types of learning rarely occurred simultaneously.

Conclusion

To understand learning within regional cross-sectoral partnerships for health, we suggest to specify LHS-aim(s), operationalize design elements, and choose deliberately appropriate learning type(s).

Creating a provincial post COVID-19 interdisciplinary clinical care network as a learning health system during the pandemic: Integrating clinical care and research

Introduction

Coronavirus Disease-2019 (COVID-19) affects multiple organ systems in the acute phase and also has long-term sequelae. Research on the long-term impacts of COVID-19 is limited. The Post COVID-19 Interdisciplinary Clinical Care Network (PC-ICCN), conceived in July 2020, is a provincially funded resource that is modelled as a Learning Health System (LHS), focused on those people with persistent symptoms post COVID-19 infection.

Methods

The PC-ICCN emerged through collaboration among over 60 clinical specialists, researchers, patients, and health administrators. At the core of the network are the post COVID-19 Recovery Clinics (PCRCs), which provide direct patient care that includes standardized testing and education at regular follow-up intervals for a minimum of 12 months post enrolment. The PC-ICCN patient registry captures data on all COVID-19 patients with confirmed infection, by laboratory testing or epi-linkage, who have been referred to one of five post COVID-19 Recovery Clinics at the time of referral, with data stored in a fully encrypted Oracle-based provincial database. The PC-ICCN has centralized administrative and operational oversight, multi-stakeholder governance, purpose built data collection supported through clinical operations geographically dispersed across the province, and research operations including data analytics.

Results

To date, 5364 patients have been referred, with an increasing number and capacity of these clinics, and 2354 people have had at least one clinic visit. Since inception, the PC-ICCN has received over 30 research proposal requests. This is aligned with the goal of creating infrastructure to support a wide variety of research to improve care and outcomes for patients experiencing long-term symptoms following COVID-19 infection.

Conclusions

The PC-ICCN is a first-in-kind initiative in British Columbia to enhance knowledge and understanding of the sequelae of COVID-19 infection over time. This provincial initiative serves as a model for other national and international endeavors to enable care as research and research as care.

A Scalable Service to Improve Health Care Quality Through Precision Audit and Feedback: Proposal for a Randomized Controlled Trial

BACKGROUND

Health care delivery organizations lack evidence-based strategies for using quality measurement data to improve performance. Audit and feedback (A&F), the delivery of clinical performance summaries to providers, demonstrates the potential for large effects on clinical practice but is currently implemented as a blunt one size fits most intervention. Each provider in a care setting typically receives a performance summary of identical metrics in a common format despite the growing recognition that precisionizing interventions hold significant promise in improving their impact. A precision approach to A&F prioritizes the display of information in a single metric that, for each recipient, carries the highest value for performance improvement, such as when the metric's level drops below a peer benchmark or minimum standard for the first time, thereby revealing an actionable performance gap. Furthermore, precision A&F uses an optimal message format (including framing and visual displays) based on what is known about the recipient and the intended gist meaning being communicated to improve message interpretation while reducing the cognitive processing burden. Well-established psychological principles, frameworks, and theories form a feedback intervention knowledge base to achieve precision A&F. From an informatics perspective, precision A&F requires a knowledge-based system that enables mass customization by representing knowledge configurable at the group and individual levels.

OBJECTIVE

This study aims to implement and evaluate a demonstration system for precision A&F in anesthesia care and to assess the effect of precision feedback emails on care quality and outcomes in a national quality improvement consortium.

METHODS

We propose to achieve our aims by conducting 3 studies: a requirements analysis and preferences elicitation study using human-centered design and conjoint analysis methods, a software service development and implementation study, and a cluster randomized controlled trial of a precision A&F service with a concurrent process evaluation. This study will be conducted with the Multicenter Perioperative Outcomes Group, a national anesthesia quality improvement consortium with >60 member hospitals in >20 US states. This study will extend the Multicenter Perioperative Outcomes Group quality improvement infrastructure by using existing data and performance measurement processes.

RESULTS

The proposal was funded in September 2021 with a 4-year timeline. Data collection for Aim 1 began in March 2022. We plan for a 24-month trial timeline, with the intervention period of the trial beginning in March 2024.

CONCLUSIONS

The proposed aims will collectively demonstrate a precision feedback service developed using an open-source technical infrastructure for computable knowledge management. By implementing and evaluating a demonstration system for precision feedback, we create the potential to observe the conditions under which feedback interventions are effective.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/34990.

Learning environments, reliability enhancing work practices, employee engagement, and safety climate in VA cardiac catheterization laboratories

OBJECTIVE

To characterize the relationship between learning environments (the educational approaches, cultural context, and settings in which teaching and learning happen) and reliability enhancing work practices (hiring, training, decision making) with employee engagement, retention, and safety climate.

DATA SOURCE

We collected data using the Learning Environment and High Reliability Practices Survey (LEHRs) from 231 physicians, nurses, and technicians at 67 Veterans Affairs cardiac catheterization laboratories who care for high-risk Veterans.

STUDY DESIGN

The association between the average LEHRs score and employee job satisfaction, burnout, intent to leave, turnover, and safety climate were modeled in separate linear mixed effect models adjusting for other covariates.

DATA COLLECTION

Participants responded to a web-only survey from August through September 2020.

PRINCIPAL FINDINGS

There was a significant association between higher average LEHRs scores and (1) higher job satisfaction (2) lower burnout, (3) lower intent to leave, (4) lower cath lab turnover in the previous 12 months, and (5) higher perceived safety climate.

CONCLUSIONS

Learning environments and use of reliability enhancing work practices are potential new avenues to support satisfaction and safety climate while lowering burnout, intent to leave, and turnover in a diverse US health care workforce that serves a vulnerable and marginalized population.

Streamlined versus traditional consent for low-risk comparative effectiveness trials: a randomized experimental study to measure patients' and public attitudes

Aim: Streamlining consent for low-risk comparative effectiveness research (CER) could facilitate research, while safeguarding patients' rights. Materials & methods: 2618 adults were randomized to one of seven consent approaches (six streamlined and one traditional) for a hypothetical, low-risk CER study. A survey measured understanding, voluntariness, and feelings of respect. Results: Participants in all arms had a high understanding of the trial and positive attitudes toward the consent interaction. Highest satisfaction was with a streamlined approach showing a video before the medical appointment. Participants in streamlined were more likely to mistakenly think a signature was required. Conclusion: Streamlined consent was no less acceptable than traditional, signed consent. Streamlined and traditional approaches achieved similar levels of understanding, voluntariness and a feeling that the doctor-patient interaction was respectful.

Standardized Health data and Research Exchange (SHaRE): promoting a learning health system

Aggregate de-identified data from electronic health records (EHRs) provide a valuable resource for research. The Standardized Health data and Research Exchange (SHaRE) is a diverse group of US healthcare organizations contributing to the Cerner Health Facts (HF) and Cerner Real-World Data (CRWD) initiatives. The 51 facilities at the 7 founding organizations have provided data about more than 4.8 million patients with 63 million encounters to HF and 7.4 million patients and 119 million encounters to CRWD. SHaRE organizations unmask their organization IDs and provide 3-digit zip code (zip3) data to support epidemiology and disparity research. SHaRE enables communication between members, facilitating data validation and collaboration as we demonstrate by comparing imputed EHR module usage to actual usage. Unlike other data sharing initiatives, no additional technology installation is required. SHaRE establishes a foundation for members to engage in discussions that bridge data science research and patient care, promoting the learning health system.

Supporting the Utilization of Community-Based Primary Health Care Implementation Research in Ghana

Ever since the 1990s, implementation research in Ghana has guided the development of policies and practices that are essential to establishing community-based primary health care. In response to evidence emerging from this research, the Community-based Health Planning and Services (CHPS) policy was promulgated in 1999 to scale-up results. However, during the first decade of CHPS operation, national monitoring showed that its pace of coverage expansion was unacceptably slow. In 2010, the Ghana Health Service launched a five-year plausibility trial of CHPS reform for testing ways to accelerate scale-up. This initiative, known as the Ghana Essential Health Intervention Program (GEHIP), included a knowledge management component for establishing congruence of knowledge generation and flow with the operational system that GEHIP evidence was intended to reform. Four Upper East Region districts served as trial areas while seven districts were comparison areas. Interventions tested means of developing the upward flow of information based on perspectives of district managers, sub-district supervisors, and community-level workers. GEHIP also endeavored to improve procedures for the downward flow and utilization of policy guidelines. Field exchanges were convened for providing national, regional, and district leaders with opportunities for participatory learning about GEHIP implementation innovations. This systems approach facilitated the process of augmenting the communication of evidence with practical field experience. Scientific rigor associated with the production of evidence was thereby integrated into management decision-making processes in ways that institutionalized learning at all levels. The GEHIP knowledge management system functioned as a prototype for guiding the planning of a national knowledge management strategy. A follow-up project transferred its mechanisms from the Upper East Regional Health Administration to the Policy Planning Monitoring and Evaluation Division of the Ghana Health Service in Accra.

Developing organizational learning for scaling-up community-based primary health care in Ghana

INTRODUCTION

Achieving effective community-based primary health care requires evidence for guiding strategic decisions that must be made. However, research processes often limit data collection to particular organizational levels or disseminate results to specific audiences. Decision-making that emerges can fail to account for the contrasting perspectives and needs of managers at each organizational level. The Ghana Health Service (GHS) addressed this problem with a multilevel and sequential research and action approach that has provided two decades of implementation learning for guiding community-based primary health care development.

METHOD

The GHS implementation research initiatives progressed from (i) a participatory pilot investigation to (ii) an experimental trial of strategies that emerged to (iii) replication research for testing scale-up, culminating in (iv) evidence-based scale-up of a national community-based primary health care program. A reform process subsequently repeated this sequence in a manner that involved stakeholders at the community, sub-district, district, and regional levels of the system. The conduct, interpretation, and dissemination of results that emerged comprised a strategy for achieving systems learning by conducting investigations in phases in conjunction with bottom-up knowledge capture, lateral exchanges for fostering peer learning at each system level, and top-down processes for communicating results as policy. Continuous accumulation of qualitative data on stakeholder reactions to operations at each organizational level was conducted in conjunction with quantitative monitoring of field operations.

RESULTS

Implementation policies were enhanced by results associated with each phase. A quasi-experiment for testing the reform process showed that scale-up of community-based primary health care was accelerated, leading to improvements in childhood survival and reduced fertility.

CONCLUSION

Challenges to system learning were overcome despite severe resource constraints. The integration of knowledge generation with ongoing management processes institutionalized learning for achieving evidence-driven program action.

Using a Multi-Institutional Pediatric Learning Health System to Identify Systemic Lupus Erythematosus and Lupus Nephritis: Development and Validation of Computable Phenotypes

BACKGROUND AND OBJECTIVES

Performing adequately powered clinical trials in pediatric diseases, such as SLE, is challenging. Improved recruitment strategies are needed for identifying patients.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

Electronic health record algorithms were developed and tested to identify children with SLE both with and without lupus nephritis. We used single-center electronic health record data to develop computable phenotypes composed of diagnosis, medication, procedure, and utilization codes. These were evaluated iteratively against a manually assembled database of patients with SLE. The highest-performing phenotypes were then evaluated across institutions in PEDSnet, a national health care systems network of >6.7 million children. Reviewers blinded to case status used standardized forms to review random samples of cases (n=350) and noncases (n=350).

RESULTS

Final algorithms consisted of both utilization and diagnostic criteria. For both, utilization criteria included two or more in-person visits with nephrology or rheumatology and ≥60 days follow-up. SLE diagnostic criteria included absence of neonatal lupus, one or more hydroxychloroquine exposures, and either three or more qualifying diagnosis codes separated by ≥30 days or one or more diagnosis codes and one or more kidney biopsy procedure codes. Sensitivity was 100% (95% confidence interval [95% CI], 99 to 100), specificity was 92% (95% CI, 88 to 94), positive predictive value was 91% (95% CI, 87 to 94), and negative predictive value was 100% (95% CI, 99 to 100). Lupus nephritis diagnostic criteria included either three or more qualifying lupus nephritis diagnosis codes (or SLE codes on the same day as glomerular/kidney codes) separated by ≥30 days or one or more SLE diagnosis codes and one or more kidney biopsy procedure codes. Sensitivity was 90% (95% CI, 85 to 94), specificity was 93% (95% CI, 89 to 97), positive predictive value was 94% (95% CI, 89 to 97), and negative predictive value was 90% (95% CI, 84 to 94). Algorithms identified 1508 children with SLE at PEDSnet institutions (537 with lupus nephritis), 809 of whom were seen in the past 12 months.

CONCLUSIONS

Electronic health record-based algorithms for SLE and lupus nephritis demonstrated excellent classification accuracy across PEDSnet institutions.

Collaborative conversations during the time of COVID-19: Building a "meta"-learning community

PROBLEM

COVID-19 created new research, clinical, educational, and personal challenges, while simultaneously separating work teams who were under work-from-home restrictions. Addressing these challenges required new forms of collaborative groups.

APPROACH

To support the department community and the rapid sharing of new research, educational, clinical, and personal efforts, a Core Team from the Department of Learning Health Sciences at the University of Michigan developed a meeting series called the COVID Conversations. This Experience Report shares the organizational structure of the COVID Conversations, proposes a comparison to traditional Learning Communities, and reports the results of a questionnaire that gathered details about department members' COVID-related activities.

OUTCOMES

We identify and describe salient similarities and differences between the COVID Conversations and the characteristics of Learning Communities. We also developed and piloted a taxonomy for characterizing LHS research projects that may be further developed for use in Learning Community planning, in conjunction with other maturity grids and ontologies. We propose the term "Meta-Learning Community" to describe the structure and function of the COVID Conversations.

NEXT STEPS

In academic medicine, remote work, telemedicine, and virtual learning may be here to stay. The COVID Conversations constitute a distinct and innovative form of collaborative work in which separate teams addressing distinct goals, yet sharing a common passion to tackle the issues brought by the pandemic, are able to share experiences and learn from one other. The challenges of COVID-19 have made evident the need for multiple forms of organizing teamwork, and our study contributes the notion of a "Meta"-Learning Community as a new form of collaborative work.

An Implementation Science Laboratory as One Approach to Whole System Improvement: A Canadian Healthcare Perspective

Implementation science (IS) has emerged as an integral component for evidence-based whole system improvement. IS studies the best methods to promote the systematic uptake of evidence-based interventions into routine practice to improve the quality and effectiveness of health service delivery and patient care. IS laboratories (IS labs) are one mechanism to integrate implementation science as an evidence-based approach to whole system improvement and to support a learning health system. This paper aims to examine if IS labs are a suitable approach to whole system improvement. We retrospectively analyzed an existing IS lab (Alberta, Canada’s Implementation Science Collaborative) to assess the potential of IS labs to perform as a whole system approach to improvement and to identify key activities and considerations for designing IS labs specifically to support learning health systems. Results from our evaluation show the extent to which IS labs support learning health systems through enabling infrastructures for system-wide improvement and research.

GA4GH: International policies and standards for data sharing across genomic research and healthcare

The Global Alliance for Genomics and Health (GA4GH) aims to accelerate biomedical advances by enabling the responsible sharing of clinical and genomic data through both harmonized data aggregation and federated approaches. The decreasing cost of genomic sequencing (along with other genome-wide molecular assays) and increasing evidence of its clinical utility will soon drive the generation of sequence data from tens of millions of humans, with increasing levels of diversity. In this perspective, we present the GA4GH strategies for addressing the major challenges of this data revolution. We describe the GA4GH organization, which is fueled by the development efforts of eight Work Streams and informed by the needs of 24 Driver Projects and other key stakeholders. We present the GA4GH suite of secure, interoperable technical standards and policy frameworks and review the current status of standards, their relevance to key domains of research and clinical care, and future plans of GA4GH. Broad international participation in building, adopting, and deploying GA4GH standards and frameworks will catalyze an unprecedented effort in data sharing that will be critical to advancing genomic medicine and ensuring that all populations can access its benefits.

A roadmap to operationalize and evaluate impact in a learning health system

BACKGROUND

Many health systems invest in initiatives to accelerate translation of knowledge into practice. However, organizations lack guidance on how to develop and operationalize such Learning Health System (LHS) programs and evaluate their impact. Kaiser Permanente Washington (KPWA) launched our LHS program in June 2017 and developed a logic model as a foundation to evaluate the program's impact.

OBJECTIVE

To develop a roadmap for organizations that want to establish an LHS program, understand how LHS core components relate to one another when operationalized in practice, and evaluate and improve their progress.

METHODS

We conducted a narrative review on LHS models, key model components, and measurement approaches.

RESULTS

The KPWA LHS Logic Model provides a broad set of constructs relevant to LHS programs, depicts their relationship to LHS operations, harmonizes terms across models, and offers measurable operationalizations of each construct to guide other health systems. The model identifies essential LHS inputs, provides transparency into LHS activities, and defines key outcomes to evaluate LHS processes and impact. We provide reflections on the most helpful components of the model and identify areas that need further improvement using illustrative examples from deployment of the LHS model during the COVID-19 pandemic.

CONCLUSION

The KPWA LHS Logic Model is a starting point for future LHS implementation research and a practical guide for healthcare organizations that are building, operationalizing, and evaluating LHS initiatives.

A learning health systems approach to integrating electronic patient-reported outcomes across the health care organization

INTRODUCTION

Foundational to a learning health system (LHS) is the presence of a data infrastructure that can support continuous learning and improve patient outcomes. To advance their capacity to drive patient-centered care, health systems are increasingly looking to expand the electronic capture of patient data, such as electronic patient-reported outcome (ePRO) measures. Yet ePROs bring unique considerations around workflow, measurement, and technology that health systems may not be poised to navigate. We report on our effort to develop generalizable learnings that can support the integration of ePROs into clinical practice within an LHS framework.

METHODS

Guided by action research methodology, we engaged in iterative cycles of planning, acting, observing, and reflecting around ePRO use with two primary goals: (1) mobilize an ePRO community of practice to facilitate knowledge sharing, and (2) establish guidelines for ePRO use in the context of LHS practice. Multiple, emergent data collection activities generated generalizable guidelines that document the tangible best practices for ePRO use in clinical care. We organized guidelines around thematic areas that reflect LHS structures and stakeholders.

RESULTS

Three core thematic areas (and 24 guidelines) emerged. The theme of governance reflects the importance of leadership, knowledge management, and facilitating organizational learning around best practice models for ePRO use. The theme of integration considers the intersection of workflow, technology, and human factors for ePROs across areas of care delivery. Lastly, the theme of reporting reflects critical considerations for curating data and information, designing system functions and interactions, and presentation of ePRO data to support the translation of knowledge to action.

CONCLUSIONS

The guidelines produced from this work highlight the complex, multidisciplinary nature of implementing change within LHS contexts, and the value of action research approaches to enable rapid, iterative learning that leverages the knowledge and experience of communities of practice.

General orders for the embedded researcher: Moorings for a developing profession

Learning health systems increasingly welcome embedded researchers as stakeholders poised to inform evidence-based practice. While care systems are potentially familiar with the embedded researcher tools and techniques, care systems may less frequently consider embedded research as a vocation. This insensitivity potentially reduces embedded researchers merely to instruments, as opposed to professional partners in transdisciplinary research. This discussion outlines "general orders" for embedded researchers. The general orders outline embedded researchers' fundamental identity and guide conduct as a means to encourage a shared identity among embedded researchers and clarify embedded researchers' roles in learning health system teams. Students and embedded researchers newly engaging learning health systems may particularly benefit from this rudimentary order list.

The triple helix of clinical, research, and education missions in academic health centers: A qualitative study of diverse stakeholder perspectives

INTRODUCTION

Academic health centers are poised to improve health through their clinical, education, and research missions. However, these missions often operate in silos. The authors explored stakeholder perspectives at diverse institutions to understand challenges and identify alignment strategies.

METHODS

Authors used an exploratory qualitative design and thematic analysis approach with data obtained from electronic surveys sent to participants at five U.S. academic health centers (2017-18), with four different types of medical school/health system partnerships. Participants included educators, researchers, system leaders, administrators, clinical providers, resident/fellow physicians, and students. Investigators coded data using constant comparative analysis, met regularly to reconcile uncertainties, and collapsed/combined categories.

RESULTS

Of 175 participants invited, 113 completed the survey (65%). Three results categories were identified. First, five higher-order themes emerged related to aligning missions, including (a) shared vision and strategies, (b) alignment of strategy with community needs, (c) tension of economic drivers, (d) coproduction of knowledge, and (e) unifying set of concepts spanning all missions. Second, strategies for each mission were identified, including education (new competencies, instructional methods, recruitment), research (shifting agenda, developing partnerships, operations), and clinical operations (delivery models, focus on patient factors/needs, value-based care, well-being). Lastly, strategies for integrating each dyadic mission pair, including research-education, clinical operations education, and research-clinical operations, were identified.

CONCLUSIONS

Academic health centers are at a crossroads in regard to identity and alignment across the tripartite missions. The study's results provide pragmatic strategies to advance the tripartite missions and lead necessary change for improved patient health.

Identifying strategies to overcome roadblocks to utilising near real-time healthcare and administrative data to create a Scotland-wide learning health system

Creating a learning health system could help reduce variations in quality of care. Success is dependent on timely access to health data. To explore the barriers and facilitators to timely access to patients' data, we conducted in-depth semi-structured interviews with 37 purposively sampled participants from government, the NHS and academia across Scotland. Interviews were analysed using the framework approach. Participants were of the view that Scotland could play a leading role in the exploitation of routine data to drive forward service improvements, but highlighted major impediments: (i) persistence of paper-based records and a variety of information systems; (ii) the need for a proportionate approach to managing information governance; and (iii) the need for support structures to facilitate accrual, processing, linking, analysis and timely use and reuse of data for patient benefit. There is a pressing need to digitise and integrate existing health information infrastructures, guided by a nationwide proportionate information governance approach and the need to enhance technological and human capabilities to support these efforts.