Prospective application of implementation science theories and frameworks to inform use of PROMs in routine clinical care within an integrated pain network

Electronic Implementation of Patient-Reported Outcome Measures in Primary Health Care: Mixed Methods Systematic Review

BACKGROUND

Managing chronic diseases remains a critical challenge in primary health care (PHC) across the Organization for Economic Co-operation and Development countries. Electronic patient-reported outcome measures (ePROMs) are emerging as valuable tools for enhancing patient engagement, facilitating clinical decision-making, and improving health outcomes. However, their implementation in PHC remains limited, with significant variability in effectiveness and adoption.

OBJECTIVE

This systematic review aimed to assess the implementation and effectiveness of ePROMs in chronic disease management within PHC settings and to identify key barriers and facilitators influencing their integration.

METHODS

A mixed methods systematic review was conducted following the Cochrane Methods and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We included studies that implemented ePROMs among adults for chronic disease management in PHC. The extracted data included patient health outcomes, provider workflow implications, implementation factors, and cost considerations. The reach, effectiveness, adoption, implementation, and maintenance framework was used.

RESULTS

Our search yielded 12,525 references, from which 22 (0.18%) studies were included after screening and exclusions. These studies, primarily conducted in the United States (n=9, 41%) and Canada (n=8, 36%), covered various chronic diseases and used diverse ePROM tools, predominantly mobile apps (n=9, 41%). While some studies (n=10, 45%) reported improvements in patient health outcomes and self-management, others (n=12, 55%) indicated no significant change. Key barriers included digital literacy gaps, integration challenges within clinical workflows, and increased provider workload. Facilitators included strong patient-provider relationships, personalized interventions, and technical support for users. While some studies (n=10, 45%) demonstrated improved patient engagement and self-management, long-term cost-effectiveness and sustainability remain uncertain.

CONCLUSIONS

Success in implementing ePROMs in PHC appears to hinge on addressing digital literacy, ensuring personalization and meaningful patient-provider interactions, carefully integrating technology into clinical workflows, and conducting thorough research on their long-term impacts and cost-effectiveness. Future efforts should focus on these areas to fully realize the benefits of digital health technologies for patients, providers, and health care systems.

TRIAL REGISTRATION

PROSPERO CRD42022333513; https://www.crd.york.ac.uk/PROSPERO/view/CRD42022333513.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/48155.

Characteristics of Quality Improvement Projects in Health Services: A Systematic Scoping Review

OBJECTIVE

Current QI reports within the literature frequently fail to provide enough information regarding interventions, and a significant number of publications do not mention the utilization of a guiding model or framework. The objective of this scoping review was to synthesize the characteristics of hospital-based QI interventions and assess their alignment with recommended quality goals.

METHODS

This scoping review followed the JBI methodology for scoping reviews to synthesize existing literature on hospital-based QI interventions and reporting using the PRISMA Extension for scoping reviews. Included studies involved a hospital-based QI intervention that was evaluated through the Development of the Quality Improvement Minimum Quality Criteria Set (QI-MQCS) framework, reporting on hospital users' (i.e., practitioners and patients) data. We searched Medline, CINAHL, Embase and PubMed databases for primary research published between 2015 and 2024. Grey literature was also examined. A narrative synthesis guided the integration of findings.

RESULTS

From 1398 identified records, 70 relevant records were included. Results indicate a wide variation in QI frameworks and methods used by the included studies. The QI interventions most frequently assessed were organizational-focused (n = 59), followed by professional-related interventions (n = 41) and patient-care interventions (n = 24). There were multiple facilitators and barriers across organizational, professional, and patient care levels found in the included studies. Examples of facilitators were instrumental in driving successful QI initiatives included education, training, active leadership, and stakeholder engagement. Conversely, barriers such as time constraints, resource limitations, and resistance were highlighted.

CONCLUSION

Existing QI publications lack sufficient detail to replicate interventions. Using a model or framework to guide the conduct of a QI-activity may support a more robustly designed and well-conducted project. The variation of reporting characteristics suggests that future research should focus on the development of a pragmatic tool for use by front-line clinicians to support consistent and rigorous conduct of QI projects.

The NASSS (Non-Adoption, Abandonment, Scale-Up, Spread and Sustainability) framework use over time: A scoping review

The Non-adoption, Abandonment, Scale-up, Spread, Sustainability (NASSS) framework (2017) was established as an evidence-based, theory-informed tool to predict and evaluate the success of implementing health and care technologies. While the NASSS is gaining popularity, its use has not been systematically described. Literature reviews on the applications of popular implementation frameworks, such as the RE-AIM and the CFIR, have enabled their advancement in implementation science. Similarly, we sought to advance the science of implementation and application of theories, models, and frameworks (TMFs) in research by exploring the application of the NASSS in the five years since its inception. We aim to understand the characteristics of studies that used the NASSS, how it was used, and the lessons learned from its application. We conducted a scoping review following the JBI methodology. On December 20, 2022, we searched the following databases: Ovid MEDLINE, EMBASE, PsychINFO, CINAHL, Scopus, Web of Science, and LISTA. We used typologies and frameworks to characterize evidence to address our aim. This review included 57 studies that were qualitative (n=28), mixed/multi-methods (n=13), case studies (n=6), observational (n=3), experimental (n=3), and other designs (e.g., quality improvement) (n=4). The four most common types of digital applications being implemented were telemedicine/virtual care (n=24), personal health devices (n=10), digital interventions such as internet Cognitive Behavioural Therapies (n=10), and knowledge generation applications (n=9). Studies used the NASSS to inform study design (n=9), data collection (n=35), analysis (n=41), data presentation (n=33), and interpretation (n=39). Most studies applied the NASSS retrospectively to implementation (n=33). The remainder applied the NASSS prospectively (n=15) or concurrently (n=8) with implementation. We also collated reported barriers and enablers to implementation. We found the most reported barriers fell within the Organization and Adopter System domains, and the most frequently reported enablers fell within the Value Proposition domain. Eighteen studies highlighted the NASSS as a valuable and practical resource, particularly for unravelling complexities, comprehending implementation context, understanding contextual relevance in implementing health technology, and recognizing its adaptable nature to cater to researchers' requirements. Most studies used the NASSS retrospectively, which may be attributed to the framework's novelty. However, this finding highlights the need for prospective and concurrent application of the NASSS within the implementation process. In addition, almost all included studies reported multiple domains as barriers and enablers to implementation, indicating that implementation is a highly complex process that requires careful preparation to ensure implementation success. Finally, we identified a need for better reporting when using the NASSS in implementation research to contribute to the collective knowledge in the field.

Evaluating the implementation of PROMs and PREMs in routine clinical care: co-design of tools from the perspective of patients and healthcare professionals

BACKGROUND

Implementation of patient-reported measures (PRMs) is an integral element for patient-centered models; however, there is still hardly any quantitative evidence regarding its impact in routine care settings. The objective of this study was to codesign two concise tools that allow for a standardized and longitudinal assessment of the implementation of PRMs in routine care in terms of acceptability and perceived value from the perspective of both patients and healthcare professionals.

METHODS

A list of constructs and items to be presented, separately, to patients and healthcare professionals was created from evidence gathered through a narrative literature review. Focus groups, composed of either patients or healthcare professionals from different chronic conditions, were conducted for the co-design of independent assessments. Once agreement was reached, the content validity was examined in separate consensus meetings.

RESULTS

A total of 10 patients and 10 healthcare professionals participated in the focus groups. After 7 focus groups, the PRMs Implementation Assessment Tool for patients (PRMIAT-P) was developed with 33 items in 9 constructs, and the tool for healthcare professionals (PRMIAT-HP) had 33 items in 16 constructs. Content validity was confirmed for both tools.

CONCLUSIONS

The perspective of patients and healthcare professionals regarding the implementation of PRMs in routine care can be evaluated quantitively with the PRMIAT tools. These tools are understandable, concise and comprehensive, and can be used in multiple settings and for different chronic conditions. They have been codesigned as a standard set to facilitate both longitudinal assessments and performing benchmarking among different initiatives.

Enhancing representativeness of patient-reported outcomes in routine radiation oncology care: a quality improvement protocol to address non-response

INTRODUCTION

Non-response significantly undermines the representativeness of patient-reported outcome (PRO) data, thereby compromising its utility for facilitating high-value, equitable, patient-centred care in cancer clinics. Quality improvement studies are needed to assess the representativeness of PRO data collected in routine care, identify the underlying causes of non-response and develop novel methods to ensure data representativeness. Using a multilevel framework and a mixed-methods approach, we have three aims: (1) characterise the non-response of the Global-10 across clinic, provider and patient levels; (2) identify multilevel causes of non-response and potential strategies to improve representativeness in PRO collection; and (3) develop effective modifications to missing-data methods to enhance the representativeness of pre-existing PRO data.

METHODS AND ANALYSIS

Our primary data source is the Patient Reported Outcomes Measurement Information System Global-10, collected as part of routine care at the Radiation Oncology clinics within the Mass General Brigham (MGB) healthcare system. Other sources include (1) Harvard Catalyst for provider-specific data, (2) MGB administrative data, (3) public Centers for Medicare & Medicaid Services data and (4) the National Plan and Provider Enumeration System. We will conduct quantitative analyses to assess variations in Global-10 non-response across multilevel factors. Additionally, we will use qualitative interviews with patients and clinical professionals to understand the causes of non-response and to formulate strategies to expand the reach of PRO collection to underrepresented cancer patients, improve their completions and enhance overall data representativeness. Finally, we will integrate implementation science knowledge and findings from the first two aims into missing-data methods to manage non-response in the pre-existing Global-10 data and to evaluate their performance in preserving representativeness.

ETHICS AND DISSEMINATION

The study protocol was reviewed and approved by the Institutional Review Board at the Dana-Farber/Harvard Cancer Center (24-225). Written informed consent will be obtained from participants. Study findings will be disseminated through peer-reviewed publications and presentations at national and international conferences.

Barriers and facilitators to high-volume evidence-based innovation and implementation in a large, community-based learning health system

BACKGROUND

Broad-scale, rapid health care change is critically needed to improve value-based, effective health care. Health care providers and systems need to address common barriers and facilitators across the evidence to implementation pathway, across diverse specialties. However, most evidence translation / implementation research evaluates single topic areas, and may be of limited value for informing comprehensive efforts. This project's objective was to identify, characterize, and illustrate common trans-topic facilitators and barriers of translating new health care evidence results to clinical implementation across multiple medical specialties.

METHODS

This study was an evaluation of all evidence-based innovation projects completed during 2019-2021. Each project was created with medical group clinical leaders and was intended to inform clinical care. The evaluation took place in a large community-based integrated health care system, and an embedded delivery science and applied research program. Clinical investigators, scientific investigators, and clinical operational leaders received structured questionnaires regarding barriers and facilitators for the operational implementation of new research findings for each project. Responses were mapped to the Consolidated Framework for Implementation Research to identify perceived implementation barriers and facilitators.

RESULTS

All 48 projects completed between 2019 and 2021 were evaluated; responses were received for 45 (94%) and 34 had comments mappable to framework domains. Potential barriers and facilitators to clinical implementation of new research results were identified across all five framework domains and, within these, the 38 constructs or sub-constructs. Among 245 total comments, the most commonly cited facilitators were how the new research evidence generated, compelled change (n = 29), specialty communication networks for disseminating results and initiating change (n = 20), leadership engagement in the project (n = 19), and the innovation's relative advantage over existing practices (n = 11). The most commonly cited barriers were inadequate resource commitment for next-step implementation (n = 15), insufficient learning/implementation culture (n = 5), and insufficient individual-level willingness/ability for change (n = 5).

CONCLUSIONS

A novel large-scale evaluation of barriers and facilitators across the evidence to implementation pathway identified common factors across multiple topic areas and specialties. These common potentially replicable facilitators and modifiable barriers can focus health systems and leaders pursuing large-volume evidence-to-implementation initiatives on those areas with the likely greatest benefit-for-effort, for accelerating health care change.

Understanding factors impacting patient-reported outcome measures integration in routine clinical practice: an umbrella review

PURPOSE

Patient-report outcome measures (PROMs) have gained widespread support as a mechanism to improve healthcare quality. We aimed to map out key enablers and barriers influencing PROMs implementation strategies in routine clinical practice.

METHODS

An umbrella review was conducted to identify reviews exploring enablers and barriers related to the integration of PROMs in routine clinical practice from January 2000 to June 2023. Information on key enablers and barriers was extracted and summarised thematically according to the Theoretical Domains Framework.

RESULTS

34 reviews met our criteria for inclusion. Identified reviews highlighted barriers such as limited PROMs awareness among clinicians and patients, perceived low value by clinicians and patients, PROMs that were too complex or difficult for patients to complete, poor usability of PROMs systems, delayed feedback of PROMs data, clinician concerns related to use of PROMs as a performance management tool, patient concerns regarding privacy and security, and resource constraints. Enablers encompassed phased implementation, professional training, stakeholder engagement prior to implementation, clear strategies and goals, 'change champions' to support PROMs implementation, systems to respond to issues raised by PROMs, and integration into patient pathways. No consensus favoured paper or electronic PROMs, yet offering both options to mitigate digital literacy bias and integrating PROMs into electronic health records emerged as important facilitators.

CONCLUSIONS

The sustainable implementation of PROMs is a complex process that requires multicomponent organisational strategies covering training and guidance, necessary time and resources, roles and responsibilities, and consultation with patients and clinicians.

Use of implementation mapping to develop a multifaceted implementation strategy for an electronic prospective surveillance model for cancer rehabilitation

BACKGROUND

Electronic Prospective Surveillance Models (ePSMs) remotely monitor the rehabilitation needs of people with cancer via patient-reported outcomes at pre-defined time points during cancer care and deliver support, including links to self-management education and community programs, and recommendations for further clinical screening and rehabilitation referrals. Previous guidance on implementing ePSMs lacks sufficient detail on approaches to select implementation strategies for these systems. The purpose of this article is to describe how we developed an implementation plan for REACH, an ePSM system designed for breast, colorectal, lymphoma, and head and neck cancers.

METHODS

Implementation Mapping guided the process of developing the implementation plan. We integrated findings from a scoping review and qualitative study our team conducted to identify determinants to implementation, implementation actors and actions, and relevant outcomes. Determinants were categorized using the Consolidated Framework for Implementation Research (CFIR), and the implementation outcomes taxonomy guided the identification of outcomes. Next, determinants were mapped to the Expert Recommendations for Implementing Change (ERIC) taxonomy of strategies using the CFIR-ERIC Matching Tool. The list of strategies produced was refined through discussion amongst our team and feedback from knowledge users considering each strategy's feasibility and importance rating via the Go-Zone plot, feasibility and applicability to the clinical contexts, and use among other ePSMs reported in our scoping review.

RESULTS

Of the 39 CFIR constructs, 22 were identified as relevant determinants. Clinic managers, information technology teams, and healthcare providers with key roles in patient education were identified as important actors. The CFIR-ERIC Matching Tool resulted in 50 strategies with Level 1 endorsement and 13 strategies with Level 2 endorsement. The final list of strategies included 1) purposefully re-examine the implementation, 2) tailor strategies, 3) change record systems, 4) conduct educational meetings, 5) distribute educational materials, 6) intervene with patients to enhance uptake and adherence, 7) centralize technical assistance, and 8) use advisory boards and workgroups.

CONCLUSION

We present a generalizable method that incorporates steps from Implementation Mapping, engages various knowledge users, and leverages implementation science frameworks to facilitate the development of an implementation strategy. An evaluation of implementation success using the implementation outcomes framework is underway.

Theoretical Domains Framework: A Bibliometric and Visualization Analysis from 2005-2023

Background

The Theoretical Domains Framework (TDF) is among the most extensively utilised foundational frameworks in implementation science. It was developed from 33 psychological theories, with the latest version identifying 14 domains encompassing 84 theoretical constructs. These domains and constructs capture the complexity of factors that affect behaviours, making the framework a valuable tool for designing and implementing interventions within health and social care settings.

Objective

To summarise the development, hot topics, and future trends in TDF-related research and provide implementation practitioners with more information on the application of TDF.

Methods

We used TDF as the topic and searched the ISI Web of Science Core Collection, identifying 1382 relevant publications. We used analytical tools such as Excel, Tableau, VOSviewer, and Citespace to conduct a bibliometric analysis of the relevant publication.

Results

We identified the United Kingdom as the primary contributor, with University College London as the key institution. Susan Michie ranked highest in total citations. The analysis highlighted cancer and stroke as primary clinic medicine-related topics using TDF. Emerging themes encompass abuse, violence, maternal health, antenatal care, patient involvement, and trauma-informed care et al. "Nurse" and "qualitative research" emerged as recent and enduring hotspots, possibly indicating future research trends.

Conclusion

This article represents the first attempt to summarise the TDF using bibliometric analysis. We suggest this method can be used to analyse other theoretical frameworks in scientific implementation of its objectivity and quantifiability. Overall, the application scope of TDF is shifting from public health towards more specialised clinical directions, although its application in the field of public health is continuously expanding. In the future, the number of users of TDF is also expected to expand from implementation scientists to professional technical personnel.

Barriers to and enablers of the use of the Otology Questionnaire Amsterdam in clinical practice-a qualitative post-implementation study

BACKGROUND

The Otology Questionnaire Amsterdam (OQUA) is developed to evaluate multiple ear complaints and their impact on patients' daily lives. The current clinical use of this questionnaire is below the potential utilization.

AIM

To identify the barriers and enablers of using the OQUA as perceived by ENT surgeons and patients and provide recommendations for an implementation strategy.

METHODS

Prospective and qualitative analysis was performed using focus groups and interviews with ENT professionals (n = 15) and patients (n = 25) with ear complaints of one tertiary referral hospital and two regional hospitals. Barriers and enablers were identified and classified by using the Capability-Opportunity-Motivation-Behavior model and the Theoretical Domains Framework. Suggestions for an implementation strategy will be made accordingly.

RESULTS

ENT professionals' barriers included lack of knowledge and skills to use the OQUA, inadequate technological support and perceived time constraints during consultation, uncertainty about the clinical relevance and lack of feedback on the outcomes of the OQUA. Enablers included beneficial consequences of the OQUA for the professional, organization and science. Patients' barriers included lack of knowledge about the objective and usefulness of the OQUA, perceived burden, difficulties in completing the questionnaire and insufficient feedback during consultation. Patient enablers included beliefs about beneficial consequences of the OQUA for the patient, health care and society. Suggested interventions involved education, training, environmental restructuring and incentivisation.

CONCLUSION

Based on the findings, we propose an implementation strategy should focus on education and training about the objective, outcomes and relevance of the OQUA, environmental restructuring regarding the optimal use of the OQUA, and incentivisation with feedback on the valuable outcomes of the OQUA for the patient, professional and healthcare. Future research is needed to determine the feasibility of the implementation strategy.

Translation and linguistic validation of 24 PROMIS item banks into French

PURPOSE

The Patient-Reported Outcome Measurement Information System (PROMIS®) was developed to provide reliable, valid, and normed item banks to measure health. The item banks provide standardized scores on a common metric allowing for individualized, brief assessment (computerized adaptive tests), short forms (e.g. heart failure specific), or profile assessments (e.g. PROMIS-29). The objective of this study was to translate and linguistically validate 24 PROMIS adult item banks into French and highlight cultural nuances arising during the translation process.

METHODS

We used the FACIT translation methodology. Forward translation into French by two native French-speaking translators was followed by reconciliation by a third native French-speaking translator. A native English-speaking translator fluent in French then completed a back translation of the reconciled version from French into English. Three independent reviews by bilingual translators were completed to assess the clarity and consistency of terminology and equivalency across the English source and French translations. Reconciled versions were evaluated in cognitive interviews for conceptual and linguistic equivalence.

RESULTS

Twenty-four adult item banks were translated: 12 mental health, 10 physical health, and two social health. Interview data revealed that 577 items of the 590 items translated required no revisions. Conceptual and linguistic differences were evident for 11 items that required iterations to improve conceptual equivalence and two items were revised to accurately reflect the English source.

CONCLUSION

French translations of 24 item banks were created for routine clinical use and research. Initial translation supported conceptual equivalence and comprehensibility. Next steps will include validation of the item banks.

Electronic Collection of Patient-Reported Outcomes to Improve Kidney Care: Benefits, Drawbacks, and Next Steps

Kidney services worldwide are increasingly using digital health technologies to deliver care. This includes kidney electronic patient-reported outcome (ePRO) systems: ambulatory digital technologies that enable the capture of PRO data electronically from people with kidney disease remotely and in real time to be shared with their kidney care team. Current kidney ePRO systems commonly aim to support the monitoring and management of symptoms in patients with kidney disease. The majority have thus far only been implemented in research settings and are not yet routinely used in clinical practice, leaving their readiness for real-world implementation largely unknown. Compared with paper-based PRO collection, ePRO systems have certain advantages, which we categorize as efficiency benefits (e.g., lower administrative burden), direct patient care benefits (e.g., automated PRO-based patient education), and health system and research benefits (e.g., collecting ePRO data once for multiple purposes). At the same time, kidney ePRO systems come with drawbacks, such as their potential to exacerbate existing inequities in care and outcomes and to negatively affect staff burden and patients' experience of kidney care. Areas that hold promise for expediting the development and uptake of kidney ePRO systems at the local, organizational, and national level include harnessing national kidney registries as enabling infrastructures; using novel data-driven technologies (e.g., computerized adaptive test systems, configurable dashboards); applying implementation science and action research approaches to enhance translation of ePRO research findings into clinical practice; and engaging stakeholders, including patients and carers, health care professionals, policymakers, payers, ePRO experts, technology providers, and organizations that monitor and improve the quality of kidney services.

Developing feasible and acceptable strategies for integrating the use of patient-reported outcome measures (PROMs) in gender-affirming care: An implementation study

OBJECTIVE

Use CFIR guidance to create comprehensive, evidence-based, feasible, and acceptable gender-affirming care PROM implementation strategies.

DESIGN, SETTING, PARTICIPANTS

A 3-Phase participatory process was followed to design feasible and acceptable strategies for integrating PROMs in gender-affirming care. In Phase 1, barriers and enablers to PROM implementation for gender-affirming care were identified from a previous systematic review and our prior qualitative study. We used the CFIR-ERIC tool to match previously identified barriers and enablers with expert-endorsed implementation strategies. In Phase 2, implementation strategy outputs from CFIR-ERIC were organised according to cumulative percentage value. In Phase 3, gender-affirming care PROM implementation strategies underwent iterative refinement based on rounds of stakeholder feedback with seven patient and public partners and a gender-affirming healthcare professional.

RESULTS

The systematic review and qualitative study identified barriers and enablers to PROM implementation spanning all five CFIR domains, and 30 CFIR constructs. The top healthcare professional-relevant strategies to PROM implementation from the CFIR-ERIC output include: identifying and preparing implementation champions, collecting feedback on PROM implementation, and capturing and sharing local knowledge between clinics on implementation. Top patient-relevant strategies include: having educational material on PROMs, ensuring adaptability of PROMs, and collaborating with key local organisations who may be able to support patients.

CONCLUSIONS

This study developed evidence-based, feasible, and acceptable strategies for integrating PROMs in gender-affirming care, representing evidence from a systematic review of 286 international articles, a qualitative study of 24 gender-affirming care patients and healthcare professionals, and iteration from 7 patient and public partners and a gender-affirming healthcare professional. The finalised strategies include patient- and healthcare professional-relevant strategies for implementing PROMs in gender-affirming care. Clinicians and researchers can select and tailor implementation strategies best applying to their gender-affirming care setting.

Usability of an Automated System for Real-Time Monitoring of Shared Decision-Making for Surgery: Mixed Methods Evaluation

BACKGROUND

Improving shared decision-making (SDM) for patients has become a health policy priority in many countries. Achieving high-quality SDM is particularly important for approximately 313 million surgical treatment decisions patients make globally every year. Large-scale monitoring of surgical patients' experience of SDM in real time is needed to identify the failings of SDM before surgery is performed. We developed a novel approach to automating real-time data collection using an electronic measurement system to address this. Examining usability will facilitate its optimization and wider implementation to inform interventions aimed at improving SDM.

OBJECTIVE

This study examined the usability of an electronic real-time measurement system to monitor surgical patients' experience of SDM. We aimed to evaluate the metrics and indicators relevant to system effectiveness, system efficiency, and user satisfaction.

METHODS

We performed a mixed methods usability evaluation using multiple participant cohorts. The measurement system was implemented in a large UK hospital to measure patients' experience of SDM electronically before surgery using 2 validated measures (CollaboRATE and SDM-Q-9). Quantitative data (collected between April 1 and December 31, 2021) provided measurement system metrics to assess system effectiveness and efficiency. We included adult patients booked for urgent and elective surgery across 7 specialties and excluded patients without the capacity to consent for medical procedures, those without access to an internet-enabled device, and those undergoing emergency or endoscopic procedures. Additional groups of service users (group 1: public members who had not engaged with the system; group 2: a subset of patients who completed the measurement system) completed user-testing sessions and semistructured interviews to assess system effectiveness and user satisfaction. We conducted quantitative data analysis using descriptive statistics and calculated the task completion rate and survey response rate (system effectiveness) as well as the task completion time, task efficiency, and relative efficiency (system efficiency). Qualitative thematic analysis identified indicators of and barriers to good usability (user satisfaction).

RESULTS

A total of 2254 completed surveys were returned to the measurement system. A total of 25 service users (group 1: n=9; group 2: n=16) participated in user-testing sessions and interviews. The task completion rate was high (169/171, 98.8%) and the survey response rate was good (2254/5794, 38.9%). The median task completion time was 3 (IQR 2-13) minutes, suggesting good system efficiency and effectiveness. The qualitative findings emphasized good user satisfaction. The identified themes suggested that the measurement system is acceptable, easy to use, and easy to access. Service users identified potential barriers and solutions to acceptability and ease of access.

CONCLUSIONS

A mixed methods evaluation of an electronic measurement system for automated, real-time monitoring of patients' experience of SDM showed that usability among patients was high. Future pilot work will optimize the system for wider implementation to ultimately inform intervention development to improve SDM.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2023-079155.

Using Implementation Science Frameworks to Guide the Use of Electronic Patient-Reported Outcome Symptom Monitoring in Routine Cancer Care

PURPOSE

Electronic patient-reported outcomes (ePROs) are an evidence-based means of detecting symptoms earlier and improving patient outcomes. However, there are few examples of successful implementation in routine cancer care. We conducted a qualitative study to identify barriers and facilitators to implementing ePRO symptom monitoring in routine cancer care using the Consolidated Framework for Implementation Research (CFIR).

METHODS

Participants were adult patients with cancer, their caregivers, or health care professionals involved in ePRO monitoring or processes. Focus groups or individual interviews were conducted using a semistructured approach informed by the CFIR. Data were analyzed deductively using the CFIR. Barriers were matched to theory-informed implementation strategies using the CFIR-Expert Recommendations for Implementing Change (ERIC) matching tool.

RESULTS

Thirty participants were interviewed: 22 females (73%), aged 31-70 years (28, 94%), comprising patients (n = 8), caregivers (n = 2), medical oncologists (n = 4), nurses (n = 4), hospital leaders (n = 6), clinic administrators (n = 2), pharmacists (n = 2), and information technology specialists (n = 2). Barriers pertaining to four CFIR domains were identified and several were novel, including the challenge of adapting ePROs for different anticancer treatments. Facilitators pertaining to all CFIR domains were identified, such as leveraging acceptability of remote care post-COVID-19 to drive implementation. Conducting consensus discussions with stakeholders to tailor ePROs to the local setting, identifying/preparing individual and group-level champions, and assessing readiness for change (including leveraging technological advances and increased confidence in using remote monitoring post-COVID-19) were the most frequently recommended implementation strategies.

CONCLUSION

The CFIR facilitated identification of known and novel barriers and facilitators to implementing ePRO symptom monitoring in routine cancer care. Implementation strategies summarized in a conceptual framework will be used to codesign an ePRO symptom monitoring system for immunotherapy side effects.

Patient and healthcare professional perspectives on implementing patient-reported outcome measures in gender-affirming care: a qualitative study

OBJECTIVES

Patient and healthcare professional perspectives are needed to develop a gender-affirming care patient-reported outcome measure (PROM) implementation plan. We aimed to identify top considerations relevant to gender-affirming care PROM implementation from patient and healthcare professional perspectives.

DESIGN, SETTINGS AND PARTICIPANTS

This qualitative study conducted in the UK between January and April 2023 includes focus groups with a patient sample diverse in age and gender identity, and a healthcare professional sample diverse in age and role. Established methods in implementation science and the Consolidated Framework for Implementation Research were used to create interview guides, and analyse data. Focus groups were audio recorded, transcribed verbatim and analysed by two independent researchers. Patient and healthcare professional focus groups were conducted separately.

PRIMARY OUTCOME MEASURES

Patient and healthcare professional perspectives on PROM implementation were explored through focus groups and until data saturation.

RESULTS

A total of 7 virtual focus groups were conducted with 24 participants (14 patients, mean (SD) age, 43 (14.5); 10 healthcare professionals, mean (SD) age, 46 (11.3)). From patient perspectives, key barriers to PROM implementation were mistrust with PROMs, lack of accessibility, burden, and lack of communication on why PROMs are important and how they will help care. From healthcare professional perspectives, key barriers to PROM implementation were lack of accessibility, burden with PROM administration and scoring, costs of implementation (financial and time), and lack of communication on what PROMs are and how they benefit service provision.

CONCLUSION

Gender-affirming care PROM implementation must address: patient mistrust with PROMs, accessibility, communication on what PROMs are and how they can be used, reducing burden, and hybridised implementation. These factors may also be applicable to other clinical areas interested in implementing PROMs.

Mapping the Theoretical Domain Framework to the Consolidated Framework for Implementation Research: do multiple frameworks add value?

BACKGROUND

Implementation researchers often combine the Theoretical Domain Framework (TDF) and Consolidated Framework for Implementation Research (CFIR) in their studies. However there is some debate on the merits of using multiple frameworks-whether they contribute to results or provide superfluous analysis. Our recent research combined the TDF and CFIR to identify determinants to widespread incorporation of patient held medication lists (PHML) in healthcare practice. The aim of this report is to provide guidance on the use of the TDF and CFIR; by assessing the degree of overlap between the two frameworks in their application to interviews about PHML.

METHODS

Semi-structured telephone interviews were conducted with healthcare professionals (HCPs) and non HCPs (people taking multiple medicines and caregivers).Interview data were transcribed and analysed using the TDF and CFIR. Within paired domains substantial intersection/overlap across constructs and domains within the two frameworks was classified as > 75% of coding references, consistent intersection/overlap was defined as > 50% and ≤ 75%, average intersection/overlap was defined as ≤ 50% and > 25% and non-substantial intersection/overlap was classified as ≤ 25% of coding references.

RESULTS

Interview data were collected from 39 participants - 21 HCPs and 18 non HCPs. Mapping of TDF domains to CFIR domains/constructs identified key determinants in six TDF domains: Environmental context & resources, Beliefs about capabilities, Beliefs about consequences, Social influences, Behavioural regulation and Social/professional role & identity; and five CFIR domains: Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individual and Process. A pattern of substantial intersection/overlap in coding emerged with broad TDF domains such as Environmental context & resources often linked to well-defined CFIR domains and constructs (e.g. design quality & packaging within Intervention Characteristics). Broad CFIR constructs such as knowledge & beliefs about intervention within Characteristics of Individuals also linked to more descriptive TDF domains like Beliefs about capabilities. In addition there was some unexpected non-substantial intersection/overlap in coding with the TDF domain Social influences less frequently linked to the CFIR Inner Setting domain and constructs such as networks and communications.

CONCLUSIONS

Identifying intersections/overlaps in coding between CFIR and TDF can assist interpretation of findings in implementation research. The strengths of each framework were exploited in a reciprocal process which provided more information to broad/poorly defined domains and enabled identification of implementation determinants and innovation determinants.

The Feasibility, Acceptability, and Effectiveness of Electronic Patient-Reported Outcome Symptom Monitoring for Immune Checkpoint Inhibitor Toxicities: A Systematic Review

PURPOSE

Increasing use of immune checkpoint inhibitors (ICIs) in routine cancer care will increase the incidence of immune-related adverse events (irAEs). Systems are needed to support remote monitoring for irAEs. Electronic patient-reported outcome (ePRO) symptom monitoring systems can help monitor and manage symptoms and side effects. We assessed the content and features of ePRO symptom monitoring systems for irAEs, and their feasibility, acceptability, and impact on patient outcomes and health care utilization.

METHODS

A systematic literature search was conducted in May 2022 on MEDLINE, Embase, PsycINFO, and Cochrane Central Register of Controlled Trials. Quantitative and qualitative data relevant to the review questions were extracted and synthesized in tables.

RESULTS

Seven papers describing five ePRO systems were included. All systems collected PROs between clinic visits. Two of five used validated symptom questionnaires, 3/5 provided prompts to complete questionnaires, 4/5 provided reminders to self-report, and 3/5 provided clinician alerts for severe/worsening side effects. Four of five provided coverage of ≥26/30 irAEs in the ASCO irAE guideline. Feasibility and acceptability were demonstrated with consent rates of 54%-100%, 17%-27% of questionnaires generating alerts, and adherence rates of 74%-75%. One paper showed a reduction in grade 3-4 irAEs, treatment discontinuation, clinic visit duration, and emergency department presentations, while another showed no difference in these outcomes or the rate of steroid use.

CONCLUSION

There is preliminary evidence of the feasibility and acceptability of ePRO symptom monitoring for irAEs. However, further studies are needed to confirm the impact on ICI-specific outcomes, such as the frequency of grade 3-4 irAEs and duration of immunosuppression. Suggestions for the content and features of future ePRO systems for irAEs are provided.

Building an infrastructure to support the development, conduct, and reporting of informative clinical studies: The Rockefeller University experience

Introduction

Clinical trials are a vital component of translational science, providing crucial information on the efficacy and safety of new interventions and forming the basis for regulatory approval and/or clinical adoption. At the same time, they are complex to design, conduct, monitor, and report successfully. Concerns over the last two decades about the quality of the design and the lack of completion and reporting of clinical trials, characterized as a lack of "informativeness," highlighted by the experience during the COVID-19 pandemic, have led to several initiatives to address the serious shortcomings of the United States clinical research enterprise.

Methods and Results

Against this background, we detail the policies, procedures, and programs that we have developed in The Rockefeller University Center for Clinical and Translational Science (CCTS), supported by a Clinical and Translational Science Award (CTSA) program grant since 2006, to support the development, conduct, and reporting of informative clinical studies.

Conclusions

We have focused on building a data-driven infrastructure to both assist individual investigators and bring translational science to each element of the clinical investigation process, with the goal of both generating new knowledge and accelerating the uptake of that knowledge into practice.

PROMs and PREMs in routine perinatal care: mixed methods evaluation of their implementation into integrated obstetric care networks

BACKGROUND

In the transition towards value-based healthcare, patient-reported outcome and experience measures (PROM and PREM) are recommended by international collaborations and government programs to guide clinical practice and quality improvement. For many conditions, using PROM/PREM over the complete continuum of care requires implementation across care organizations and disciplines. Along PROM/PREM implementation in obstetric care networks (OCN), we aimed to evaluate implementation outcomes and the processes influencing these outcomes in the complex context of care networks across the continuum of perinatal care.

METHODS

Three OCN in the Netherlands implemented PROM/PREM in routine practice, using an internationally developed outcomes set with care professionals and patient advocates. Their aim was to use PROM/PREM results individually to guide patient-specific care decisions and at group-level to improve quality of care. The implementation process was designed following the principles of action research: iteratively planning implementation, action, data generation and reflection to refine subsequent actions, involving both researchers and care professionals. During the one-year implementation period in each OCN, implementation outcomes and processes were evaluated in this mixed-methods study. Data generation (including observation, surveys and focus groups) and analysis were guided by two theoretical implementation frameworks: the Normalization Process Theory and Proctor's taxonomy for implementation outcomes. Qualitative findings were supplemented with survey data to solidify findings in a broader group of care professionals.

RESULTS

Care professionals in OCN found the use of PROM/PREM acceptable and appropriate, recognized their benefits and felt facilitated in their patient-centered goals and vision. However, feasibility for daily practice was low, mainly due to IT issues and time constraints. Hence PROM/PREM implementation did not sustain, but strategies for future PROM/PREM implementation were formulated in all OCN. Processes contributing positively to implementation outcomes were internalization (understand the value) and initiation (driven by key-participants), whereas challenges in relational integration (maintain confidence) and reconfiguration (refine activities) affected implementation negatively.

CONCLUSION

Although implementation did not sustain, network-broad PROM/PREM use in clinic and quality improvement matched professionals' motivation. This study provides recommendations to implement PROM/PREM meaningfully in practice in ways that support professionals in their drive towards patient-centered care. In order for PROM/PREM to fulfill their potential for value-based healthcare, our work highlights the need for sustainable IT infrastructures, as well as an iterative approach to refine their complex implementation into local contexts.

Factors affecting implementation of patient-reported outcome and experience measures in a pediatric health system

BACKGROUND

The use of patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) in pediatric clinical practice can enhance clinical care and bring children and families' perspectives into evaluations of healthcare services. Implementing these measures is complex and requires a thorough assessment of the context of implementation The purpose of this study is to describe the barriers and facilitators to PROMs and PREMs implementation and to recommend strategies for implementing these measures in a pediatric health system.

METHODS

We used a qualitative descriptive approach to analyse data from interviews to understand the experiences of PROMs and PREMs users across different pediatric settings in a single Canadian healthcare system.

RESULTS

There were 23 participants representing a variety of roles within the healthcare system and pediatric populations. We found five main factors that affected implementation of PROMs and PREMs in pediatric settings: 1) Characteristics of PROMs and PREMs; 2) Individual's beliefs; 3) Administering PROMs and PREMs; 4) Designing clinical workflows; and 5) Incentives for using PROMs and PREMs. Thirteen recommendations for integrating PROMs and PREMs in pediatric health settings are provided.

CONCLUSIONS

Implementing and sustaining the use of PROMs and PREMs in pediatric health settings presents several challenges. The information presented will be useful for individuals who are planning or evaluating the implementation of PROMs and PREMs in pediatric settings.

Stepping into the real world: a mixed-methods evaluation of the implementation of electronic patient reported outcomes in routine lung cancer care

Background

To realize the broader benefits of electronic patient-reported outcome measures (ePROMs) in routine care, we used the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework to inform the translation of a clinically effective ePROM system (hereafter referred to as the PRM system) into practice. The study aimed to evaluate the processes and success of implementing the PRM system in the routine care of patients diagnosed with lung cancer.

Method

A controlled before-and-after mixed-methods study was undertaken. Data sources included a self-report questionnaire and interviews with healthcare providers, electronic health record data for PRMs patients and historical controls, and field notes. Descriptive statistics, logistic regression modelling, negative binomial models, generalized estimating equations and repeated measures ANOVA were used to analyze quantitative data. Qualitative data was thematically analyzed.

Results

A total of 48/79 eligible people diagnosed with lung cancer completed 90 assessments during the 5-month implementation period (RE-AIM reach). Every assessment breached the pre-defined threshold and care coordinators reviewed and actioned 95.6% of breaches, resulting in 146 referrals to allied health services, most frequently for social work (25.3%), dietetics (18.5%), physiotherapy (18.5%) and occupational therapy (17.1%). PRMs patients had significantly fewer visits to the cancer assessment unit for problematic symptoms (M = 0.23 vs. M = 0.43; p = 0.035), and were significantly more likely to be offered referrals (71% vs. 29%, p < 0.0001) than historical controls (RE-AIM effect). The levels of ‘organizational readiness for implementing change’ (ORIC) did not show much differences between baseline and follow-up, though this was already high at baseline; but significantly more staff reported improved confidence when asking patients to complete assessments (64.7% at baseline vs. 88.2% at follow-up, p = 0.0046), and when describing the assessment tool to patients (64.7% at baseline vs. 76.47% at follow-up, p = 0.0018) (RE-AIM adoption). A total of 78 staff received PRM system training, and 95.6% of the PRM system alerts were actioned (RE-AIM implementation); and all lung cancer care coordinators were engaged with the PRM system beyond the end of the study period (RE-AIM maintenance).

Conclusion

This study demonstrates the potential of the PRM system in enhancing the routine care of lung cancer patients, through leveraging the capabilities of automated web-based care options.

Plain English summary

Research has shown the clear benefits of using electronically collected patient-reported outcome measures (ePROMs) for cancer patients and health services. However, we need to better understand how to implement ePROMs as part of routine care. This study evaluated the processes and outcomes of implementing an ePROMs system in the routine care of patients diagnosed with lung cancer. Key findings included: (a) a majority of eligible patients completed the scheduled assessments; (b) patient concerns were identified in every assessment, and care coordinators reviewed and actioned almost all of these, including making significantly more referrals to allied health services; (c) patients completing assessments regularly were less likely to present to the cancer assessment unit with problematic symptoms, suggesting that ePROMs identified patient concerns early and this led to a timely response to concerns; (d) staff training and engagement was high, and staff reporting increased confidence when asking patients to complete assessments and when describing the assessment tool to patients at the end of the implementation period. This study shows that implementing ePROMs in routine care is feasible and can lead to improvements in patient care.

Misconceiving patient reported outcome measures (PROMs) as primarily a reporting requirement rather than a quality improvement tool: perceptions of independent healthcare sector stakeholders in the UK

Background

The independent healthcare sector in the UK collects PROMs for several surgical procedures, but implementation has been challenging. We aimed to understand the enablers and barriers to PROMs implementation in the independent healthcare sector in the UK.

Method

Between January and May 2021, we remotely conducted semi-structured interviews with hospital consultants, hospital managers and other clinical staff using a topic guide developed from an implementation science framework called the Theoretical Domains Framework (TDF).

Results

We interviewed 6 hospital consultants, 5 hospital managers, and 3 other clinical staff (1 nurse and 2 physiotherapists) across 8 hospitals. Common barriers included: the perception that PROMs are predominantly a reporting requirement rather than a quality improvement tool, absence of feedback mechanisms for PROMs data for clinicians, poor awareness of PROMs among healthcare professionals and the public, absence of direction or commitment from leadership, and limited support from hospital consultants. Common enablers included: regular feedback of PROMs data to clinicians, designating roles and responsibilities, formally embedding PROMs collection into patient pathways, and involvement of hospital consultants in developing strategies to improve PROMs uptake.

Conclusion

To support PROMs implementation, independent hospitals need to develop long-term organisational strategies that involve sustained leadership commitment, goals or targets, training opportunities to staff, and regular feedback of PROMs data at clinical or governance meetings. The primary purpose of PROMs needs to be reframed to independent healthcare sector stakeholders as a quality improvement tool rather than a reporting requirement.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41687-022-00511-5.

Facilitators, barriers and considerations for the implementation of healthcare innovation: A qualitative rapid systematic review

Family caregiving scholars recommend that health providers receive competency-based education to partner with and support family caregivers to care and to maintain their own health. While it may be relatively easy to develop competency-based education for healthcare providers, ensuring widespread uptake and spread and scale of healthcare education is critical to ensuring consistent person-centered support for all family caregivers (FCGs) throughout the care trajectory. The development of novel healthcare innovations requires implementation strategies for uptake and spread, with implementation involving the use of strategies to integrate a novel innovation into healthcare. Research suggests that there are many factors involved in successful implementation and a synthesis of potential factors is warranted. The purpose of this review is to provide an in-depth examination of facilitators, barriers and considerations for implementation of a novel healthcare innovation that will be used to develop an implementation plan for spread and scale of our competency-based education for health providers to learn about person-centered care for FCGs. A systematic review of published and grey literature was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA [Moher et al., 2015]) guidelines. The systematic review involved searching four databases for original research articles that described barriers, facilitators and/or other considerations when implementing innovations. Twenty-eight articles were included in the qualitative thematic analyses and described three areas of implementation research: barriers, facilitators and recommendations. There were major and parallel themes that emerged under facilitators and barriers. There were a wide variety of strategies that were identified as recommendations. The findings were synthesised into five considerations for implementation: Research and information sharing, intentional implementation planning, organisational underpinnings, creating the clinical context and facilitative training. This review provides an integrative overview of identified facilitators, barriers and recommendations for implementation that may aid in developing implementation strategies that can be tailored to the local context or innovation being implemented.

Improving the patient-centred care of children with life-altering skin conditions using feedback from electronic patient-reported outcome measures: protocol for a hybrid effectiveness-implementation study (PEDS-ePROM)

INTRODUCTION

Using patient-reported outcome measures (PROMs) with children have been described as 'giving a voice to the child'. Few studies have examined the routine use of these measures as potentially therapeutic interventions. This study aims to investigate: (1) the effectiveness of feedback using graphical displays of information from electronic PROMs (ePROMs) that target health-related quality of life, to improve health outcomes, referrals and treatment satisfaction and (2) the implementation of ePROMs and graphical displays by assessing acceptability, sustainability, cost, fidelity and context of the intervention and study processes.

METHODS AND ANALYSIS

A hybrid II effectiveness-implementation study will be conducted from February 2020 with children with life-altering skin conditions attending two outpatient clinics at a specialist paediatric children's hospital. A pragmatic randomised controlled trial and mixed methods process evaluation will be completed. Randomisation will occur at the child participant level. Children or parent proxies completing baseline ePROMs will be randomised to: (1) completion of ePROMs plus graphical displays of ePROM results to treating clinicians in consultations, versus (2) completion of ePROMs without graphical display of ePROM results. The primary outcome of the effectiveness trial will be overall health-related quality of life of children. Secondary outcomes will include other health-related quality of life outcomes (eg, child psychosocial and physical health, parent psychosocial health), referrals and treatment satisfaction. Trial data will be primarily analysed using linear mixed-effects models; and implementation data using inductive thematic analysis of interviews, meeting minutes, observational field notes and study communication mapped to the Consolidated Framework for Implementation Research.

ETHICS AND DISSEMINATION

Ethical approval was obtained from Children's Health Queensland Human Research Ethics Committee (HREC/2019/QCHQ/56290), The University of Queensland (2019002233) and Queensland University of Technology (1900000847). Dissemination will occur through stakeholder groups, scientific meetings and peer-reviewed publications.

TRIAL REGISTRATION NUMBER

Australian New Zealand Clinical Trials Registry (ACTRN12620000174987).