Physiology education and teaching in Chinese mainland medical schools: the status quo and the changes over the past two decades

Physiology is a critical subject that bridges basic and clinical medicine. The reform of physiology education is crucial to improving the quality of teaching and supporting student learning; however, there is a lack of comprehensive data documenting the current state of physiology education in medical programs in China. This study conducted an online survey among experienced teachers from the top 100 medical schools in the Chinese mainland in 2022. A total of 89 valid questionnaires were included in this study. In 57.3% of the surveyed medical schools, 70% of the physiology teachers have a Ph.D. degree, while the number of faculty members with a medical background has tended to decrease in 53.9% of the surveyed medical schools. Of the 89 medical schools, 62 have implemented physiology course integration, with 69.4% of the respondents believing that curriculum integration could enhance students' knowledge acquisition and application. The lecture hours per term are 50-70 and 71-90 in 49.4% and 42.7% of the 89 medical schools, respectively. The ratio of course duration of lecture to laboratory was 2:1 in 47.2% of the surveyed medical schools, while the ratio was between 1:1 and 2:1 in 47.2% of the surveyed medical schools, and only 5.6% of medical schools opted for a 1:1 course arrangement. For teaching methods, about 60% of medical schools used problem-based learning, case-based learning, and task-based learning (PBL/CBL/TBL) for clinical medicine students only, and 25.4% of medical schools applied PBL/CBL/TBL to all students. This study is conducive to promoting dialogue and communication among physiology teachers, administrators, and researchers.NEW & NOTEWORTHY This study conducted a nationwide survey of the top 100 medical schools in the Chinese mainland in 2022 to provide first-hand data on the current situation and changes in physiology education and teaching in the Chinese mainland with focuses on the faculty composition, curriculum integration, teaching methods, and assessment modes, which is a microcosm of the reform and development of the medical education in the Chinese mainland, hoping to improve the quality of preclinical medical education.

SWOT Analysis and Recommendations for Community Health Workers and Stakeholders Responding to COVID-19 Health Inequities

Background. By 2023, 1,080,000 cases of COVID-19 have been reported in Harris County. Systemic inequity and vaccine hesitancy have contributed to COVID-19 disparities. Community Health Workers provide health education and instrumental support to alleviate health disparities among vulnerable communities. We conducted an analysis of Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis in June 2022 among a broad coalition of Community Health Work stakeholders to better understand the local landscape in the context of the COVID-19 pandemic. Methods. We recruited 33 community health workers and industry stakeholders in Harris County, Texas, to participate in the SWOT analysis. Participants were asked to describe their opinions on the SWOT facing the Community Health Work landscape and then rank the outcomes of the analysis to prioritize action. Results. A total of 19 themes were identified. Weaknesses included lack of respect and resources for Community Health Workers and poor coordination and capacity among the workforce infrastructure. Limited funding and lack of appreciation for Community Health Workers were deemed important threats. Diversity and community connection were critical strengths, and strong education, training, and raising awareness for community health work were considered opportunities to overcome identified weaknesses and threats. Discussion. Increased funding, greater coordination, greater respect, and amplified training can improve capacity for Community Health Workers and, therefore, improve public health outcomes for respiratory illness and viral infections such as COVID-19. This analysis helps fill an important research gap on the topic Community Health Workers responding to public health crises with racially disparate outcomes.

Internet-Delivered Cognitive Behavioral Therapy Tailored to Spouses and Significant Others of Public Safety Personnel: Formative Evaluation Study

BACKGROUND

Spouses and significant others (SSOs) of public safety personnel (PSP) are affected by the risks and requirements of these occupations. Internet-delivered cognitive behavioral therapy (ICBT) provides a convenient and accessible treatment format that can be tailored to the needs of SSOs of PSP.

OBJECTIVE

This study aimed to assess the initial use and client perceptions (eg, likes, helpfulness, and areas for improvement) of a self-guided, transdiagnostic ICBT course designed for Canadian SSOs of PSP and identify opportunities to further tailor ICBT for this group.

METHODS

SSOs were invited to complete a 5-lesson, self-guided, transdiagnostic ICBT course. Descriptive statistics were used to analyze the demographic and clinical characteristics of participants. Content analysis was used to analyze the data from open-ended survey responses and interviews to understand their experiences with ICBT.

RESULTS

Clients (N=118) endorsed various mental health concerns (eg, depression, anxiety, posttraumatic stress symptoms, and relationship concerns) with a range of severity levels. Most clients identified as White (110/116, 94.8%) and women (108/116, 91.5%), with a mean age of 42.03 (SD 9.36) years. Of the 26 clients who were interviewed, 89% (23/26) reported believing that ICBT is helpful and 92% (24/26) reported finding at least 1 skill helpful. Clients provided suggestions for course improvements. On the basis of this feedback and quantitative data, changes were made to areas such as the delivery of materials, content, case stories, and timelines. Overall, the results indicated that many SSOs of PSP had positive perceptions of ICBT tailored to their needs and found several aspects of the course helpful, supporting the continued delivery of tailored ICBT to this population. However, there remains a need for continued promotion of the course and outreach to diverse groups of SSOs of PSP.

CONCLUSIONS

Findings from this formative evaluation provide insight into the unique experiences and needs of SSOs of PSP and provide preliminary evidence for the use of tailored ICBT to support the mental health of this group in Canada.

A Digital Coach (E-Supporter 1.0) to Support Physical Activity and a Healthy Diet in People With Type 2 Diabetes: Acceptability and Limited Efficacy Testing

BACKGROUND

A healthy lifestyle, including regular physical activity and a healthy diet, is increasingly part of type 2 diabetes (T2D) management. As many people with T2D have difficulty living and maintaining a healthy lifestyle, there is a need for effective interventions. eHealth interventions that incorporate behavior change theories and tailoring are considered effective tools for supporting a healthy lifestyle. The E-Supporter 1.0 digital coach contains eHealth content for app-based eHealth interventions and offers tailored coaching regarding physical activity and a healthy diet for people with T2D.

OBJECTIVE

This study aimed to assess the acceptability of E-Supporter 1.0 and explore its limited efficacy on physical activity, dietary behavior, the phase of behavior change, and self-efficacy levels.

METHODS

Over a span of 9 weeks, 20 individuals with T2D received daily motivational messages and weekly feedback derived from behavioral change theories and determinants through E-Supporter 1.0. The acceptability of the intervention was assessed using telephone-conducted, semistructured interviews. The interview transcripts were coded using inductive thematic analysis. The limited efficacy of E-Supporter 1.0 was explored using the Fitbit Charge 2 to monitor step count to assess physical activity and questionnaires to assess dietary behavior (using the Dutch Healthy Diet index), phase of behavior change (using the single-question Self-Assessment Scale Stages of Change), and self-efficacy levels (using the Exercise Self-Efficacy Scale).

RESULTS

In total, 5 main themes emerged from the interviews: perceptions regarding remote coaching, perceptions regarding the content, intervention intensity and duration, perceived effectiveness, and overall appreciation. The participants were predominantly positive about E-Supporter 1.0. Overall, they experienced E-Supporter 1.0 as a useful and easy-to-use intervention to support a better lifestyle. Participants expressed a preference for combining E-Supporter with face-to-face guidance from a health care professional. Many participants found the intensity and duration of the intervention to be acceptable, despite the coaching period appearing relatively short to facilitate long-term behavior maintenance. As expected, the degree of tailoring concerning the individual and external factors that influence a healthy lifestyle was perceived as limited. The limited efficacy testing showed a significant improvement in the daily step count (z=-2.040; P=.04) and self-efficacy levels (z=-1.997; P=.046) between baseline and postintervention. Diet was improved through better adherence to Dutch dietary guidelines. No significant improvement was found in the phase of behavior change (P=.17), as most participants were already in the maintenance phase at baseline.

CONCLUSIONS

On the basis of this explorative feasibility study, we expect E-Supporter 1.0 to be an acceptable and potentially useful intervention to promote physical activity and a healthy diet in people with T2D. Additional work needs to be done to further tailor the E-Supporter content and evaluate its effects more extensively on lifestyle behaviors.

Team-Based Qualitative Rapid Analysis: Approach and Considerations for Conducting Developmental Formative Evaluation for Intervention Design

Qualitative rapid analysis is one of many rapid research approaches that offer a solution to the problem of time constrained health services evaluations and avoids sacrificing the richness of qualitative data that is needed for intervention design. We describe modifications to an established team-based, rapid analysis approach that we used to rapidly collect and analyze semi-structured interview data for a developmental formative evaluation of a cardiovascular disease prevention intervention. Over 18 weeks, we conducted and analyzed 35 semi-structured interviews that were conducted with patients and health care providers in the Veterans Health Administration to identify targets for adapting the intervention in preparation for a clinical trial. We identified 12 key themes describing actionable targets for intervention modification. We highlight important methodological decisions that allowed us to maintain rigor when using qualitative rapid analysis for intervention adaptation and we provide practical guidance on the resources needed to execute similar qualitative studies. We additionally reflect on the benefits and challenges of the described approach when working within a remote research team environment.ClinicalTrials.gov: NCT04545489.

Instruments to Assess Evidence-Based Practice Among Health Care Professionals: A Systematic Review

BACKGROUND

The use of measurement instruments to assess the use of Evidence-Based Practice by health professionals has been frequently reported in studies.

AIMS

This systematic review aimed to summarize, describe, and evaluate the measurement properties of the instruments that evaluate the use of Evidence-Based Practice in health professionals.

METHODS

The search was carried out in four databases considering three groups of search terms: evidence-based practice, evaluation, and measurement proprieties. Studies were included that described the use of instruments to assess Evidence-Based Practice in health professionals, with the full-text publication, which analyzed the measurement properties, in English. The methodological quality of the studies was evaluated using COnsensus-based Standards for the selection of health Measurement INstruments.

RESULTS

In total, 6,429 were found and only 92 were eligible for data analysis. Forty new instruments were identified most were developed for nursing and physical therapist. The investigators performed at least 1 type of validity test on 73% of the instruments. Reliability was tested at 90%, through internal consistency. Responsiveness was tested on less than half of the instruments (30%). Most of the instruments identified are reliable and valid to measure evidence-based practice in health professionals.

CONCLUSION

Although the Fresno Test remains the most complete instrument, and adequate for use. The COnsensus-based Standards for the selection of health Measurement INstruments checklist classified 7 (seven) instruments as suitable for the target audience.

Money Matters: A Three-Step Process for Using Budget Data in Program Evaluation to Assess the Design and Management of a Novel Public Health Program

We applied a three-step process, abstracting and analyzing program budgets to examine how Colorectal Cancer Control Program (CRCCP) awardees are structuring their programs and to assess the fidelity of program design to the CRCCP public health model. We reviewed 23 state, one tribal organization, and six university awardee budgets. We assessed resource allocations, staffing structures, and contracted partners and their activities. Awardees allocated 83% of all funds to contracts and personnel. Program managers were the most budgeted personnel type across three measures: number of people, full-time equivalency, and personnel costs. Awardees not only contracted with health care systems and clinics (39% of all contracts) but also contracted other partner types. Contractors were mainly funded to implement evidence-based interventions (25%) and conduct evaluation (24%). Program design varied among awardees in the number of staff (0-22), number of full-time equivalencies (0-5.4), and the number of contracts (1-11) budgeted. State awardees budgeted more resources to contracts, compared with university awardees (57% vs. 31%), while universities budgeted more for total personnel costs (41% vs. 30%). We learned that awardees designed their programs with fidelity to the CRCCP model. Although implementation approaches varied, overall results suggest implementation requires a combination of internal capacities and contracted partners. Budgets provide opportunities to use already existing program data to evaluate program design, partnerships, and planned activities.

Making Connections that Count - a Case Study of the Family Referral Service in Schools Program on the Central Coast, New South Wales, Australia

Introduction

Adverse childhood experiences (ACEs) are associated with health and social problems in later life, with an early intervention highly desirable for better outcomes.

Description

The Family-Referral-Services-In-Schools (FRSIS) is an early-intervention case management program for children and families with complex unmet needs, providing access to family support, housing, mental health care, and/or drug and alcohol services. The in-school trial setting was aimed at improving service uptake which was low in its community counterpart.

Discussion

FRSIS was a well-regarded intervention that reduced barriers to access for vulnerable families. The school setting and non-government agency service provision led to increased acceptability and trust. The program reached 5% of the student population. Support was tailored to family need, which was often complex and involved both children and caregivers. Initially, the multi-agency partnership and governance oversight group championed the service and enabled the pilot to be established, however funding uncertainty and competing priorities saw leadership support ebb away despite operational success.

Conclusion

The FRSIS model breaks down numerous barriers to accessing care for vulnerable families by its generalist nature and tailored approach and represents a high-trust approach to brokering appropriate care. Consistency in leadership support was a missed opportunity for program sustainability.

Development of a Web-Based Diabetes Prevention Program (DPP) for Chinese Americans: A Formative Evaluation Approach

Increasing evidence demonstrates that an online Diabetes Prevention Program (DPP) can delay the onset of type 2 diabetes. However, little has been done for Chinese Americans. This study, using Community-Based Participatory Research and Intervention Mapping approaches, describes a formative research process in the development of a culturally and linguistically tailored online DPP program among Chinese Americans with prediabetes living in New York City. Using a triangulation approach, data were collected to inform the development of an online DPP curriculum through (1) a literature review, (2) three focus groups (n = 24), and (3) a community advisory board meeting among 10 key informants knowledgeable in community needs, diabetes care, and lifestyle interventions. Participants indicated online DPPs would be very useful and easily accessible. However, key barriers including low computer skills/literacy and technology self-efficacy were identified. In addition, taking meal photos and tracking pedometer steps daily were found to be acceptable self-motoring tools for sustaining a healthy lifestyle. Furthermore, the integration of features such as text message reminders and the creation of social support groups into the online DPP curriculum was proposed to minimize attrition. This theory-based formative research to develop a culturally and linguistically appropriate web-based DPP curriculum was well-received by Chinese Americans and warrants testing in future intervention studies.

A Voice App Design for Heart Failure Self-management: Proof-of-Concept Implementation Study

BACKGROUND

Voice user interfaces are becoming more prevalent in health care and are commonly being used for patient engagement. There is a growing interest in identifying the potential this form of interface has on patient engagement with digital therapeutics (DTx) in chronic disease management. Making DTx accessible through an alternative interaction model also has the potential to better meet the needs of some patients, such as older adults and those with physical and cognitive impairments, based on existing research.

OBJECTIVE

This study aimed to evaluate how participants with heart failure interacted with a voice app version of a DTx, Medly, through a proof-of-concept implementation study design. The objective was to understand whether the voice app would enable the participants to successfully interact with the DTx, with a focus on acceptability and feasibility.

METHODS

A mixed methods concurrent triangulation design was used to better understand the acceptability and feasibility of the use of the Medly voice app with the study participants (N=20) over a 4-week period. Quantitative data included engagement levels, accuracy rates, and questionnaires, which were analyzed using descriptive statistics. Qualitative data included semistructured interviews and were analyzed using a qualitative descriptive approach.

RESULTS

The overall average engagement level was 73% (SD 9.5%), with a 14% decline between results of weeks 1 and 4. The biggest difference was between the average engagement levels of the oldest and youngest demographics, 84% and 43%, respectively, but these results were not significant-Kruskal-Wallis test, H(2)=3.8 (P=.14). The Medly voice app had an overall accuracy rate of 97.8% and was successful in sending data to the clinic. From an acceptability perspective, the voice app was ranked in the 80th percentile, and overall, the users felt that the voice app was not a lot of work (average of 2.1 on a 7-point Likert scale). However, the overall average score for whether users would use it in the future declined by 13%. Thematic analysis revealed the following: the theme feasibility of clinical integration had 2 subthemes, namely users adapted to the voice app's conversational style and device unreliability, and the theme voice app acceptability had 3 subthemes, namely the device integrated well within household and users' lives, users blamed themselves when problems arose with the voice app, and voice app was missing specific, desirable user features.

CONCLUSIONS

In conclusion, participants were largely successful in using the Medly voice app despite some of the barriers faced, proving that an app such as this could be feasible to be deployed in the clinic. Our data begin to piece together the patient profile this technology may be most suitable for, namely those who are older, have flexible schedules, are confident in using technology, and are experiencing other medical conditions.

Design of a Patient Voice App Experience for Heart Failure Management: Usability Study

BACKGROUND

The use of digital therapeutics (DTx) in the prevention and management of medical conditions has increased through the years, with an estimated 44 million people using one as part of their treatment plan in 2021, nearly double the number from the previous year. DTx are commonly accessed through smartphone apps, but offering these treatments through additional platforms can improve the accessibility of these interventions. Voice apps are an emerging technology in the digital health field; not only do they have the potential to improve DTx adherence, but they can also create a better user experience for some user groups.

OBJECTIVE

This research aimed to identify the acceptability and feasibility of offering a voice app for a chronic disease self-management program. The objective of this project was to design, develop, and evaluate a voice app of an already-existing smartphone-based heart failure self-management program, Medly, to be used as a case study.

METHODS

A voice app version of Medly was designed and developed through a user-centered design process. We conducted a usability study and semistructured interviews with patients with heart failure (N=8) at the Peter Munk Cardiac Clinic in Toronto General Hospital to better understand the user experience. A Medly voice app prototype was built using a software development kit in tandem with a cloud computing platform and was verified and validated before the usability study. Data collection and analysis were guided by a mixed methods triangulation convergence design.

RESULTS

Common themes were identified in the results of the usability study, which involved 8 participants with heart failure. Almost all participants (7/8, 88%) were satisfied with the voice app and felt confident using it, although half of the participants (4/8, 50%) were unsure about using it in the future. Six main themes were identified: changes in physical behavior, preference between voice app and smartphone, importance of music during voice app interaction, lack of privacy concerns, desired reassurances during voice app interaction, and helpful aids during voice app interaction. These findings were triangulated with the quantitative data, and it concluded that the main area for improvement was related to the ease of use; design changes were then implemented to better improve the user experience.

CONCLUSIONS

This work offered preliminary insight into the acceptability and feasibility of a Medly voice app. Given the recent emergence of voice apps in health care, we believe that this research offered invaluable insight into successfully deploying DTx for chronic disease self-management using this technology.

Applying formative evaluation in the mentoring of student intern nurses in an emergency department

Objective

To explore the effectiveness of formative evaluation in the mentoring of student nursing interns in an emergency department.

Methods

A total of 144 intern nursing students in the emergency department of a tertiary care hospital in Fuzhou were selected as the study subjects from July 2020 to February 2021. Adopting quasi-experimental studies methods, the students were divided into the experiment group (n = 74) and the control group (n = 70), based on their practicing rotation times. Formative evaluation methods such as in-person interviews, clinical scenario simulations, and clinical operation skills exams were conducted in the experiment group, while traditional summative evaluation methods were adopted for the control group. At the end of the intern period, a unified examination paper on professional knowledge concerning the emergency department, a cardiopulmonary resuscitation skill assessment, and a self-rating scale of self-directed learning was employed to evaluate professional theory performance, clinical practice ability, self-directed learning ability, and academic satisfaction among the nursing students, respectively.

Results

The professional theoretical performance, clinical practice ability assessment scores, academic satisfaction, and self-directed learning abilities of the nursing students were significantly higher in the experiment group compared with the control group (P < 0.05).

Conclusion

The application of formative evaluation during the mentoring of student intern nurses in an emergency department improved their professional theoretical performance, clinical practice skills, academic satisfaction, and self-directed learning abilities.

Development, implementation and formative evaluation of a motivational-volitional intervention to promote sustainable physical activity in breast cancer survivors

OBJECTIVE

The aim of the current project was the development, implementation and evaluation of the programme, Motivational-Volitional Intervention-Movement After Breast Cancer (Mo-Vo-BnB), an intervention for the sustainable promotion of physical activity of breast cancer survivors.

METHODS

In a multi-stage interdisciplinary development process, the pedagogical-didactic, psychological and physical evidence-based programme was developed and implemented for women after breast cancer who were approved for medical rehabilitation and were minimally, physically active (<60 min/week). Train-the-trainer seminars were carried out for the implementation. Four sessions were implemented in two German clinics. The training quality, didactic methods and accompanying material were evaluated 6 weeks and 12 months after implementation by patients, trainers and project members (n = 127 evaluations).

RESULTS

The standardised and published MoVo-BnB programme can provide practical and quality training. Content and methods can be implemented according to the manual. Training quality, didactic methods, and accompanying materials were evaluated positively.

CONCLUSION

The results suggest that MoVo-BnB is a useful standardised intervention for promoting the physical activity of breast cancer survivors. The demonstrated process is also suitable for other projects.

CLINICAL TRIAL REGISTRATION

German Clinical Trials Register (DRKS): DRKS00011122; Trial registration date: 2016 October 13.

Data Collection Framework for Context-Aware Virtual Reality Application Development in Unity: Case of Avatar Embodiment

Virtual Reality (VR) has been adopted as a leading technology for the metaverse, yet most previous VR systems provide one-size-fits-all experiences to users. Context-awareness in VR enables personalized experiences in the metaverse, such as improved embodiment and deeper integration of the real world and virtual worlds. Personalization requires context data from diverse sources. We proposed a reusable and extensible context data collection framework, ManySense VR, which unifies data collection from diverse sources for VR applications. ManySense VR was implemented in Unity based on extensible context data managers collecting data from data sources such as an eye tracker, electroencephalogram, pulse, respiration, galvanic skin response, facial tracker, and Open Weather Map. We used ManySense VR to build a context-aware embodiment VR scene where the user's avatar is synchronized with their bodily actions. The performance evaluation of ManySense VR showed good performance in processor usage, frame rate, and memory footprint. Additionally, we conducted a qualitative formative evaluation by interviewing five developers (two males and three females; mean age: 22) after they used and extended ManySense VR. The participants expressed advantages (e.g., ease-of-use, learnability, familiarity, quickness, and extensibility), disadvantages (e.g., inconvenient/error-prone data query method and lack of diversity in callback methods), future application ideas, and improvement suggestions that indicate potential and can guide future development. In conclusion, ManySense VR is an efficient tool for researchers and developers to easily integrate context data into their Unity-based VR applications for the metaverse.

A Ugandan Parenting Programme to Prevent Gender-Based Violence: Description and Formative Evaluation

Purpose: To develop a culturally-sensitive intervention for the early prevention of gender-based violence (GBV) in Uganda. Methods: Programme design followed the 6SQuID model of intervention development and multi-sectorial advice. A formative evaluation was conducted in two communities with six groups and 138 participants. Findings: Four familial predictors of GBV were identified as potentially malleable: poor parent-child attachment, harsh parenting, inequitable gendered socialization and parental conflict. A community-based parenting programme was developed to address them. Its programme theory incorporates Attachment Theory, the concept that positive behavioural control develops emotional control, and Social Learning Theory. Its rationale, structure and content are presented using the TIDieR checklist. A formative evaluation showed the programme to be widely acceptable, culturally appropriate, and perceived to be effective, but also identified challenges. Conclusion: The careful development of this community-based parenting programme shows promise for the early prevention of GBV.

A Reusable Multiplayer Game for Promoting Active School Transport: Development Study

Background

Most children and adolescents in Sweden do not meet the recommended daily physical activity levels of the World Health Organization. Active school transport (AST) and gamification are potential methods for increasing children’s daily physical activity. We previously developed a game named Tic-Tac-Training for promoting active transport at workplaces; however, the game has not been applied to AST.

Objective

The objectives of this study are to investigate how Tic-Tac-Training functions to promote AST among schoolchildren in northern Sweden, improve the game to be more suitable for schoolchildren, and construct a road map for future development based on children’s ideas.

Methods

First, we developed Tic-Tac-Training using the Scrum agile software development method. Second, we conducted a questionnaire-based formative evaluation of the game with schoolchildren (n=16; 9/16, 56% male; 6/16, 38% female; and 1/16, 6% other aged 11-12 years) in Luleå, Sweden. Third, we conducted focus group interviews with 33 children (13/33, 39% male and 20/33, 61% female aged 12-13 years) to gather ideas for gamifying AST. We mapped the interview results to the Octalysis gamification framework and established a road map for future development.

Results

The formative evaluation revealed several issues, including a lack of interesting game features, lack of support for continuous engagement, disliked competitive features, and lack of incentives for discourse and participation. New features such as rewards, collectibles, and levels were implemented based on the results. The focus group interviews revealed additional ideas for gamifying AST, such as using avatars, in-game currency and trading, and context-sensitive tasks.

Conclusions

The results have several potential impacts on how reusable, gamified AST interventions can be developed and what kind of gamification elements schoolchildren in northern Sweden wish to see. These results can interest game researchers and teachers who wish to apply gamification in school contexts. Finally, we aim to continue developing the game based on the road map.

Developing a Primary Care-Focused Intervention to Engage Patients With Osteoarthritis in Physical Activity: A Stakeholder Engagement Qualitative Study

Physical activity (PA) is important for managing osteoarthritis (OA), but many patients are inactive. Research is needed on strategies to leverage clinical encounters to engage patients in PA. Guided by the socioecological model of health behavior, this study aimed to engage stakeholders in the process of refining an Osteoarthritis Physical Activity Care Pathway (OA-PCP). Six focus groups and seven individual interviews were conducted with key stakeholders. Focus groups were specific to stakeholder roles and included patients with OA, support partners, and clinic personnel (n = 6 focus groups). Interview participants were local and national PA program representatives (n = 7 interviews). Data were analyzed by thematic analysis. Themes identified in the data included ways the OA-PCP can help patients with OA address challenges to PA engagement, strategies for connecting patients with PA resources, methods for implementing OA-PCP into clinical settings and potential use of PA trackers in the OA-PCP program. Stakeholders' comments were summarized into key recommendations for OA-PCP. Some recommendations reinforced and led to refinements in planned aspects of OA-PCP, including tailoring to individual patients, involvement of a support partner, and addressing pain with PA. Other recommendations resulted in larger changes for OA-PCP, including the addition of three email- or mail-based contacts and not requiring use of a PA tracker. The refined OA-PCP program is being evaluated in an exploratory trial, with the ultimate goal of establishing a PA program for OA that can be successfully implemented in clinical settings.

A starter's guide to learning and teaching how to coproduce healthcare services

BACKGROUND

There has been insufficient attention paid to the role of learning in co-production-both how service users and professional service providers learn to co-produce effectively and how the lessons of co-production are captured at a service level.

OBJECTIVE

We aimed to develop and test a curriculum to support healthcare professionals' interest in learning how to co-produce health and healthcare services with patients.

METHODS

We developed a co-production curriculum that was tested iteratively in multiple in-person and virtual teaching sessions and short courses. We conducted a formative evaluation of the co-production curriculum and teaching tools to tailor the curriculum.

RESULTS

Several theories underpin our approach to learning and teaching how to co-produce healthcare services. The co-production curriculum is grounded in systems theory and shares elements of educational theories, namely, the postmodern curriculum matrix, the actor network theory and situated learning in communities of practice. Learning participants valued the sense of community, the experiential learning environment, and the practical methods to support their exploration of co-production.

CONCLUSION

This paper summarizes the educational theories that underpin our efforts to develop and implement the curriculum, reports on a formative assessment conducted with learners, and makes recommendations for creating an environment for learning how health professionals can co-produce health and healthcare with patients.

Successes of and Lessons From the First Joint eHealth Program of the Dutch University Hospitals: Evaluation Study

BACKGROUND

A total of 8 Dutch university hospitals are at the forefront of contributing meaningfully to a future-proof health care system. To stimulate nationwide collaboration and knowledge-sharing on the topic of evidence-based eHealth, the Dutch university hospitals joined forces from 2016 to 2019 with the first Citrien Fund (CF) program eHealth; 29 eHealth projects with various subjects and themes were selected, supported, and evaluated. To determine the accomplishment of the 10 deliverables for the CF program eHealth and to contribute to the theory and practice of formative evaluation of eHealth in general, a comprehensive evaluation was deemed essential.

OBJECTIVE

The first aim of this study is to evaluate whether the 10 deliverables of the CF program eHealth were accomplished. The second aim is to evaluate the progress of the 29 eHealth projects to determine the barriers to and facilitators of the development of the CF program eHealth projects.

METHODS

To achieve the first aim of this study, an evaluation study was carried out using an adapted version of the Commonwealth Scientific and Industrial Research Organization framework. A mixed methods study, consisting of a 2-part questionnaire and semistructured interviews, was conducted to analyze the second aim of the study.

RESULTS

The 10 deliverables of the CF program eHealth were successfully achieved. The program yielded 22 tangible eHealth solutions, and significant knowledge on the development and use of eHealth solutions. We have learned that the patient is enthusiastic about accessing and downloading their own medical data but the physicians are more cautious. It was not always possible to implement the Dutch set of standards for interoperability, owing to a lack of information technology (IT) capacities. In addition, more attention needed to be paid to patients with low eHealth skills, and education in such cases is important. The eHealth projects' progress aspects such as planning, IT services, and legal played an important role in the success of the 29 projects. The in-depth interviews illustrated that a novel eHealth solution should fulfill a need, that partners already having the knowledge and means to accelerate development should be involved, that clear communication with IT developers and other stakeholders is crucial, and that having a dedicated project leader with sufficient time is of utmost importance for the success of a project.

CONCLUSIONS

The 8 Dutch university hospitals were able to collaborate successfully and stimulate through a bottom-up approach, nationwide eHealth development and knowledge-sharing. In total, 22 tangible eHealth solutions were developed, and significant eHealth knowledge about their development and use was shared. The eHealth projects' progress aspects such as planning, IT services, and legal played an important role in the successful progress of the projects and should therefore be closely monitored when developing novel eHealth solutions.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1016/j.ceh.2020.12.002.

Successes of and Lessons From the First Joint eHealth Program of the Dutch University Hospitals: Evaluation Study

BACKGROUND

A total of 8 Dutch university hospitals are at the forefront of contributing meaningfully to a future-proof health care system. To stimulate nationwide collaboration and knowledge-sharing on the topic of evidence-based eHealth, the Dutch university hospitals joined forces from 2016 to 2019 with the first Citrien Fund (CF) program eHealth; 29 eHealth projects with various subjects and themes were selected, supported, and evaluated. To determine the accomplishment of the 10 deliverables for the CF program eHealth and to contribute to the theory and practice of formative evaluation of eHealth in general, a comprehensive evaluation was deemed essential.

OBJECTIVE

The first aim of this study is to evaluate whether the 10 deliverables of the CF program eHealth were accomplished. The second aim is to evaluate the progress of the 29 eHealth projects to determine the barriers to and facilitators of the development of the CF program eHealth projects.

METHODS

To achieve the first aim of this study, an evaluation study was carried out using an adapted version of the Commonwealth Scientific and Industrial Research Organization framework. A mixed methods study, consisting of a 2-part questionnaire and semistructured interviews, was conducted to analyze the second aim of the study.

RESULTS

The 10 deliverables of the CF program eHealth were successfully achieved. The program yielded 22 tangible eHealth solutions, and significant knowledge on the development and use of eHealth solutions. We have learned that the patient is enthusiastic about accessing and downloading their own medical data but the physicians are more cautious. It was not always possible to implement the Dutch set of standards for interoperability, owing to a lack of information technology (IT) capacities. In addition, more attention needed to be paid to patients with low eHealth skills, and education in such cases is important. The eHealth projects' progress aspects such as planning, IT services, and legal played an important role in the success of the 29 projects. The in-depth interviews illustrated that a novel eHealth solution should fulfill a need, that partners already having the knowledge and means to accelerate development should be involved, that clear communication with IT developers and other stakeholders is crucial, and that having a dedicated project leader with sufficient time is of utmost importance for the success of a project.

CONCLUSIONS

The 8 Dutch university hospitals were able to collaborate successfully and stimulate through a bottom-up approach, nationwide eHealth development and knowledge-sharing. In total, 22 tangible eHealth solutions were developed, and significant eHealth knowledge about their development and use was shared. The eHealth projects' progress aspects such as planning, IT services, and legal played an important role in the successful progress of the projects and should therefore be closely monitored when developing novel eHealth solutions.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1016/j.ceh.2020.12.002.

Implementation of the JOIN for ME Program for Families from Low-Income Backgrounds: The Use of Theory-Driven Formative Evaluation: Rhode Island CORD 3.0

Advances have been made in the development of effective interventions to address pediatric obesity; however, research findings often do not translate into clinical practice and a limited number of programs have been designed toward wide-spread dissemination and implementation. The Rhode Island (RI)-Childhood Obesity Research Demonstration (CORD) 3.0 Project involves adapting and testing an evidence-based pediatric weight management intervention (PWMI), JOIN for ME, for wide-scale dissemination and implementation in communities with a high proportion of families from low-income backgrounds. In this article, we describe the robust developmental formative evaluation (FE) process employed by RI-CORD as a model for the use of FE to drive dissemination of evidence-based PWMIs. The current project was guided by the Consolidated Framework for Implementation Research and Proctor Implementation Outcomes. This article also showcases examples of how the use of key informant interviews from engaged stakeholders in the community during a developmental FE process can drive selection of implementation strategies. The use of FE, driven by evidence-based theory, can help provide a roadmap to successful implementation of a pediatric weight management program, such as JOIN for ME.

Resident Preparation for the American Board of Anesthesiology Objective Standardized Clinical Examination: A Comparison of Virtual Telesimulation With In-person Simulation

BACKGROUND

The Objective Structured Clinical Examination (OSCE) is part of the American Board of Anesthesiology (ABA) certification process. A simulated OSCE can aid examination preparation, but the COVID-19 pandemic prevented in-person simulation training. Therefore, we adapted our in-person simulated OSCE (SOSCE) as a Zoom-based telesimulation OSCE (ZOSCE), permitting examinees to participate remotely. Comparing this process with historical in-person SOSCE cohorts, we hypothesized that this telesimulation-based format would still be well received by the trainees as a substitute when it was not possible to provide in-person practice and formative assessment. Subsequently, the ABA proposed a virtual-format OSCE.

METHODS

We conducted our 7-station ZOSCE according to the ABA content outline for all graduating third-year clinical anesthesia residents (CA-3) in 2020. From a main meeting room, the facilitator paired each CA-3 with a faculty proctor, assigned them to their own breakout room for each station, and rotated standardized patients in. The faculty proctor observed the CA-3's performance in real time using an assessment tool with objectives graded on a 0-2 scale. At the conclusion of the ZOSCE, proctors reviewed the assessment tool with the CA-3 and provided personalized global feedback. Assessment tool scores were used to calculate performance data for the study group that were compared with a SOSCE historical cohort from 2017 and 2018. All parties completed a Likert-style evaluation specific to the ZOSCE.

RESULTS

A total of 22 CA-3 residents participated. Mean performance scores ranged from 82.2%-94.9% (minimum = 38%, maximum = 100%). Compared with the historical SOSCE cohort, ZOSCE scores for 5 of 7 stations were not different, but scores in communication with professionals (P = .007) and ultrasound (P < .001) stations were lower. Overall, CA-3 participants rated the learning experience positively and felt it was a reasonable substitution for in-person simulation, with responses similar to those of a historical in-person SOSCE cohort.

CONCLUSIONS

A telesimulation-based practice ZOSCE for formative examination preparation for the ABA OSCE resulted in similar institutional scoring for most stations compared with in-person SOSCE, but some stations may be better practiced in person or require modifications. The virtual format may permit flexible scheduling during nonclinical times or for learners in remote locations. These findings have implications for future formative exercises and the formal summative examination process.

Development of novel home-based tampon sampling for endometrial cancer: findings from community-based focus groups with Black and White women

Novel endometrial cancer (EC) early-detection approaches may reduce racial disparities in mortality. We conducted six community-based focus groups with White and Black women (N = 57 participants) in February-March 2020 to explore acceptability of a home-based tampon sampling approach for EC. Participants also completed a survey. Data were analyzed using qualitative content analysis. Awareness of EC and risk factors was low. Acceptability regarding home sampling was high, but participants expressed concerns about instruction complexity and potential risks. Black women reported lower comfort with tampons. Increasing EC awareness, self-efficacy, and familiarization with tampons would advance prospects for at-home sample collection for EC testing.

Implementation of a Pilot SMS-Based Crisis Support Service in Australia

Background: Australia's first short message service (SMS) crisis support service was launched by Lifeline Australia in July 2018. The pilot program was independently evaluated over a 240-day period. Aims: We aimed to examine the experiences of key staff employed in the Lifeline Text pilot and identify the skills and types of support required to deliver a high-quality SMS-based crisis support service. Method: In total, 22 interviews were conducted with 14 Lifeline Text crisis supporters and in-shift supervisors (supervisors) at two time points in September 2018 and March 2019. A modified framework approach was adopted to undertake qualitative data analyses. Results: Delivering crisis support via text was initially challenging as a result of the need to translate skills from telephone crisis support to the SMS platform. This was compounded by the high degree of suicidality of help-seekers and volatility in demand for the service. Limitations: The independent evaluators were not involved in the design of the pilot. Conclusion: Lifeline text is providing an important and necessary service, using a novel mode of delivery in Australia. Maintaining service quality at peak demand, with many distressed and suicidal help-seekers, requires specialized training, experience, and exceptional skills.

Prescription drug safe storage practices in Arizona tribal communities

This paper provides an overview of an opioid poisoning prevention pilot project conducted in several American Indian/Alaskan Native communities using an applied public health approach. The intent of the project was to identify a prescription medication safeguarding option for use in the home environment. The authors engaged the target population to obtain their buy-in to select an intervention that was acceptable and appropriate for their needs. Focus groups and key informant interviews conducted in several tribal communities resulted in the selection of a heavy-duty, lockable storage box as the intervention. Through community-based partnerships, 55 boxes were installed in participating households. Along with the box, participants also received education on safe medication storage and disposal. At baseline, only 1% of the participants reported storing their medication securely. During a 60-day follow-up visit, 95% of the observed boxes were being used to store medications. Also at baseline, 31% of the participants reported a history of lost or stolen medications. There were no reported lost or stolen medications during the 60-day project period among the participants. During the follow-up visits, project staff also found the boxes being used to store other items valuable to the participants. Reportedly, having their medication and other valuables secured in one location provided a heightened feeling of security. Since the completion of this pilot project, several organisations and entities have replicated it in their communities.

Effect of an active learning methodology combined with formative assessments on performance, test anxiety, and stress of university students

The aim of this study was to evaluate the effect of an active methodology combined with a lecture on undergraduate student learning and levels of stress and anxiety. The active learning methodology consisted of a lecture of 50-min duration, study at home with a textbook, an educational game activity, and three formative assessments on the topic of the cardiac cycle. In a following class, the students provided saliva samples to evaluate their levels of stress, received an anxiety test, and then undertook an exam to assess their understanding of the cardiac cycle. The traditional teaching methodology consisted of two lectures (∼2-h duration) on blood pressure control systems, delivered orally. In the third class, the students provided saliva samples, received an anxiety test, and then undertook an exam to assess their understanding of blood pressure control systems. The level of stress was assessed with the concentrations of the stress biomarkers cortisol and alpha-amylase in saliva. Anxiety was assessed with the State-Trait Anxiety Inventory (STAI) questionnaire. The students achieved significantly higher average scores in exams when the active learning strategy was applied compared with the use of traditional theoretical classes. The active methodology resulted in significantly lower levels of stress and anxiety, as well as improved student performance, compared with the use of traditional lectures.

Formative Evaluation of Open Goals: A UK Community-Based Multi-Sport Family Programme

Community parks provide opportunities for physical activity (PA) and facilitate social interactions. This formative evaluation assesses the implementation of ‘Open Goals’ (OG), a novel multi-sport programme aiming to increase family PA and community cohesion, delivered weekly by Liverpool Football Club’s charitable foundation to local parks in Liverpool, North West England. Three Open Goals parks were chosen for the evaluation settings. Formative evaluation measures included: System for Observing Play and Recreation in Communities (SOPARC) observations (n = 10), direct session observations (n = 8), semi-structured interviews with Open Goals coaching staff (n = 3), and informal feedback from families (n = 5) about their experiences of Open Goals. Descriptive statistics and thematic analysis were applied to quantitative and qualitative data, respectively. Within the three evaluation parks, Open Goals reached 107 participants from May–July 2019, through 423 session attendances. Fidelity of the programme was high (M = 69% of session content delivered as intended). Overall park use when OG was offered compared to when it was not offered was not statistically significant (p = 0.051), however, target area use was significantly increased (p = 0.001). Overall physical activity levels in parks were significantly (p = 0.002) higher when Open Goals was being offered, compared to when it was not. Coaches reported that engagement in OG positively affected family co-participation and children’s behavioural development. Contextual issues included environmental and social barriers to programme engagement, including the co-participation element of the programme and criticism of the marketing of OG. It is evident that community-based multi-sport PA programmes endorsed by professional football clubs are well positioned to connect with local communities in deprived areas and to encourage PA and community engagement. This study suggests that such programmes may have the ability to improve park usage in specific areas, along with improving physical activity levels among families, although further research is required. Effective marketing strategies are needed for promotional purposes. Upskilling of coaches in the encouragement of family co-participation may support regular family engagement in PA in local parks.

Application of formative evaluation and teaching feedback in PBL teaching of Medical Genetics

Medical Genetics is an important research content of basic medicine and closely related to clinical medicine. Medical Genetics can not only lay a solid theoretical foundation for understanding medical problems for medical students, but also provide them an indispensable technical means for clinical medical practice. In order to improve students' understanding and mastery of the core content of Medical Genetics and cultivate senior medical talents with innovative spirit and independent thinking abilities, the PBL (problem-based learning) teaching method was introduced in the teaching of medical genetics for students enrolled in the eight-year medical education program. By integrating formative evaluation and teaching feedback mechanism into PBL teaching, we have achieved good teaching effects. In this paper, we will discuss the importance and necessity of formative evaluation and teaching feedback in PBL teaching, introduce the PBL teaching plan, teaching process and teaching effect of Medical Genetics in our school, share our thoughts on PBL teaching mode, and provide new ideas for the teaching reform of Medical Genetics.

An Evaluation of Parents' Experiences of Patient Engagement in Research to Develop a Digital Knowledge Translation Tool: Protocol for a Multi-Method Study

Background

The last decade has seen increasing calls for patient and public involvement in health-related research due to an ideological shift toward more equitable methods of knowledge development and an effort to increase the usability and relevance of knowledge by improving outcomes in clinical practice. Patient engagement includes simply informing patients to offering complete decision-making autonomy to individuals, groups, communities, caregivers, friends, and families who have personal experience and knowledge of a health issue. Despite the use of patient engagement methods in research, evaluation has lagged, resulting in a knowledge gap that makes it difficult to foster capacity and sustainability for patients and researchers alike since little is known about how effective patient collaborations in research are built, maintained, or improved. This study centers on pediatric functional constipation, a common condition that affects children and families. Since parents play a pivotal role in treatment, they are an optimal group to engage in improving the resources and support available to them.

Objective

This study aims to use patient-engagement methods to establish a research collaboration with parents to cocreate a digital knowledge translation tool for parents caring for a child with functional constipation and formally evaluate the patient engagement processes within this project to build the science of patient engagement in research.

Methods

Members of the parent collaborator group will be recruited from previous participants who expressed interest in the development of a digital knowledge translation tool. The group will collaborate with the research team to create a tool to address patients’ support and information needs when caring for a child with functional constipation. The parent collaborator group will then be evaluated in a multimethod study design. Data will be digitally and anonymously collected from all members of the parent collaborator group, using the validated Public and Patient Engagement Evaluation Tool (PPEET) patient questionnaire. Descriptive statistics will be used to report group characteristics and question responses. Qualitative analysis will be used to understand open-ended question responses. Specifically, directed content analysis will be used to assess themes of the Patient Engagement in Research (PEIR) Framework with a combination of deductive and inductive analyses. Findings will be integrated into the discussion if there are sufficient commonalities and inter-relationships. The final manuscript will include reporting of each element as described by the Good Reporting of a Mixed Methods Study criteria.

Results

Recruitment is planned for June 2020. Data collection for the evaluation of patient engagement processes will occur upon completion of the digital knowledge translation tool. The results of this study are expected to be published by the end of 2020.

Conclusions

This study will provide valuable information about parents’ experiences participating in child-health research and is a fundamental step in building the science of patient engagement in research.

International Registered Report Identifier (IRRID)

PRR1-10.2196/19108

Evaluation of USVreact: A Staff Training Program to Prevent Sexual Violence at Universities

Interventions addressing the endemic of sexual violence at European universities are scarce, particularly those that take a bystander focus to sexual violence prevention and involve university staff. Evidence-based data on their effectiveness are also lacking. This article reports the description of a pilot evaluation study of the USVreact Italian training program addressed to university staff for counteracting sexual violence. We assessed initial (T1) representations of gender-based violence, rape myth acceptance, and attitudes to bystander intervention (172 participants), and evaluated the effectiveness of the course by comparing, via paired-sample t tests, the responses before and after (T2) training (66 participants). Comparison between pre- (T1) and posttraining (T2) responses indicated that the participants' ability to recognize subtle forms of violence and reduce rape myth acceptance was increased after training. Relatively few training programs based on the bystander approach to prevent gender-based violence at university have been performed to date in Europe and data on their effectiveness are scarce. Several limitations notwithstanding, the present pilot evaluation study provides suggestions for a more systematic evaluation of training interventions that address cultural legitimation of gender-based violence and that sustain bystander interventions in sexual assault prevention.

Best Practices in Curricula Revisions: Using the Evidence-Based Eating Smart • Being Active as an Exemplar

This article describes the processes employed to revise the widely used curriculum, Eating Smart • Being Active. Because of its popularity among nutrition education programs serving the low-income population, the curriculum developers felt it was important to share the revision process after the release of the 2015-2020 Dietary Guidelines for Americans. Extensive feedback during formative evaluation, updated content from the Dietary Guidelines for Americans, and a modern look resulted in a fully revised curriculum released in 2017. Program leaders should have confidence that their educators will be able to implement this evidence-based curriculum with fidelity. An outcome evaluation of the revised Eating Smart • Being Active curriculum is recommended.

Formative Research and Cultural Tailoring of a Substance Abuse Prevention Program for American Indian Youth: Findings From the Intertribal Talking Circle Intervention

Background. Substance use among American Indians (AIs) is a critical health issue and accounts for many health problems such as chronic liver disease, cirrhosis, behavioral health conditions, homicide, suicide, and motor vehicle accidents. In 2013, the highest rates of substance use and dependence were seen among AIs when compared to all other population groups, although these rates vary across different tribes. Among AI adolescents, high rates of substance use have been associated with environmental and historical factors, including poverty, historical trauma, bicultural stress, and changing tribal/familial roles. Our project, the Intertribal Talking Circle intervention, involved adapting, tailoring, implementing, and evaluating an existing intervention for AI youth of three tribal communities in the United States. Formative Results. Community partnership committees (CPCs) identified alcohol, marijuana, and prescription medications as high priority substances. CPC concerns focused on the increasing substance use in their communities and the corresponding negative impacts on families, stating a lack of coping skills, positive role models, and hope for the future as concerns for youth. Cultural Tailoring Process Results. Each site formed a CPC that culturally tailored the intervention for their tribal community. This included translating Keetoowah-Cherokee language, cultural practices, and symbolism into the local tribal customs for relevance. The CPCs were essential for incorporating local context and perceived concerns around AI adolescent substance use. These results may be helpful to other tribal communities developing/implementing substance use prevention interventions for AI youth. It is critical that Indigenous cultures and local context be factored into such programs.

Mobile App to Help People With Chronic Illness Reflect on Their Strengths: Formative Evaluation and Usability Testing

BACKGROUND

Supporting patient engagement and empowerment is increasingly seen as essential in providing person-centered health care to people with chronic illness. Mobile apps helping patients reflect on their concerns as preparation for consultations with their health care providers can have beneficial effects on the consultation quality. However, apps focusing on empowerment and personal strengths are still scarce.

OBJECTIVE

This study aimed to (1) develop a mobile app to support patients with rheumatic diseases in reflecting on their strengths in preparation for consultations with health care providers and (2) explore patients' perceived usability of the app in a nonclinical test setting.

METHODS

A prototype app was developed based on input from patients and health care providers, as reported in previous studies. The app was designed for use in self-management support settings aiming to promote awareness of strengths and to focus attention on strengths in the patient-health care provider dialogue. The features included in the prototype were as follows: (1) introduction to the topic of strengths, (2) list of examples of strengths to promote reflection and registration of own strengths, (3) summary of registered strengths, (4) value-based goal setting, (5) linking of strengths to goals, (6) summary of all registrations, and (7) options to share summary digitally or as a print version. In this study, the app was refined through a formative evaluation with patients and health care providers recruited from a specialized rheumatology hospital unit. Patients' perceptions of the app's usability were explored in a test setting with self-report measurements and semistructured interviews. The interviews were audiotaped, transcribed, and analyzed with directed content analysis. Data from questionnaires were analyzed with descriptive statistics.

RESULTS

Developmental and formative evaluation included 18 patients and 7 health care providers. The evaluation resulted in minor adjustments to the prototype but no major changes in features. The usability testing included 12 patients. All participants found the usability acceptable; the median score on the System Usability Scale was 86.3 (range 70-100). All reported that it was meaningful and relevant to use the app. Out of 12 participants, 9 (75%) reported becoming more aware of their own strengths by using the app; 1 (8%) disagreed and 2 (17%) provided a neutral response. The results on the goal-related feature were mixed, with half of the patients finding it useful to link strengths to concrete goals. A statistically significant positive change from pre- to postintervention was identified on measures of self-efficacy and negative emotions.

CONCLUSIONS

In this formative evaluation of a mobile app to promote patients' reflections on their strengths, patients perceived the app as meaningful and supporting awareness. The results suggest the usefulness of building in functionality to support use of strengths and goal attainment. Further studies on efficacy and usability in a clinical setting, including health care providers, are needed.

Collaboration and Context in the Design of Community-Engaged Research Training

Collaboration between academic researchers and community members, clinicians, and organizations is valued at all levels of the program development process in community-engaged health research (CEnR). This descriptive study examined a convenience sample of 30 projects addressing training in CEnR methods and strategies within the Clinical and Translational Science Awards (CTSA) consortium. Projects were selected from among posters presented at an annual community engagement conference over a 3-year period. Study goals were to learn more about how community participation in the design process affected selection of training topics, how distinct community settings influenced the selection of training formats, and the role of evaluation in preparing training participants to pursue future health research programming. Results indicated (1) a modest increase in training topics that reflected community health priorities as a result of community (as well as academic) participation at the program design stage, (2) a wide range of community-based settings for CEnR training programs, and (3) the majority of respondents conducted evaluations, which led in turn to revisions in the curricula for future training sessions. Practice and research implications are that the collaboration displayed by academic community teams around CEnR training should be traced to see if this participatory practice transfers to the design of health promotion programs. Second, collaborative training design tenets, community formats and settings, and evaluation strategies should be disseminated throughout the CTSA network and beyond. Third, common evaluative metrics and indicators of success for CEnR training programs should be identified across CTSA institutions.

Thrive: A Novel Health Education Mobile Application for Mothers Who Have Experienced Intimate Partner Violence

Intimate partner violence (IPV) has well-documented adverse impact on survivors and their children. In this article, we describe the development and formative evaluation of a trauma-informed, user-friendly Smartphone-based mobile application (app) to address the unmet health needs and improve the well-being of mothers who have experienced IPV. A multidisciplinary team of IPV experts developed the app (called Thrive) in partnership with software developers. Thrive includes three sections: Myself (maternal self-care, stress coping skills), My Child (stress signs in children, talking to children about IPV, mother-child dyadic communication), and My Life (hospital- and community-based resources). Sixteen providers (social workers, IPV advocates, and health care providers) and eight IPV survivors provided feedback about Thrive via structured interviews. Participants found Thrive to be user-friendly, informative, trauma-informed, and a potential alternative to handouts. Participants had several recommendations including allowing users to add their own content and providing social support mechanisms. Initial feedback sessions have demonstrated preliminary acceptability of one of the first health education apps for mothers who have experienced IPV. Next steps include revising Thrive based on user feedback, testing Thrive via a longitudinal outcome evaluation, and working with hospital and community-based partners to disseminate Thrive to IPV survivors around the country.

Transformation of a face-to-face workshop into a Massive Open Online Course (MOOC): A design and development case

The purpose of this design and development case is to share our experiences in the transformation of a face-to-face workshop into a Massive Open Online Course (MOOC) for a prominent MOOC platform. The goal of the workshop and MOOC is to teach learners how to conduct appropriate power and sample size analysis for multilevel and longitudinal studies in social and behavioral health research. Learners include people from across the biomedical research spectrum, from students to full professors. We first describe the design and development frameworks and processes used to create the three-day, face-to-face workshop. Then, we detail the design and development approach to transform this face-to-face workshop into a MOOC. At a macro-design level, we employed backward design (Wiggins & McTighe, 1998) as an instructional design framework. At a micro-design level, we used a combination of the first principles of instruction, the cognitive theory of multimedia learning, the nine events of instruction, and design recommendations for MOOCs found in the literature. We report the results of a formative evaluation of the MOOC. Finally, we provide closing remarks, lessons learned, and the next steps for the instructional program.

Formative Evaluation to Build an Online Parenting Skills and Youth Drug Prevention Program: Mixed Methods Study

Background

Family-based drug prevention programs that use group-based formats with trained facilitators, such as the Strengthening Families Program (SFP), are effective in preventing underage drinking and youth drug use. However, these programs are resource-intensive and have high costs and logistical demands. Tailoring them for Web-based delivery is more cost-effective and makes it easier to scale these programs for widespread dissemination. This requires the active involvement of all key stakeholders to determine content and delivery format.

Objective

The aim was to obtain consumer, agency stakeholder, and expert input into the design of a Web-based parenting skills training and youth drug prevention program.

Methods

We conducted 10 focus groups with 85 adults (range 4-10, average 8 per group), 20 stakeholder interviews with family services agency staff, and discussed critical design considerations with 10 prevention scientists and e-learning experts to determine the optimal program content and technology features for SFP Online. Focus group participants also answered survey questions on perceived barriers to use, desired navigational features, preferred course format, desired content, preferred reward structures, course length, interactive components, computer efficacy, and technology use. Descriptive statistics were used to examine consumer characteristics; linear regression was used to examine relations between SFP exposure and four continuous outcome measures, including desired program content, interactive technology, and concerns that may inhibit future use of SFP Online. Logistic regression was used as a binary measure of whether consumers desired fun games in the SFP Online program.

Results

Three broad thematic categories emerged from the qualitative interviews enumerating the importance of (1) lesson content, (2) logistics for program delivery, and (3) multimedia interactivity. Among the many significant relations, parents who viewed more SFP lessons reported more reasons to use an online program (beta=1.48, P=.03) and also wanted more interactivity (6 lessons: beta=3.72, P=.01; >6 lessons: beta=2.39, P=.01), parents with less interest in a mixed delivery format (class and online) reported fewer reasons to use the online program (beta=−3.93, P=.01), comfort using computers was negatively associated with concerns about the program (beta=−1.83, P=.01), having mobile phone access was related to fewer concerns about online programs (beta=−1.63, P=.02), willingness to view an online program using a mobile phone was positively associated with wanting more online components (beta=1.95, P=.02), and parents who wanted fun games wanted more interactivity (beta=2.28, P=.01).

Conclusions

Formative evaluation based on user-centered approaches can provide rich information that fuels development of an online program. The user-centered strategies in this study lay the foundation for improving SFP Online and provide a means to accommodate user interests and ensure the product serves as an effective prevention tool that is attractive to consumers, engaging, and can overcome some of the barriers to recruitment and retention that have previously affected program outcomes in family-based prevention.

Designing an ethnographic interview for evaluation of micronutrient powder trial: Challenges and opportunities for implementation science

The evidence base for micronutrient powder (MNP) interventions predominantly consists of quantitative studies focused on measuring coverage, utilization, and/or biological outcomes. We need other types of studies to broaden the scope of our knowledge about determinants of MNP programme effectiveness. Addressing this knowledge gap, this paper focuses on the process of designing an ethnographic research protocol to obtain caregivers' perspectives on the factors that influenced their use of intervention delivery services and their adherence to MNP recommendations. The research was undertaken within the context of formative evaluations conducted in Mozambique and Ethiopia. Ethnography provides a means for acquiring and interpreting this knowledge and is an approach particularly well suited for formative evaluation to understand the response of a population to new interventions and programme delivery processes. We describe decisions made and challenges encountered in developing the protocol, and their implications for advancing methodology in implementation research science. In addition to a core team of three investigators, we added an “advisory group” of 10 experts to advise us as we developed the protocol. The advisory group reviewed multiple drafts of the interview protocol and participated in mock interviews. In the protocol development process, we faced the issues and made decisions about concerned gaps in content, cultural adaptations and comprehension, and interview guide structure and format. Differences between the core team and the advisory group in methodological approaches to the structure and content of questions call attention to the importance of establishing greater communication among implementation scientists working in nutrition interventions.

Development of a Simulated Objective Structured Clinical Exam for the APPLIED Certification Exam in Anesthesiology: A Two-Year Experience Informed by Feedback from Exam Candidates

BACKGROUND

Program directors of anesthesiology residencies agree that it is the program's responsibility to prepare residents for primary American Board of Anesthesiology (ABA) certification, although few report an Objective Standardized Clinical Exam (OSCE) program reflective of the new ABA examination. We created an authentic simulated OSCE (SOSCE) using existing resources to prepare third-year clinical anesthesia residents for the ABA APPLIED exam before graduation and identify knowledge gaps relevant to the OSCE.

METHODS

Junior anesthesiology residents and medical students acted as standardized patients for the 7 SOSCE stations. Third-year clinical anesthesia residents were evaluated on performance by faculty educators during the SOSCE and completed surveys regarding their experience. Follow-up surveys were distributed to participants after they completed the ABA APPLIED Exam.

RESULTS

Mean scores ranged from 82.6% correct (echocardiogram) to 97.2% correct (ultrasonography). Knowledge gaps were present in competencies explicitly stated as objectives by the ABA. Echocardiography scores improved from 76.1% in the first year to 90.0% in the second year (P = .009). Participants found the SOSCE to be valuable in preparing for the OSCE and the standardized patients' performance to be convincing. Participants felt better prepared for the ABA exam and thought that the SOSCE was authentic in content and process.

CONCLUSIONS

An SOSCE program can be developed with preexisting resources. This program was highly rated as useful and informative, an accurate reflection of the ABA OSCE, and helpful in preparation for the examination. Development of a SOSCE program is feasible, sustainable, and valuable.

Peer assessment platform of clinical skills in undergraduate medical education

Objective

The study objective was to enhance clinical skills among undergraduate students majoring in clinical medicine in performing physical examination by establishing a novel platform for peer assessment of clinical skills.

Methods

A total 126 Year 2012 students majoring in medicine and receiving traditional training were assigned to the control group, and 126 Year 2013 students receiving instruction via the peer assessment platform of clinical skills were allocated to the study group. Scores of the physical examination, paper exam, and peer assessment were compared using a t-test, and we performed linear correlation analysis of the data.

Results

Scores of the physical examination and peer assessment among Year 2013 students (the study group) were significantly higher than those in the control group. Paper exam scores in the study group were also significantly higher than those in the controls. The three assessment scores did not differ significantly according to sex.

Conclusions

The peer assessment platform can not only improve medical students’ skills and capabilities in physical examination, it can also enhance their theoretical knowledge of basic clinical principles. We determined that sex was not related to the assessment scores obtained by medical students.

Formative Evaluation of the Central Coast Integrated Care Program (CCICP), NSW Australia

Introduction:

Integrated care has been posited as an important strategy for overcoming service fragmentation problems and achieving the Quadruple Aim of health care. This paper describes the Central Coast Integrative Care Program (CCICP) a complex, multi-component intervention addressing 3 target populations and more than 40 sub-projects of different scale, priority and maturity. Details are provided of the implementation including activities undertaken for each target population, in the context of the Central Coast Local Health District (CCLHD) strategies and priorities. Key lessons are drawn from the formative evaluation.

Methods:

A mixed methods approach to the formative evaluation was taken. Key stakeholders, professional staff with an in-depth knowledge of the program, were invited to complete surveys (n = 27) and semi-structured interviews (n = 23). The evaluation employed co-design principles with dialogue between CCICP partners and researchers throughout the process and sought to achieve a shared understanding of the dynamic context of the program, and the barriers and enablers for the various interventions.

Key lessons and conclusion:

Seven interdependent key lessons have been identified. These distil down to the setting of clear objectives aligning with all the goals of partners, developing strong relationships, leadership at multiple levels and communication and the building of a common language.

Integrating Worksite Smoking Cessation Services Into the Construction Sector: Opportunities and Challenges

Background. Smoking prevalence among Hispanic/Latino construction workers in the United States is very high (31%). Aims. To investigate tobacco use profiles in these minority workers and explore their management's views about implementing sustainable worksite smoking cessation services. Methods. Analysis of baseline data from a smoking cessation trial among Hispanic/Latino construction workers (n = 134; adult men ≥18 years), and semistructured, 45-minute interviews with 24 key personnel at six construction companies in south Florida were conducted. Interviews were recorded, transcribed, and analyzed thematically. Results. Overall, 43.3% of workers were Cuban, and 81.3% had low acculturation level. Nicotine dependence levels were "high" in 61.8% of workers. Half of the workers had a successful quit attempt but only 9.9% received advice from a physician to quit smoking, 16.7% used medication to quit, and 79.2% did not receive assistance. Participants in the interviews stated that nothing was provided to help smokers quit smoking and considered distributing self-help materials with free medications as the most appropriate service. Challenges to integrating the service were time restriction and cost. Recommendations for implementing the service were local/state government mandate. Discussion. Tailoring tobacco treatment to Hispanic/Latino construction workers' job circumstances and culture is essential to support their cessation efforts. Integrating worksite tobacco treatment services into other available health promotion programs (e.g., safety) and enforcing smoke-free legislation in the construction sector can facilitate its adoption. Conclusion. Involving key stakeholders and mandating the service by the State and local government are necessary to integrate sustainable worksite smoking cessation services in the construction sector.

Communicating the environmental health risk assessment process: formative evaluation and increasing comprehension through visual design

The environmental health risk assessment process informs clean-up activities at hazardous waste sites. Ensuring this process is accessible and transparent to communities is crucial for environmental health literacy initiatives. The goals of this project were to develop plain language and effective visuals that can be used when communicating the risk assessment process and methods used to predict excess cancer risk(s) due to environmental exposures. In this study, a community factsheet entitled, "Understanding Environmental Health Risk Assessment" was developed and a participatory design and formative evaluation approach was implemented with a set of representative users (n = 11). Community members living in the vicinity of two Arizona hazardous waste sites as well as three public health professionals/researchers were asked to evaluate the functionality and accessibility of the factsheet, particularly the graphics and whether the text was written in plain language. Participant responses revealed the following major findings: 1) form follows function, 2) graphic elements should outweigh text, 3) line of sight and layout is critical to information accessibility, 4) color coding dramatically aids the reader, 5) content should be strategically grouped, 6) concepts per figure should be minimized to ensure comprehension, 7) interactive content is preferred over static content, and 8) communication efforts need to interweave new information with the targeted audience's past and current environmental health understandings to aid in their ability to retain new concepts. Based on participant feedback, new and improved layout decisions, infographics and accompanying text were designed and prepared. This research demonstrates the need and importance of participatory design, information design prototyping, formative evaluation, and a cultural model of risk communication.

Changing healthcare provider and parent behaviors in the pediatric post-anesthesia-care-unit to reduce child pain: Nurse and parent training in postoperative stress

BACKGROUND

Children who undergo surgery experience significant pain in the post anesthesia care unit. Nurse and parent behaviors in the post anesthesia care unit directly impact child postoperative pain. Therefore, we have developed and evaluated (Phase 1) and then tested (Phase 2) the feasibility of a new intervention (Nurse and Parent Training in Postoperative Stress) to alter parent and nurse behaviors in a way consistent with reducing child postoperative pain.

METHODS

In Phase 1, a multidisciplinary team of experts (physicians, nurses, and psychologists) developed an empirically-based intervention which was then evaluated by experienced nurses (N = 8) and parents (N = 9) during focus groups. After revising the intervention based on focus group feedback, it was tested in Phase 2 using a pre-post study design. Nurses (N = 23) who worked in the recovery room were recruited to be part of both pre- and post-intervention data collection periods. Parents were recruited to be part of either the pre- (N = 52) or post-intervention (N = 60) data collection periods. Nurses and parent-child dyads were recorded in the post anesthesia care unit and videos were coded for the desired (ie, behaviors that may decrease child pain) and non-desired (ie, behaviors that may increase child pain) behaviors. Pain data was collected from the children's medical records to assess pain after surgery. The intervention was given to the nurses and parents in the post-intervention data collection period.

RESULTS

Nurses significantly increased their rate of desired behaviors by 231% (P = 0.001; Somer's D = 1) and significantly decreased their rate of non-desired behaviors by 62% (P = 0.004, Somer's D = -0.88, 95% CI [-1.74, -0.03]). Parents significantly increased their rate of desired behaviors by 124% (P = 0.033). Moreover, the intervention significantly decreased child pain in the post anesthesia care unit (b = -2.19, SE = 0.63, z = -3.46, P = 0.001, 95%CI [-3.43, -0.95]).

CONCLUSION

The intervention was effective in changing nurse and parent behaviors as well as child pain after surgery.

Using the Project INTEGRATE Framework in Practice in Central Coast, Australia

Introduction:

Integrated care implies sustained change in complex systems and progress is not always linear or easy to assess. The Central Coast integrated Care Program (CCICP) was planned as a ten-year place-based system change. This paper reports the first formative evaluation to provide a detailed description of the implementation of the CCICP, after two years of activity, and the current progress towards integrated care.

Theory and Methods:

Progress towards integrated care achieved by the CCICP was evaluated using the Project INTEGRATE Framework data in a mixed methods approach included semi-structured interviews (n = 23) and Project INTEGRATE Framework based surveys (n = 27). All data collected involved key stakeholders, with close involvement in the program, self-reporting.

Results:

Progress has been mixed. Gains had most clearly been made in the areas of clinical and professional integration; specifically, relationship building and improved collaboration and cooperation between service providers. The areas of systemic and functional integration were least improved with funding uncertainty being an ongoing significant problem. The evaluation also showed that the Project INTEGRATE framework provided a consistent language for CCICP partners and for evaluators and consistent indicators of progress. The framework also helped to identify key facilitators and barriers.

Discussion and Conclusion:

The findings highlight the willingness and commitment of key staff but also the importance of leadership, good communication, relationship building, and cultural transformation.

The Effects of Feedback Fatigue and Sex Disparities in Medical Student Feedback Assessed Using a Minute Feedback System

OBJECTIVE

Feedback is critical to the development of medical students. To enhance feedback, we created a web application, the Minute Feedback System (MFS). This app allows students to request precise, timely, written feedback from residents and staff without the burden of vague, end-of-rotation surveys. In this study, we investigate variations in response rates and feedback fatigue based on sex and rank (resident/fellow vs. faculty).

DESIGN

Data were collected from May 2015-October 2016. The MFS stores student requests for feedback along with faculty responses allowing for analysis of feedback response rate as well as sex and rank identification. Variation in response rate was analyzed using Chi-square and log-rank testing. Feedback fatigue was assessed using Cox regression modeling.

SETTING

University Affiliated, Tertiary Care Center.

PARTICIPANTS

Medical Students, Residents and Faculty.

RESULTS

About 98.6% of students (138 women, 140 men) used the MFS on their surgery clerkship. They requested feedback from 159 trainees (residents or fellows) and 114 surgical faculty. Feedback was requested more from faculty (26.3 requests per individual) compared to trainees (16.4 requests per individual). The overall evaluator response rate was 60%. Male students were 13% less likely to receive feedback than female students. There was a higher prevalence of feedback fatigue among female faculty (11% less likely to respond) and residents (23% less likely to respond). Regression analysis showed that the overall hazard of nonresponse over time was 1.05, indicative of overall feedback fatigue among all respondents.

CONCLUSIONS

The MFS is a novel tool for feedback used by nearly all M3 students during their surgery clerkship at our institution. Evaluation of response rates demonstrated feedback fatigue, especially among women faculty and residents. Feedback fatigue was more likely to affect male students, although the reason for this is unclear. Further analysis is necessary to understand this sex-associated response disparity and its effect on student feedback in the learning environment.

Identifying Implementation Science Characteristics for a Prenatal Oral Health eHealth Application

Background. Oral health is a significant public health issue; yet barriers to implementing the prenatal oral health guidelines into practice remain. This formative research aimed to identify key implementation science characteristics to inform the development of an eHealth application (app) to assist providers in implementing the prenatal oral health guidelines during prenatal visits. Method. Guided by the Consolidated Framework for Implementation Research, the clinic's infrastructure, workflow, and contextual factors were assessed via clinic observation, technology assessment, prenatal provider interviews (n = 4), clinic staff interviews (n = 8), and two focus groups with oral health providers (n = 16). Results. System-level factors influencing future implementation were identified regarding structural characteristic, networks/communication, culture, external policy/incentives, relative advantage, complexity, design quality/packaging, knowledge/beliefs, and personal attributes. Discussion. Findings provided vital information and will directly inform the design and implementation of an eHealth app that aims to facilitate the translation of the interprofessional prenatal oral health guidelines into clinical prenatal oral health practices.

The Development of Complex Digital Health Solutions: Formative Evaluation Combining Different Methodologies

Background

The development of digital health solutions for current health care settings requires an understanding of the complexities of the health care system, organizational setting, and stakeholder groups and of the underlying interplay between stakeholders and the technology. The digital health solution was founded on the basis of an information and communication technology platform and point-of-care devices enabling home-based monitoring of disease progression and treatment outcome for patients with rheumatoid arthritis (RA).

Objective

The aim of this paper is to describe and discuss the applicability of an iterative evaluation process in guiding the development of a digital health solution as a technical and organizational entity in three different health care systems.

Methods

The formative evaluation comprised the methodologies of contextual understanding, participatory design, and feasibility studies and included patients, healthcare professionals, and hardware and software developers. In total, the evaluation involved 45 patients and 25 health care professionals at 3 clinical sites in Europe.

Results

The formative evaluation served as ongoing and relevant input to the development process of the digital health solution. Through initial field studies key stakeholder groups were identified and knowledge obtained about the different health care systems, the professional competencies involved in routine RA treatment, the clinics’ working procedures, and the use of communication technologies. A theory-based stakeholder evaluation achieved a multifaceted picture of the ideas and assumptions held by stakeholder groups at the three clinical sites, which also represented the diversity of three different language zones and cultures. Experiences and suggestions from the patients and health care professionals were sought through participatory design processes and real-life testing and actively used for adjusting the visual, conceptual, and practical design of the solution. The learnings captured through these activities aided in forming the solution and in developing a common understanding of the overall vision and aim of this solution. During this process, the 3 participating sites learned from each other’s feed-back with the ensuing multicultural inspiration. Moreover, these efforts also enabled the consortium to identify a ‘tipping point’ during a pilot study, revealing serious challenges and a need for further development of the solution. We achieved valuable learning during the evaluation activities, and the remaining challenges have been clarified more extensively than a single-site development would have discovered. The further obstacles have been defined as has the need to resolve these before designing and conducting a real-life clinical test to assess the outcome from a digital health solution for RA treatment.

Conclusions

A formative evaluation process with ongoing involvement of stakeholder groups from 3 different cultures and countries have helped to inform and influence the development of a novel digital health solution, and provided constructive input and feedback enabling the consortium to control the development process.

Using Patients and Their Caregivers Feedback to Develop ENABLE CHF-PC: An Early Palliative Care Intervention for Advanced Heart Failure

OBJECTIVE:

Models of early, community-based palliative care for individuals with New York Heart Association (NYHA) class III/IV heart failure and their families are lacking. We used the Medical Research Council process of developing complex interventions to conduct a formative evaluation study to translate an early palliative care intervention from cancer to heart failure.

METHOD:

One component of the parent formative evaluation pilot study was qualitative satisfaction interviews with 8 patient-caregiver dyad participants who completed Educate, Nurture, Advise, Before Life Ends Comprehensive Heartcare For Patient and Caregivers (ENABLE CHF-PC) intervention. The ENABLE CHF-PC consists of an in-person palliative care assessment, weekly telehealth coaching sessions, and monthly follow-up. Subsequent to completing the coaching sessions, patient and caregiver participants were interviewed to elicit their experiences with ENABLE CHF-PC. Digitally recorded interviews were transcribed and analyzed using a thematic approach.

RESULTS:

Patients (n = 8) mean age was 67.3, 62.5% were female, 75% were married/living with a partner; caregivers (n = 8) mean age was 56.8, and 87.5% were female. Four themes related to experiences with ENABLE CHF-PC included "allowed me to vent," "gained perspective," "helped me plan," and "gained illness management and decision-making skills." Recommendations for intervention modification included (1) start program at diagnosis, (2) maintain phone-based approach, and (3) expand topics and modify format.

CONCLUSION:

Patients and caregivers unanimously found the intervention to be helpful and acceptable. After incorporating modifications, ENABLE CHF-PC is currently undergoing efficacy testing in a large randomized controlled trial.

Development and validation of the Social Determinants of Health Questionnaire and implications for "Promoting Food Security and Healthy Lifestyles" in a complex urban food ecosystem

In this case study, a formative evaluation was conducted for "Promoting Food Security and Healthy Lifestyles" pilot intervention at a Community-Based Organization in a marginalized neighborhood in Bedford-Stuyvesant in New York City. Utilizing a rigorous, theoretically grounded, and mixed methods approach, a survey was designed to encompass the social, environmental, and behavioral determinants of food insecurity and health promotion for Emergency Food Assistance System users. The final survey tested well for face and content validity and meets the criteria for internal reliability. This will aid to develop culturally tailored programs and policies for low-income, food insecure populations facing social and health disparities in this large urban neighborhood.