Implementation and Evaluation of a National Well-Being Curriculum for KL2 Scholars

Introduction: The study aimed to pilot test a well-being curriculum for KL2 scholars to be used across the Clinical and Translational Science Award consortium. Methods: Between November 2022, and May 2023, 36 KL2 scholars from 25 hubs participated in the program. The General Well-Being Index for U.S. Workers and the Patient Reported Outcomes Measurement Information System (PROMIS-29) were completed by scholars before and after the program. Results: Postparticipation, there was a trend of improvement in the domains of well-being, sleep, anxiety, and fatigue. Conclusion: Implementing a virtual synchronous well-being curriculum allowed the scholars to connect across the consortium and improve their well-being.

The Evolve to Next-Gen ACT Network: An evolving open-access, real-world data resource primed for real-world evidence research across the Clinical and Translational Science Award Consortium

The ACT Network was funded by NIH to provide investigators from across the Clinical and Translational Science Award (CTSA) Consortium the ability to directly query national federated electronic health record (EHR) data for cohort discovery and feasibility assessment of multi-site studies. NIH refunded the program for expanded research application to become "Evolve to Next-Gen ACT" (ENACT). In parallel, the US Food and Drug Administration has been evaluating the use of real-world data (RWD), including EHR data, as sources of real-world evidence (RWE) for its regulatory decisions involving drug and biological products. Using insights from implementation science, six lessons learned from ACT for developing and sustaining RWD/RWE infrastructures and networks across the CTSA Consortium are presented in order to inform ENACT's development from the outset. Lessons include intentional institutional relationship management, end-user engagement, beta-testing, and customer-driven adaptation. The ENACT team is also conducting customer discovery interviews with CTSA hub and investigators using Innovation-Corps@NCATS (I-Corps™) methodology for biomedical entrepreneurs to uncover unmet RWD needs. Possible ENACT value proposition hypotheses are presented by stage of research. Developing evidence about methods for sustaining academically derived data infrastructures and support can advance the science of translation and support our nation's RWD/RWE research capacity.

Scholarly Productivity Evaluation of KL2 Scholars Using Bibliometrics and Federal Follow-on Funding: Cross-Institution Study

Background

Evaluating outcomes of the clinical and translational research (CTR) training of a Clinical and Translational Science Award (CTSA) hub (eg, the KL2 program) requires the selection of reliable, accessible, and standardized measures. As measures of scholarly success usually focus on publication output and extramural funding, CTSA hubs have started to use bibliometrics to evaluate the impact of their supported scholarly activities. However, the evaluation of KL2 programs across CTSAs is limited, and the use of bibliometrics and follow-on funding is minimal.

Objective

This study seeks to evaluate scholarly productivity, impact, and collaboration using bibliometrics and federal follow-on funding of KL2 scholars from 3 CTSA hubs and to define and assess CTR training success indicators.

Methods

The sample included KL2 scholars from 3 CTSA institutions (A-C). Bibliometric data for each scholar in the sample were collected from both SciVal and iCite, including scholarly productivity, citation impact, and research collaboration. Three federal follow-on funding measures (at the 5-year, 8-year, and overall time points) were collected internally and confirmed by examining a federal funding database. Both descriptive and inferential statistical analyses were computed using SPSS to assess the bibliometric and federal follow-on funding results.

Results

A total of 143 KL2 scholars were included in the sample with relatively equal groups across the 3 CTSA institutions. The included KL2 scholars produced more publications and citation counts per year on average at the 8-year time point (3.75 publications and 26.44 citation counts) than the 5-year time point (3.4 publications vs 26.16 citation counts). Overall, the KL2 publications from all 3 institutions were cited twice as much as others in their fields based on the relative citation ratio. KL2 scholars published work with researchers from other US institutions over 2 times (5-year time point) or 3.5 times (8-year time point) more than others in their research fields. Within 5 years and 8 years postmatriculation, 44.1% (63/143) and 51.7% (74/143) of KL2 scholars achieved federal funding, respectively. The KL2-scholars of Institution C had a significantly higher citation rate per publication than the other institutions (P<.001). Institution A had a significantly lower rate of nationally field-weighted collaboration than did the other institutions (P<.001). Institution B scholars were more likely to have received federal funding than scholars at Institution A or C (P<.001).

Conclusions

Multi-institutional data showed a high level of scholarly productivity, impact, collaboration, and federal follow-on funding achieved by KL2 scholars. This study provides insights on the use of bibliometric and federal follow-on funding data to evaluate CTR training success across institutions. CTSA KL2 programs and other CTR career training programs can benefit from these findings in terms of understanding metrics of career success and using that knowledge to develop highly targeted strategies to support early-stage career development of CTR investigators.

Using bibliometrics to evaluate translational science training: evidence for early career success of KL2 scholars

Introduction:

Evaluating clinical and translational research (CTR) mentored training programs is challenging because no two programs are alike. Careful selection of appropriate metrics is required to make valid comparisons between individuals and between programs. The KL2 program provides mentored-training for early-stage CTR investigators. Clinical and Translational Awards across the country have unique KL2 programs. The evaluation of KL2 programs has begun to incorporate bibliometrics to measure KL2 scholar and program impact.

Methods:

This study investigated demographic differences in bibliometric performance and post-K award funding of KL2 scholars and compared the bibliometric performance and post-K award federal funding of KL2 scholars and other mentored-K awardees at the same institution. Data for this study included SciVal and iCite bibliometrics and National Institutions of Health RePORTER grant information for mentored-K awardees (K08, K23, and KL2) at Case Western Reserve University between 2005 and 2013.

Results:

Results showed no demographics differences within the KL2 program scholars. Bibliometric differences between KL2 and other mentored-K awardee indicated an initial KL2 advantage for the number of publications at 5 years’ post-matriculation (i.e., the start of the K award). Regression analyses indicated the number of initial publications was a significant predictor of federal grant funding at the same time point. Analysis beyond the 5-year post-matriculation point did not result in a sustained, significant KL2 advantage.

Conclusions:

Factors that contributed to the grant funding advantage need to be determined. Additionally, differences between translational and clinical bibliometrics must be interpreted with caution, and appropriate metrics for translational science must be established.

The integration of emerging omics approaches to advance precision medicine: How can regulatory science help?

Building on the recent advances in next-generation sequencing, the integration of genomics, proteomics, metabolomics, and other approaches hold tremendous promise for precision medicine. The approval and adoption of these rapidly advancing technologies and methods presents several regulatory science considerations that need to be addressed. To better understand and address these regulatory science issues, a Clinical and Translational Science Award Working Group convened the Regulatory Science to Advance Precision Medicine Forum. The Forum identified an initial set of regulatory science gaps. The final set of key findings and recommendations provided here address issues related to the lack of standardization of complex tests, preclinical issues, establishing clinical validity and utility, pharmacogenomics considerations, and knowledge gaps.

Increasing research mentor training among biomedical researchers at Clinical and Translational Science Award hubs: The impact of the facilitator training initiative

As part of their mission, Clinical and Translational Science Award (CTSA) hubs are charged with developing, testing, and disseminating evidence-based practices to other CTSA hubs. Over the past 7 years, the University of Wisconsin-Madison has answered this charge by implementing the facilitator training (FT) initiative for research mentors. Three elements to advance training across the CTSA hubs have been critical: (1) using an FT model to empower others to build research mentor training at their local institutions; (2) tracking implementation of training events across the CTSA hubs over time; and (3) partnering with implementation sites to build local capacity and evaluate the effectiveness and quality of training. Here we report that facilitators have been trained at 75% of CTSA hubs. These facilitators report high satisfaction with the training and increased confidence in their ability to implement mentor training, and plan to implement local mentor training. These findings demonstrate that the FT initiative can serve as a model for dissemination and implementation of other workforce development interventions across the CTSA hubs.

CLARA: an integrated clinical research administration system

Administration of human subject research is complex, involving not only the institutional review board but also many other regulatory and compliance entities within a research enterprise. Its efficiency has a direct and substantial impact on the conduct and management of clinical research. In this paper, we report on the Clinical Research Administration (CLARA) platform developed at the University of Arkansas for Medical Sciences. CLARA is a comprehensive web-based system that can streamline research administrative tasks such as submissions, reviews, and approval processes for both investigators and different review committees on a single integrated platform. CLARA not only helps investigators to meet regulatory requirements but also provides tools for managing other clinical research activities including budgeting, contracting, and participant schedule planning.

The CTSA mentored career development program: supporting the careers of child health investigators

Training translational scientists is a priority of the Clinical and Translational Science Award (CTSA) consortium.

OBJECTIVES

1) Describe the landscape of CTSA Mentored Research Career Development Awards (CDA) and 2) evaluate participation and outcomes of child health investigators in these programs.

DESIGN

Survey of the CTSA Child Health Oversight Committee (CC-CHOC) and review of nonresponders' CTSA Websites.

RESULTS

Thirty-two of 53 CC-CHOC members (60%) responded and all nonresponder Websites were reviewed. Institutions supported 1,166 CDA positions from 2006 to 2011, with 134 awarded to child health investigators (11.5%). Respondents reported a mean of 29.8 KL2 positions (95% CI 17.5-42.2) during their award period, with a mean of 2.8 (95% CI 1.8-3.8) awarded to child health investigators. The proportion of child health awardees varied from 0% to 50% across institutions. We identified 45 subsequent National Institutes of Health (NIH) awards to the 134 child health investigators (34%).

CONCLUSIONS

The CTSA program contributes substantially to training the next generation of translational investigators. One-third of child health investigators obtained subsequent NIH awards in the short follow-up period demonstrating success of the CTSA CDA programs. Child health investigators are represented variably across the consortium. Pediatric institutions can partner with the CTSA program to further support training child health investigators.

Expanding research to provide an evidence base for nutritional interventions for the management of inborn errors of metabolism

A trans-National Institutes of Health initiative, Nutrition and Dietary Supplement Interventions for Inborn Errors of Metabolism (NDSI-IEM), was launched in 2010 to identify gaps in knowledge regarding the safety and utility of nutritional interventions for the management of inborn errors of metabolism (IEM) that need to be filled with evidence-based research. IEM include inherited biochemical disorders in which specific enzyme defects interfere with the normal metabolism of exogenous (dietary) or endogenous protein, carbohydrate, or fat. For some of these IEM, effective management depends primarily on nutritional interventions. Further research is needed to demonstrate the impact of nutritional interventions on individual health outcomes and on the psychosocial issues identified by patients and their families. A series of meetings and discussions were convened to explore the current United States' funding and regulatory infrastructure and the challenges to the conduct of research for nutritional interventions for the management of IEM. Although the research and regulatory infrastructure are well-established, a collaborative pathway that includes the professional and advocacy rare disease community and federal regulatory and research agencies will be needed to overcome current barriers.