Measuring Faculty Effort: A Quantitative Approach That Aligns Personal and Institutional Goals in Pathology at Yale

Development and initial psychometric properties of the Research Complexity Index

Objective

Research study complexity refers to variables that contribute to the difficulty of a clinical trial or study. This includes variables such as intervention type, design, sample, and data management. High complexity often requires more resources, advanced planning, and specialized expertise to execute studies effectively. However, there are limited instruments that scale study complexity across research designs. The purpose of this study was to develop and establish initial psychometric properties of an instrument that scales research study complexity.

Methods

Technical and grammatical principles were followed to produce clear, concise items using language familiar to researchers. Items underwent face, content, and cognitive validity testing through quantitative surveys and qualitative interviews. Content validity indices were calculated, and iterative scale revision was performed. The instrument underwent pilot testing using 2 exemplar protocols, asking participants (n = 31) to score 25 items (e.g., study arms, data collection procedures).

Results

The instrument (Research Complexity Index) demonstrated face, content, and cognitive validity. Item mean and standard deviation ranged from 1.0 to 2.75 (Protocol 1) and 1.31 to 2.86 (Protocol 2). Corrected item-total correlations ranged from .030 to .618. Eight elements appear to be under correlated to other elements. Cronbach's alpha was 0.586 (Protocol 1) and 0.764 (Protocol 2). Inter-rater reliability was fair (kappa = 0.338).

Conclusion

Initial pilot testing demonstrates face, content, and cognitive validity, moderate internal consistency reliability and fair inter-rater reliability. Further refinement of the instrument may increase reliability thus providing a comprehensive method to assess study complexity and related resource quantification (e.g., staffing requirements).

AOA Critical Issues Symposium: So, You Want to Be a Department Leader: Essentials for Success

ABSTRACT

Most health systems are vertically integrated, and the leaders of orthopaedic surgery departments or service lines must have a comprehensive understanding of their role in the strategic plan of the health system. Orthopaedic surgery departments must be profitable while supporting the tripartite mission of excellence in clinical care, research, and education. This symposium had 4 specific objectives: to discuss how to (1) create synergy between the department or service line and the health system, (2) develop a strategy to enhance financial stability and revenue growth, (3) develop a comprehensive plan to enhance recruitment and retention of a diverse faculty, and (4) consider alternative strategies to foster education and research, even when the health system may be more focused on revenue generation.

A National Survey of Perceptions Around Conditions Associated With Pharmacy Faculty Workload Equity

OBJECTIVE

To assess pharmacy faculty members' perceptions of conditions associated with workload equity and factors that can improve workload equity.

METHODS

A 26-item survey instrument was developed and distributed via email to members of the American Association of Colleges of Pharmacy Council of Faculties. Questions pertained to the workload distribution, fairness in assignment, and perception of the conditions associated with workload equity (transparency, context, credit, clarity, norms, and accountability) as well as institutional and individual demographics.

RESULTS

A total of 662 responses were obtained (response rate 15.9%). Respondents' demographics were comparable to available national data. Approximately 41% of respondents reported their institutions did not have a written faculty workload policy. Most respondents reported their workload assignment was fair (highest with research/scholarship) but reported only moderate alignment between assigned and actual workloads. The rating level for what domains the primary decision maker uses to assign workload was highest for context, followed by credit, clarity, and transparency. Transparency was reported as the most needed condition to improve faculty perception of workload equity. Respondents also rated increasing trust between leadership and faculty and increasing productivity and accountability as the most important reasons to minimize workload inequities.

CONCLUSION

This was the first national survey of pharmacy faculty perceptions around the conditions associated with workload equity. Though additional research is needed in this area, programs can work to implement strategies associated with all of the conditions, particularly transparency, to improve faculty perceptions of equity.

Educational Relative Value Units as a Measure of Academic Productivity: A Systematic Review

INTRODUCTION

Academic Health Centers (AHCs) have complex, often competing missions. Many have developed mission-based management (MBM) systems to support their clinical and non-clinical missions. There are limited data on MBM use for their educational missions. Our scoping review explored how AHCs employed such systems.  Materials and methods: Arksey and O'Malley's six-stage framework guided our review. Based on pre-defined criteria, English language articles from PubMed, EMBASE, SCOPUS, and the Healthcare Administration Database published between 2010 and 2020 were loaded into a reference manager. The search included all health professions education schools. Articles were excluded if they were review articles, commentaries, or clearly did not involve funding for education. From the final list of selected articles, data were extracted using a data extraction sheet we developed. Two researchers reviewed each article again to ensure extracted data were reported consistently and with sufficient detail.  Results: Of the 1729 manuscripts identified, 35 met inclusion criteria. Sixteen (46%) contained data in some form but did not have a formal methods section describing the specific approach to data collection and analysis. Moreover, there was marked variability in how educational effort was quantified, what counted as educational effort (educational scholarship versus teaching) and the impacts of such quantification (departmental funding versus individual faculty incentives). None of the studies reported on the impact on faculty promotion. Faculty satisfaction with the system was reported in seven studies (20%) and was generally positive.

CONCLUSIONS

A systematic description of how systems were developed to support the educational mission was lacking. Clear goals, methods of development, uniform data on educational productivity and quality, and program evaluation were not defined by most articles. This lack of process clarity presents a challenge, but more importantly an opportunity for academic health centers to unify efforts and continue to further their educational mission.