"You can't always get what you want…": economic thinking, constrained optimization and health professions education

Global research aims for the study of cost and value in health professions education: A Delphi study of international experts

BACKGROUND

Health professions educators constantly make difficult choices about the allocation of finite resources. However, there is limited sound research available to guide their decision-making. The purpose of this study was to address this gap by establishing international consensus on research aims, considering diverse economic and cultural contexts.

METHODS

The authors conducted a three-round Delphi study, engaging an international panel of 73 experts in education research. Panelists were asked to identify (round 1), rank (round 2), and revise (round 3) research aims important for the study of cost and value in medical education. Round 3 results were discussed by an international steering group of nine medical education scientists actively involved in cost and value research, who finalized a list of 20 research aims. Steering group narratives were analyzed to identify additional conceptual insights.

RESULTS

From 597 research aims suggested in round 1, 20 research aims were identified after steering group discussion. These were clustered into three categories: (1) funding mechanisms for medical education (e.g. financial policies, cost-effectiveness, and equity impacts); (2) cost and outcomes, e.g. how costs in health professions education relate to concrete outcomes; and (3) economic evaluation of teaching, assessment, and training approaches; e.g. designing and applying formal economic evaluation methods. Steering group discussions noted the limited integration of economic theories into medical education research and the need for foundational studies beyond immediate, practical priorities. They further noted lack of consensus on definitions of cost and value, and appropriate methodologies; underutilization of accepted health economics approaches; and infrequent interdisciplinary collaborations. These collectively act as barriers to advancing the field.

CONCLUSION

The field of cost and value in health professions education remains theoretically and empirically underdeveloped. The research aims identified herein provide a strategic framework for addressing cost and value comprehensively.

Go back to the original sources please!

This column is intended to address the kinds of knotty problems and dilemmas with which many scholars grapple in studying health professions education. In this article, we address the dilemma of engaging with foundational works versus depending on summary articles. We argue that an over-dependence on secondary sources can lead to prejudices and unquestioned assumptions, and limit the constructive development of our field. We urge health professions education scholars to honour the original literature not just for their own learning and understanding, but so that the field can move forward in developing theory and methodology.

Is There a Problem With Evidence in Health Professions Education?

ABSTRACT

What constitutes evidence, what value evidence has, and how the needs of knowledge producers and those who consume this knowledge might be better aligned are questions that continue to challenge the health sciences. In health professions education (HPE), debates on these questions have ebbed and flowed with little sense of resolution or progress. In this article, the authors explore whether there is a problem with evidence in HPE using thought experiments anchored in Argyris' learning loops framework.From a single-loop perspective ("How are we doing?"), there may be many problems with evidence in HPE, but little is known about how research evidence is being used in practice and policy. A double-loop perspective ("Could we do better?") suggests expectations of knowledge producers and knowledge consumers might be too high, which suggests more system-wide approaches to evidence-informed practice in HPE are needed. A triple-loop perspective ("Are we asking the right questions?") highlights misalignments between the dynamics of research and decision-making, such that scholarly inquiry may be better approached as a way of advancing broader conversations, rather than contributing to specific decision-making processes.The authors ask knowledge producers and consumers to be more attentive to the translation from knowledge to evidence. They also argue for more systematic tracking and audit of how research knowledge is used as evidence. Given that research does not always have to serve practical purposes or address the problems of a particular program or institution, the relationship between knowledge and evidence should be understood in terms of changing conversations and influencing decisions.

The impact of community of inquiry and self-efficacy on student attitudes in sustained remote health professions learning environments

BACKGROUND

Sustained remote learning environments, like those experienced in late 2020 due to the COVID-19 pandemic, share characteristics with online courses but were not intentionally designed to delivered virtually. The purpose of this study was to investigate the impact of Community of Inquiry, a widely used online learning environment framework, and self-efficacy on perceived student attitudes within sustained remote learning environments.

METHODS

An interinstitutional team of health professions education researchers collected survey data from 205 students representing a wide range of health professions in five U.S. institutions. Latent mediation models under structural equation modeling framework were used to examine whether student self-efficacy mediates the relationship between Community of Inquiry presence and student's favorability of sustained remote learning delivered in the prolonged stages of the COVID-19 pandemic.

RESULTS

Higher levels of teaching presence and social presence in the remote learning environment were associated with higher levels of remote learning self-efficacy which, in turn, predicts variance in positive attitudes toward remote learning. When mediated by self-efficacy, significant variance in student's favorability of sustained remote learning was explained by teaching presence (61%), social presence (64%), and cognitive presence (88%) and self-efficacy. Significant direct and indirect effects for teaching and social presence, and only direct effects for cognitive presence were observed.

CONCLUSIONS

This study establishes the Community of Inquiry and its three presence types as a relevant and stable framework for investigating sustained remote health professions teaching and learning environments, not only carefully designed online learning environments. Faculty may focus course design strategies which enhance presence and increase student self-efficacy for the sustained remote learning environment.

We know but we hope: A qualitative study of the opinions and experiences on the inclusion of management, health economics and research in the medical curriculum

The achievement of global and national health goals requires a health workforce that is sufficient and trained. Despite considerable steps in medical education, the teaching of management, health economics and research skills for medical doctors are often neglected in medical curricula. This study explored the opinions and experiences of medical doctors and academic educationalists on the inclusion of management, health economics and research in the medical curriculum. A qualitative study was undertaken at four medical schools in Southern Africa (February to April 2021). The study population was medical doctors and academic educationalists. Semi-structured interviews with purposively sampled participants were conducted. All interviews were recorded and professionally transcribed. Constructivist grounded theory guided the analysis with the use of ATLAS.ti version 9.1.7.0 software. In total, 21 academic educationalists and 28 medical doctors were interviewed. In the first theme We know, participants acknowledged the constraints of medical schools but were adamant that management needed to be taught intentionally and explicitly. The teaching and assessment of management and health economics was generally reported to be ad hoc and unstructured. There was a desire that graduates are able to use, but not necessarily do research. In comparison to management and research, support for the inclusion of health economics in the curriculum was insignificant. Under We hope, educationalists hoped that the formal clinical teaching will somehow instil values and best practices of management and that medical doctors would become health advocates. Most participants wished that research training could be optimised, especially in relation to the duration of allocated time; the timing in the curriculum and the learning outcomes. Despite acknowledgement that management and research are topics that need to be taught, educationalists appeared to rely on chance to teach and assess management in particular. These qualitative study findings will be used to develop a discrete choice experiment to inform optimal curricula design.

"Important but risky": attitudes of global thought leaders towards cost and value research in health professions education

Studies of cost and value can inform educational decision making, yet our understanding of the barriers to such research is incomplete. To address this gap, our aim was to explore the attitudes of global thought leaders in HPE towards cost and value research. This was a qualitative virtual interview study underpinned by social constructionism. In telephone or videoconference interviews in 2018-2019, we asked global healthcare professional thought leaders their views regarding HPE cost and value research, outstanding research questions in this area and why addressing these questions was important. Analysis was inductive and thematic, and incorporated review and comments from the original interviewees (member checking). We interviewed 11 thought leaders, nine of whom gave later feedback on our data interpretation (member checking). We identified four themes: Cost research is really important but potentially risky (quantifying and reporting costs provides evidence for decision-making but could lead to increased accountability and loss of autonomy); I don't have the knowledge and skills (lack of economic literacy); it's not what I went into education research to do (professional identity); and it's difficult to generate generalizable findings (the importance of context). This study contributes to a wider conversation in the literature about cost and value research by bringing in the views of global HPE thought leaders. Our findings provide insight to inform how best to engage and empower educators and researchers in the processes of asking and answering meaningful, acceptable and relevant cost and value questions in HPE.

BMAT's predictive validity for medical school performance: A retrospective cohort study

BACKGROUND

Although used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools.

METHODS

This was a retrospective cohort study of four ICSM (1188 students, entering 2010-2013) and three LKCMedicine cohorts (222 students, 2013-2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A-levels) in a subset of ICSM students.

RESULTS

When BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08-0.19; LKCMedicine: 0.22-0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09-0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A-level grades.

DISCUSSION

Schools who wish to assess scientific knowledge independently of A-levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

How to conduct cost and value analyses in health professions education: AMEE Guide No. 139

Growing demand for accountability, transparency, and efficiency in health professions education is expected to drive increased demand for, and use of, cost and value analyses. In this AMEE Guide, we introduce key concepts, methods, and literature that will enable novices in economics to conduct simple cost and value analyses, hold informed discussions with economic specialists, and undertake further learning on more advanced economic topics. The practical structure for conducting analyses provided in this guide will enable researchers to produce robust results that are meaningful and useful for improving educational practice. Key steps include defining the economic research question, identifying an appropriate economic study design, carefully identifying cost ingredients, quantifying, and pricing the ingredients consumed, and conducting sensitivity analyses to explore uncertainties in the results.