Evaluation of a Training Program to Improve Organizational Capacity for Health Systems Analytics

OBJECTIVE

The Leadership in Analytics and Data Science (LEADS) course was evaluated for effectiveness. LEADS was a 6-month program for working biomedical and health informatics (BMHI) professionals designed to improve analytics skills, knowledge of enterprise applications, data stewardship, and to foster an analytics community of practice through lectures, hands-on skill building workshops, networking events, and small group projects.

METHODS

The effectiveness of the LEADS course was evaluated using the Kirkpatrick Model by assessing pre- and postcourse knowledge, analytics capabilities, goals, practice, class lecture reaction, and change in the size of participant professional networks. Differences in pre- and postcourse responses were analyzed with a Wilcoxon signed rank test to determine significance, and effect sizes were computed using a z-statistic.

RESULTS

Twenty-nine students completed the course with 96% of respondents reporting that they were "very" or "extremely" likely to recommend the course. Participants reported improvement in several analytics capabilities including Epic data warehousing (p = 0.017), institutional review board policy (p = 0.005), and data stewardship (p = 0.007). Changes in practice patterns mirrored those in self-reported capability. On average, the participant professional network doubled.

CONCLUSION

LEADS was the first course targeted to working BMHI professional at a large academic medical center to have a formal effectiveness evaluation be published in the literature. The course achieved the goals of expansion of BMHI knowledge, skills, and professional networks. The LEADS course provides a template for continuing education of working BMHI professionals.

Curriculum inventory: Modeling, sharing and comparing medical education programs

Abstract descriptions of how curricula are structured and run. The American National Standards Institute (ANSI) MedBiquitous Curriculum Inventory Standard provides a technical syntax through which a wide range of different curricula can be expressed and subsequently compared and analyzed. This standard has the potential to shift curriculum mapping and reporting from a somewhat disjointed and institution-specific undertaking to something that is shared among multiple medical schools and across whole medical education systems. Given the current explosion of different models of curricula (time-free, competency-based, socially accountable, distributed, accelerated, etc.), the ability to consider this diversity using a common model has particular value in medical education management and scholarship. This article describes the development and structure of the Curriculum Inventory Standard as a way of standardizing the modeling of different curricula for audit, evaluation and research purposes. It also considers the strengths and limitations of the current standard and the implications for a medical education world in which this level of commonality, precision, and accountability for curricular practice is the norm rather than the exception.

Virtual patients come of age

On-screen simulations of clinical settings have been used for educational purposes since the 1970s. Despite this, it is only now that these 'virtual patients' are increasingly forming a part of the medical education mainstream. Enabling factors for these changes include a requirement for more assured clinical encounters, changes in patient availability (in particular, in tertiary contexts), diminishing technical and cost barriers and ongoing changes in educational practices as a whole. This special edition of Medical Teacher presents a number of papers covering key factors in the development, use and evaluation of virtual patients in contemporary medical education practice.

eViP: sharing virtual patients across Europe

Virtual Patients are interactive computer programs that simulate real life clinical scenarios for educational purposes. The European Commission co-funded Electronic Virtual Patient (eViP) programme is a collaboration among 8 universities working towards creating a shareable bank of virtual patients. eViP is creating an application profile of the MedBiquitous Virtual Patient specification to enable the exchange of interoperable virtual patient activities across institutions.

The MERG Suite: Tools for discovering competencies and associated learning resources

BACKGROUND

As the demands for competency-based education grow, the need for standards-based tools to allow for publishing and discovery of competency-based learning content is more pressing. This project focused on developing federated discovery services for competency-based medical e-learning content.

METHODS

We built a tool suite for authoring and discovery of medical e-learning metadata. The end-user usability of the tool suite was evaluated through a web-based survey.

RESULTS

The suite, implemented as an open-source system, was evaluated to identify areas for improvement.

CONCLUSION

The MERG suite is a starting point for organizations implementing competency-based e-learning resources.

Sharing innovation: the case for technology standards in health professions education

Information technologies have provided fertile ground for innovation in healthcare education, but too often these innovations have been limited in scope and impact. One way of addressing these limitations is the development of common and open technology standards to scale innovation across organizational boundaries. Research on the diffusion of standards indicates that environmental forces, such as regulatory changes, top-down management support, and feasibility are key determinants of standards adoption. This paper describes the perspective and work of MedBiquitous, the only internationally recognized standards body in healthcare education. Many innovators are implementing MedBiquitous healthcare education standards to effect change within and across organizations. In a resource-constrained and knowledge intensive domain such as healthcare education, collaboration is an imperative. Technology standards are essential to raise the quality of healthcare education and assessment in a cost-effective manner.

An XML standard for virtual patients: exchanging case-based simulations in medical education

Virtual Patients are computer-based simulations of a clinical encounter where the user plays the role of a healthcare provider while receiving in-context instruction. This unique pedagogical approach enables active case-based learning for learners. Academic institutions around the world have developed high-quality virtual patients using many different authoring and playback technologies. However, sustainability and scalability have proved challenging due to the number of cases needed and production costs. In an effort to promote sharing of Virtual Patients and broader adoption into medical education at all levels, MedBiquitous organized an international working group to create an XML-based "MedBiquitous Virtual Patient Standard" (MVP) describing a common structure for virtual patient content and activities. The MVP enables virtual patient exchange across systems, modification, and display within conformant player software.

Adopting e-learning standards in health care: competency-based learning in the medical informatics domain

Like many forms of education, health professions education is increasingly competency-based. At the same time, there is growing use of e-learning technologies, which can be linked to competencies via emerging e-learning standards. Health care has been slow to adopt competencies and e-learning standards. We report our efforts to facilitate access to competencies and e-learning content in the medical informatics domain, linked by content-competency associations, based on standards developed by the MedBiquitous Consortium. We demonstrate that such standards can be successfully used and their implementation in other domains is warranted.

A standards-based approach for facilitating discovery of learning objects at the point of care

Recent concerns about the quality and safety of healthcare practice provide an imperative for discovering and accessing learning resources. The growing ubiquity of the Internet, World Wide Web, and on-line educational content provide opportunity for healthcare practitioners to identify and master learning in a granular and rapid fashion. The e-learning community at large has developed a number of standards to facilitate interoperability of learner competencies, metadata describing on-line content, and packaging and navigation of such content. The overall goal of our project is to enable healthcare professionals to easily and rapidly discover learning content at the point of care. This discovery and access of learning content will be based on healthcare-specific extensions of existing e-learning standards, which are themselves based on other Web standards, such as Web Services.