Using Test Question Templates to teach physiology core concepts

The impact of instruction on undergraduates' understanding of homeostasis: results from administering the homeostasis concept inventory

The Homeostasis Concept Inventory (HCI) is a validated instrument for measuring students' knowledge of homeostasis. It is comprised of 20 multiple-choice questions covering key components of the previously validated Homeostasis Conceptual Framework (HCF). In this paper, we present the first multi-institutional study of the impact of physiology instruction on students' HCI performance. Five cohorts of physiology or anatomy and physiology (A&P) students at four academic institutions took the HCI both at the start of their academic term (pretest) and at the end of their term (posttest). Statistically significant but relatively modest improvements in overall scores were seen from pretest to posttest. Among the 20 questions, 8 questions had incorrect choices identified as "attractive distractors" on the pretest, meaning that they were chosen at higher-than-random frequencies. From pretest to posttest, there were only modest declines in selections of incorrect answers generally and of attractive distractors in particular. Three attractive distractors that all target one specific misconception, that homeostatic mechanisms are active only when a regulated variable is not at its setpoint, remained persistently attractive except for students of one instructor who directly addressed that misconception in lecture and lab. These data are sobering in that they show a limited impact of instruction on HCI performance. However, these data also include encouraging evidence that instructional targeting of a specific misconception may help students overcome that misconception.NEW & NOTEWORTHY How is undergraduate students' understanding of homeostasis impacted by a physiology course? This study indicates that many students do not improve that much on a validated multiple-choice concept inventory but may improve noticeably on questions about a misconception if that misconception is specifically targeted by the instructor.

Frontier model chatbots can help instructors create, improve, and use learning objectives

Learning objectives (LOs) are a pillar of course design and execution and thus a focus of curricular reforms. This study explored the extent to which the creation and usage of LOs might be facilitated by three leading chatbots: ChatGPT-4o, Claude 3.5 Sonnet, and Google Gemini Advanced. We posed three main questions, as follows: question A: when given course content, can chatbots create LOs that are consistent with five best practices in writing LOs?; question B: when given LOs for a low level of the revised Bloom's taxonomy, can chatbots convert them to a higher level?; and question C: when given LOs, can chatbots create assessment questions that meet six criteria of quality? We explored these questions in the context of four undergraduate courses: Applied Exercise Physiology, Human Anatomy, Human Physiology, and Motor Learning. According to instructor ratings, chatbots had a >70% success rate on most individual criteria for questions A-C. However, chatbots' "difficulties" with a few criteria (e.g., provision of appropriate context for an LO's action, assignment of an appropriate revised Bloom's taxonomy level) meant that, overall, only 38.3% of chatbot outputs fully met all criteria and thus were possibly ready for use with students. Our findings thus underscore the continuing need for instructor oversight of chatbot outputs but also illustrate chatbots' potential to expedite the design and improvement of LOs and LO-related curricular materials such as test question templates (TQTs), which directly align LOs with assessment questions.NEW & NOTEWORTHY Amid much recent interest in the impact of generative artificial intelligence on education, one relatively underexplored issue is the extent to which instructors can use chatbots to work more efficiently. This study determined whether the challenging tasks of writing learning objectives (LOs) and writing LO-linked assessment questions can be delegated to advanced chatbots. The chatbots' outputs were often impressive yet often imperfect and thus can be useful as solid drafts that still require instructor oversight.

Chatbot responses suggest that hypothetical biology questions are harder than realistic ones

The biology education literature includes compelling assertions that unfamiliar problems are especially useful for revealing students' true understanding of biology. However, there is only limited evidence that such novel problems have different cognitive requirements than more familiar problems. Here, we sought additional evidence by using chatbots based on large language models as models of biology students. For human physiology and cell biology, we developed sets of realistic and hypothetical problems matched to the same lesson learning objectives (LLOs). Problems were considered hypothetical if (i) known biological entities (molecules and organs) were given atypical or counterfactual properties (redefinition) or (ii) fictitious biological entities were introduced (invention). Several chatbots scored significantly worse on hypothetical problems than on realistic problems, with scores declining by an average of 13%. Among hypothetical questions, redefinition questions appeared especially difficult, with many chatbots scoring as if guessing randomly. These results suggest that, for a given LLO, hypothetical problems may have different cognitive demands than realistic problems and may more accurately reveal students' ability to apply biology core concepts to diverse contexts. The Test Question Templates (TQT) framework, which explicitly connects LLOs with examples of assessment questions, can help educators generate problems that are challenging (due to their novelty), yet fair (due to their alignment with pre-specified LLOs). Finally, ChatGPT's rapid improvement toward expert-level answers suggests that future educators cannot reasonably expect to ignore or outwit chatbots but must do what we can to make assessments fair and equitable.

Teaching science with the "universal language" of music: alignment with the Universal Design for Learning framework

The idea of teaching science through music has undeniable appeal in implying that learning can be engaging and fun while also covering content efficiently. Indeed, there is little doubt that songs can be uniquely memorable, suggesting mnemonic options for core content. However, many classroom implementations of science music have limitations such as an overemphasis on rote memorization, rather than a constructivist building of understanding. In this brief review, we ask how music might facilitate the learning of science content in a manner consistent with the well-known pedagogical framework of Universal Design for Learning (UDL). In our view, UDL suggests certain distinct possible benefits of incorporating music into curricula, leading us to propose four models of practice. These four models are as follows: 1) students enjoy music together, 2) students critically analyze songs as texts, 3) students creatively augment existing songs, and 4) students create new songs. Model 1 can contribute to an inclusive learning environment, while models 2-4 can encourage cognitively rich active learning, and models 3-4 can additionally help students channel scientific understanding into the creation of authentic products. We conclude with comments on logistical issues that arise in implementing these four models, including the use of appropriate rubrics and the prioritization of artistic quality.NEW & NOTEWORTHY Instructors and students often find it fun to incorporate music into science classes. However, the casual usage of music in this context can unintentionally convey that science courses are mostly about memorizing scientific facts. In this article, the authors argue for a more nuanced approach to teaching science with music, rooted in Universal Design for Learning (UDL).