Can Critical Thinking Skills Be Transferred from the Accomplished Scientist to the Beginning Scientist?

We seldom apply rigorous methods to how scientists think as they make new discoveries and open new avenues of inquiry. The progress of science relies on one generation of scientists transferring their thought processes and methods to the next. In a culture of inquiry, respect, rigor, and role modeling, critical thinking can thrive as novices become accomplished scientists. Although scientists are typically judged on the basis of results (publication, symposium, etc.) rather than the thought process used to get that result, the thought process is of paramount importance for training the next generation of scientists. As a profession pays closer attention to the educational process whereby novices become accomplished scientists, a wider discussion of how accomplished scientists think is vital to help novice scientists positively advance through the early vulnerable years. Thinking critically about our goals and values-including the value of emulation, experience, biases, and doubt-is a necessary part of that process.

Life Science Research Immersion Program Improves STEM-Specific Skills and Science Attitudes among Precollege Students

A predicted rapid growth in science, technology, engineering, and math (STEM) careers demands a vast and talented workforce, but students most commonly abandon STEM majors within the first 2 years of college. Performance in introductory courses, scientific literacy, and the ability to critically reason are main predictors of retention in STEM, highlighting the importance of precollege and early college experience. The Life Science Research Immersion Program (LSRIP) is a novel science education model that focuses on the development of scientific research skills, thus preparing students for introductory college courses and beyond. To evaluate the efficacy of the LSRIP, pre- and postprogram assessments and surveys were administered to three precollege student cohorts. Scientific reasoning assessment scores improved by 4.70% in Summer 2019 (P < 0.01), 9.44% in Fall 2019 (P < 0.05), and 0.97% in Winter 2020 cohorts, with two of five questions showing statistically significant improvement. Surveyed attitudes toward science improved in 62.9% of questions across all cohorts. These results suggest that research immersion experiences are an effective educational instrument for improving and promoting scientific reasoning and attitudes among precollege students. To better prepare students for success in STEM higher education and careers, we recommend implementing LSRIPs to complement traditional precollege science curricula.

Consideration of culture in cognition: How we can enrich methodology and theory

In this paper, we argue that adopting an inclusive approach where diverse cultures are represented in research is of prime importance for cognitive psychology. The overrepresentation of participant samples and researchers from WEIRD (Western, Educated, Industrialized, Rich, and Democratic) cultures limits the generalizability of findings and fails to capture potential sources of variability, impeding understanding of human cognition. In an analysis of articles in representative cognitive psychology journals over the five-year period of 2016-2020, we find that only approximately 7% of articles consider culture, broadly defined. Of these articles, a majority (83%) focus on language or bilingualism, with small numbers of articles considering other aspects of culture. We argue that methodology and theory developed in the last century of cognitive research not only can be leveraged, but will be enriched by greater diversity in both populations and researchers. Such advances pave the way to uncover cognitive processes that may be universal or systematically differ as a function of cultural variations, and the individual differences in relation to cultural variations. To make a case for broadening this scope, we characterize relevant cross-cultural research, sample classic cognitive research that is congruent with such an approach, and discuss compatibility between a cross-cultural perspective and the classic tenets of cognitive psychology. We make recommendations for large and small steps for the field to incorporate greater cultural representation in the study of cognition, while recognizing the challenges associated with these efforts and acknowledging that not every research question calls for a cross-cultural perspective.

Causal theory error in college students' understanding of science studies

When reasoning about science studies, people often make causal theory errors by inferring or accepting a causal claim based on correlational evidence. While humans naturally think in terms of causal relationships, reasoning about science findings requires understanding how evidence supports—or fails to support—a causal claim. This study investigated college students’ thinking about causal claims presented in brief media reports describing behavioral science findings. How do science students reason about causal claims from correlational evidence? And can their reasoning be improved through instruction clarifying the nature of causal theory error? We examined these questions through a series of written reasoning exercises given to advanced college students over three weeks within a psychology methods course. In a pretest session, students critiqued study quality and support for a causal claim from a brief media report  suggesting an association between two variables. Then, they created diagrams depicting possible alternative causal theories. At the beginning of the second session, an instructional intervention introduced students to an extended example of a causal theory error through guided questions about possible alternative causes. Then, they completed the same two tasks with new science reports immediately and again 1 week later. The results show students’ reasoning included fewer causal theory errors after the intervention, and this improvement was maintained a week later. Our findings suggest that interventions aimed at addressing reasoning about causal claims in correlational studies are needed even for advanced science students, and that training on considering alternative causal theories may be successful in reducing casual theory error.

Preservice Biology Teachers’ Scientific Reasoning Skills and Beliefs about Nature of Science: How Do They Develop and Is There a Mutual Relationship during the Development?

Scientific reasoning (SR) skills and nature of science (NOS) beliefs represent important characteristics of biology teachers’ professional competence. In particular, teacher education at university is formative for the professionalization of future teachers and is thus the focus of the current study. Our study aimed to examine the development of SR skills and NOS beliefs and their mutual relationship during teacher education. We applied paper-and-pencil tests to measure SR skills and NOS beliefs of 299 preservice biology teachers from 25 universities in Germany. The results of linear mixed models and planned comparisons revealed that both SR skills and NOS beliefs develop over the course of the study. Nevertheless, the development of SR skills and multiple aspects of NOS beliefs proceeds in different trajectories. Cross-lagged models showed a complex picture concerning the mutual relationship between SR skills and NOS beliefs during their development (both positive and negative). The current study contributes to the existing research because it is based on longitudinal data and allows—in contrast to cross-sectional research—conclusions about the development of SR skills and NOS beliefs.

High-School Students’ Topic-Specific Epistemic Beliefs about Climate Change: An Assessment-Related Study

This study belongs to assessment-related research and aimed to investigate Finnish high-school students’ (n = 211) topic-specific epistemic beliefs about climate change and whether the Norwegian topic-specific epistemic beliefs questionnaire (TSEBQ) was also valid among Finnish respondents. Thus, research data were not only derived from the TSEBQ but also from topic knowledge tests and students’ views on their favorite school subjects and interest in science subjects. Principal component analysis (PCA) showed that the statistical model, originally based on 49 questions, was congruent with the Norwegian four-factor model (Certification, Source, Justification and Simplicity). However, according to the reliability analysis and confirmatory factor analysis (CFA), the performance of the Simplicity factor was unclear. In CFA, the three-factor structure (without Simplicity) was supported. The effects of topic knowledge, topic interest and gender on the TSEBQ factors were examined by using hierarchical regression analysis (HRA). The TSEBQ was shown to be a reliable tool for measuring the topic-specific epistemic beliefs of Finnish students. More specifically, the results support the claim that topic-specific epistemic beliefs can be educationally and culturally bound. HRA showed that students’ topic knowledge in chemistry and biology was related to certainty of knowledge and justification for knowing. Moreover, female students performed significantly better in topic knowledge and more often planned to pursue a science career in the future.

Using Differential Item Functioning to Analyze the Domain Generality of a Common Scientific Reasoning Test

Abstract. A significant problem that assessments of scientific reasoning face at the level of higher education is the question of domain generality, that is, whether a test will produce biased results for students from different domains. This study applied three recently developed methods of analyzing differential item functioning (DIF) to evaluate the domain generality assumption of a common scientific reasoning test. Additionally, we evaluated the usefulness of these new, tree- and lasso-based, methods to analyze DIF and compared them with methods based on classical test theory. We gave the scientific reasoning test to 507 university students majoring in physics, biology, or medicine. All three DIF analysis methods indicated a domain bias present in about one-third of the items, mostly benefiting biology students. We did not find this bias by using methods based on classical test theory. Those methods indicated instead that all items were easier for physics students compared to biology students. Thus, the tree- and lasso-based methods provide a clear added value to test evaluation. Taken together, our analyses indicate that the scientific reasoning test is neither entirely domain-general, nor entirely domain-specific. We advise against using it in high-stakes situations involving domain comparisons.

The Development of Cognitive Reflection in China

Cognitive reflection is the tendency to override an intuitive response so as to engage in the reflection necessary to derive a correct response. Here, we examine the emergence of cognitive reflection in a culture that values nonanalytic thinking styles, Chinese culture. We administered a child-friendly version of the cognitive reflection test, the CRT-D, to 130 adults and 111 school-age children in China and compared performance on the CRT-D to several measures of rational thinking (belief bias syllogisms, base rate sensitivity, denominator neglect, and other-side thinking) and normative thinking dispositions (actively open-minded thinking and need for cognition). The CRT-D was a significant predictor of rational thinking and normative thinking dispositions in both children and adults, as previously found in American samples. Adults' performance on the CRT-D correlated with their performance on the original CRT, and children's performance on the CRT-D predicted rational thinking and normative thinking dispositions even after adjusting for age. These results demonstrate that cognitive reflection, rational thinking, and normative thinking dispositions converge even in a culture that emphasizes holistic, nonanalytic reasoning.

Student Behaviors and Interactions Influence Group Discussions in an Introductory Biology Lab Setting

Past research on group work has primarily focused on promoting change through implementation of interventions designed to increase performance. Recently, however, education researchers have called for more descriptive analyses of group interactions. Through detailed qualitative analysis of recorded discussions, we studied the natural interactions of students during group work in the context of a biology laboratory course. We analyzed multiple interactions of 30 different groups as well as data from each of the 91 individual participants to characterize the ways students engage in discussion and how group dynamics promote or prevent meaningful discussion. Using a set of codes describing 15 unique behaviors, we determined that the most common behavior seen in student dialogue was analyzing data, followed by recalling information and repeating ideas. We also classified students into one of 10 different roles for each discussion, determined by their most common behaviors. We found that, although students cooperated with one another by exchanging information, they less frequently fully collaborated to explain their conclusions through the exchange of reasoning. Within this context, these findings show that students working in groups generally choose specific roles during discussions and focus on data analysis rather than constructing logical reasoning chains to explain their conclusions.

Sustainable Education Starts in the Classroom

Definitions of a sustainable higher education focus on the different factors that are critical to the continued existence of the institution, the people it serves, and the surrounding society. If higher education is assumed to be a conduit for the acquisition of knowledge and skills that can contribute to a healthy, ethical, and sustainable society, then it has to be able to induce lasting behavioral change in its primary beneficiaries (i.e., students). In the age of fake news, misrepresentation, and rejection of scientific principles and facts, we identified cognitive operations that are key to scientific reasoning (i.e., apply, analyze and evaluate), and offered sustainable practice to students enrolled in a course devoted to scientific writing. Students were classified as possessing an inclination towards a reproductive mode of learning, which could increase their vulnerability to absorb fabrications and distortions of information. The research first asked whether practice in applying, evaluating, and analyzing induces an information processing change (as measured by the content of scientific writing). Then, it asked whether environmental disruptions (e.g., shifting from face-to-face instruction, a mode familiar to students, to online instruction due to the COVID-19 pandemic) would affect the likelihood of change. We found that this type of practice was an effective propeller of change in students’ scientific reasoning. A disposition towards reproductive learning did not impair scientific reasoning, whereas engagement and practice made a positive contribution. We concluded that behavioral change is blocked by neither the availability of technology, nor the learner’s use, but rather by one’s motivation to make use of opportunities for change. A sudden alteration in the learning environment may create uncertainty but does not substantially alter this motivation. The findings of the present study can be useful to the development of a sustainable education in the Middle East and beyond.

Student, instructor, and observer agreement regarding frequencies of scientific teaching practices using the Measurement Instrument for Scientific Teaching-Observable (MISTO)

BACKGROUND

The Scientific Teaching (ST) pedagogical framework encompasses many of the best practices recommended in the literature and highlighted in national reports. Understanding the growth and impact of ST requires instruments to accurately measure the extent to which practitioners implement ST in their courses. Researchers have typically relied on students, instructors, or observers to document course teaching practices, but it remains unclear whether and how these perspectives differ from each other. To address this issue, we modified our previously published instrument to generate the Measurement Instrument for Scientific Teaching-Observable (MISTO), which can be completed by students, instructors, and observers, and we investigated the degree of similarity between these three perspectives across 70 undergraduate science courses at seven different institutions in the USA.

RESULTS

We found that the full MISTO and Active Learning subcategory scores showed the highest correlations among the three perspectives, but the degree of correlation between perspectives varied for the other subcategories. Match scores between students and instructors were significantly higher than observer matches for the full MISTO and for the Active Learning, Inclusivity, and Responsiveness subcategories.

CONCLUSIONS

We find that the level and type of agreement between perspectives varies across MISTO subcategories and that this variation likely stems from intrinsic differences in the course access and scoring decisions of the three perspectives. Building on this data, we recommend MISTO users consider their research goals, available resources, and potential artifacts that may arise when deciding which perspective best fits their needs in measuring classroom teaching practices.

Critical Thinking Theory to Practice: Using the Expert's Thought Process as Guide for Learning and Assessment

Critical thinking skills are essential for the successful dentist, yet few explicit skillsets in critical thinking have been developed and published in peer-reviewed literature. The aims of this article are to 1) offer an assessable critical thinking teaching model with the expert's thought process as the outcome, learning guide, and assessment instrument and 2) offer three critical thinking skillsets following this model: for geriatric risk assessment, technology decision making, and situation analysis/reflections. For the objective component, the student demonstrates delivery of each step in the thought process. For the subjective component, the student is judged to have grasped the principles as applied to the patient or case. This article describes the framework and the results of pilot tests in which students in one year at this school used the model in the three areas, earning scores of 90% or above on the assessments. The model was thus judged to be successful for students to demonstrate critical thinking skillsets in the course settings. Students consistently delivered each step of the thought process and were nearly as consistent in grasping the principles behind each step. As more critical thinking skillsets are implemented, a reinforcing network develops.

Promoting the Multidimensional Character of Scientific Reasoning

This study reports part of a long-term program to help students improve scientific reasoning using higher-order cognitive tasks set in the discipline of cell biology. This skill was assessed using problems requiring the construction of valid conclusions drawn from authentic research data. We report here efforts to confirm the hypothesis that data interpretation is a complex, multifaceted exercise. Confirmation was obtained using a statistical treatment showing that various such problems rank students differently-each contains a unique set of cognitive challenges. Additional analyses of performance results have allowed us to demonstrate that individuals differ in their capacity to navigate five independent generic elements that constitute successful data interpretation: biological context, connection to course concepts, experimental protocols, data inference, and integration of isolated experimental observations into a coherent model. We offer these aspects of scientific thinking as a "data analysis skills inventory," along with usable sample problems that illustrate each element. Additionally, we show that this kind of reasoning is rigorous in that it is difficult for most novice students, who are unable to intuitively implement strategies for improving these skills. Instructors armed with knowledge of the specific challenges presented by different types of problems can provide specific helpful feedback during formative practice. The use of this instructional model is most likely to require changes in traditional classroom instruction.

Measurement Instrument for Scientific Teaching (MIST): A Tool to Measure the Frequencies of Research-Based Teaching Practices in Undergraduate Science Courses

The Scientific Teaching (ST) pedagogical framework provides various approaches for science instructors to teach in a way that more closely emulates how science is practiced by actively and inclusively engaging students in their own learning and by making instructional decisions based on student performance data. Fully understanding the impact of ST requires having mechanisms to quantify its implementation. While many useful instruments exist to document teaching practices, these instruments only partially align with the range of practices specified by ST, as described in a recently published taxonomy. Here, we describe the development, validation, and implementation of the Measurement Instrument for Scientific Teaching (MIST), a survey derived from the ST taxonomy and designed to gauge the frequencies of ST practices in undergraduate science courses. MIST showed acceptable validity and reliability based on results from 7767 students in 87 courses at nine institutions. We used factor analyses to identify eight subcategories of ST practices and used these categories to develop a short version of the instrument amenable to joint administration with other research instruments. We further discuss how MIST can be used by instructors, departments, researchers, and professional development programs to quantify and track changes in ST practices.

Assessment of Scientific Reasoning: the Effects of Task Context, Data, and Design on Student Reasoning in Control of Variables

Scientific reasoning is an important component under the cognitive strand of the 21st century skills and is highly emphasized in the new science education standards. This study focuses on the assessment of student reasoning in control of variables (COV), which is a core sub-skill of scientific reasoning. The main research question is to investigate the extent to which the existence of experimental data in questions impacts student reasoning and performance. This study also explores the effects of task contexts on student reasoning as well as students' abilities to distinguish between testability and causal influences of variables in COV experiments. Data were collected with students from both USA and China. Students received randomly one of two test versions, one with experimental data and one without. The results show that students from both populations (1) perform better when experimental data are not provided, (2) perform better in physics contexts than in real-life contexts, and (3) students have a tendency to equate non-influential variables to non-testable variables. In addition, based on the analysis of both quantitative and qualitative data, a possible progression of developmental levels of student reasoning in control of variables is proposed, which can be used to inform future development of assessment and instruction.

Scientific teaching: defining a taxonomy of observable practices

Over the past several decades, numerous reports have been published advocating for changes to undergraduate science education. These national calls inspired the formation of the National Academies Summer Institutes on Undergraduate Education in Biology (SI), a group of regional workshops to help faculty members learn and implement interactive teaching methods. The SI curriculum promotes a pedagogical framework called Scientific Teaching (ST), which aims to bring the vitality of modern research into the classroom by engaging students in the scientific discovery process and using student data to inform the ongoing development of teaching methods. With the spread of ST, the need emerges to systematically define its components in order to establish a common description for education researchers and practitioners. We describe the development of a taxonomy detailing ST's core elements and provide data from classroom observations and faculty surveys in support of its applicability within undergraduate science courses. The final taxonomy consists of 15 pedagogical goals and 37 supporting practices, specifying observable behaviors, artifacts, and features associated with ST. This taxonomy will support future educational efforts by providing a framework for researchers studying the processes and outcomes of ST-based course transformations as well as a concise guide for faculty members developing classes.

Cues Matter: Learning Assistants Influence Introductory Biology Student Interactions during Clicker-Question Discussions

The cues undergraduate biology instructors provide to students before discussions of clicker questions have previously been shown to influence student discussion. We further explored how student discussions were influenced by interactions with learning assistants (LAs, or peer coaches). We recorded and transcribed 140 clicker-question discussions in an introductory molecular biology course and coded them for features such as the use of reasoning and types of questions asked. Students who did not interact with LAs had discussions that were similar in most ways to students who did interact with LAs. When students interacted with LAs, the only significant changes in their discussions were the use of more questioning and more time spent in discussion. However, when individual LA-student interactions were examined within discussions, different LA prompts were found to generate specific student responses: question prompts promoted student use of reasoning, while students usually stopped their discussions when LAs explained reasons for answers. These results demonstrate that LA prompts directly influence student interactions during in-class discussions. Because clicker discussions can encourage student articulation of reasoning, instructors and LAs should focus on how to effectively implement questioning techniques rather than providing explanations.

Modeling for Fidelity: virtual mentorship by scientists fosters teacher self-efficacy and promotes implementation of novel high school biomedical curricula

This small-scale comparison case study evaluates the impact of an innovative approach to teacher professional development designed to promote implementation of a novel cutting edge high school neurological disorders curriculum. 'Modeling for Fidelity' (MFF) centers on an extended mentor relationship between teachers and biomedical scientists carried out in a virtual format in conjunction with extensive online educative materials. Four teachers from different diverse high schools in Massachusetts and Ohio who experienced MFF contextualized to a 6-week Neurological Disorders curriculum with the same science mentor were compared to a teacher who had experienced an intensive in-person professional development contextualized to the same curriculum with the same mentor. Fidelity of implementation was measured directly using an established metric and indirectly via student performance. The results show that teachers valued MFF, particularly the mentor relationship and were able to use it effectively to ensure critical components of the learning objectives were preserved. Moreover their students performed equivalently to those whose teacher had experienced intensive in-person professional development. Participants in all school settings demonstrated large (Cohen's d>2.0) and significant (p<0.0001 per-post) changes in conceptual knowledge as well as self-efficacy towards learning about neurological disorders (Cohen's d>1.5, p<0.0001 pre-post). The data demonstrates that the virtual mentorship format in conjunction with extensive online educative materials is an effective method of developing extended interactions between biomedical scientists and teachers that are scalable and not geographically constrained, facilitating teacher implementation of novel cutting-edge curricula.

The Great Diseases Project: a partnership between Tufts Medical School and the Boston public schools

Medical schools, although the gatekeepers of much biomedical education and research, rarely engage formally with K-12 educators to influence curriculum content or professional development. This segregation of content experts from teachers creates a knowledge gap that limits inclusion of current biomedical science into high school curricula, affecting both public health literacy and the biomedical pipeline. The authors describe how, in 2009, scientists from Tufts Medical School and Boston public school teachers established a partnership of formal scholarly dialogue to create 11th- to 12th-grade high school curricula about critical health-related concepts, with the goal of increasing scientific literacy and influencing health-related decisions. The curricula are based on the great diseases (infectious diseases, neurological disorders, metabolic disease, and cancer). Unlike most health science curricular interventions that provide circumscribed activities, the curricula are comprehensive, each filling one full term of in-class learning and providing extensive real-time support for the teacher. In this article, the authors describe how they developed and implemented the infectious disease curriculum, and its impacts. The high school teachers and students showed robust gains in content knowledge and critical thinking skills, whereas the Tufts scientists increased their pedagogical knowledge and appreciation for health-related science communication. The results show how formal interactions between medical schools and K-12 educators can be mutually beneficial.

Using the game of Mastermind to teach, practice, and discuss scientific reasoning skills

The code-breaking game Mastermind, which can be played in minutes at no cost, creates opportunities for students to discuss scientific reasoning, hypothesis-testing, effective experimental design, and sound interpretation of results.

Towards improving the integration of undergraduate biology and mathematics education

Arguments have recently asserted the need for change in undergraduate biology education, particularly with regard to the role of mathematics. The crux of these protests is that rapidly developing technology is expanding the types of measurements and subsequent data available to biologists. Thus future generations of biologists will require a set of quantitative and analytic skills that will allow them to handle these types of data in order to tackle relevant questions of interest. In this spirit, we describe here strategies (or lessons learned) for undergraduate educators with regard to better preparing undergraduate biology majors for the new types of challenges that lay ahead. The topics covered here span a broad range, from classroom approaches to the administrative level (e.g., fostering inter-departmental communication, student advising) and beyond. A key theme here is the need for an attitude shift with regard to mathematics education by both students and faculty alike. Such a shift will facilitate the development and implementation of new teaching strategies with regard to improving integration of mathematics and biology pedagogy.

Towards improving the integration of undergraduate biology and mathematics education

Arguments have recently asserted the need for change in undergraduate biology education, particularly with regard to the role of mathematics. The crux of these protests is that rapidly developing technology is expanding the types of measurements and subsequent data available to biologists. Thus future generations of biologists will require a set of quantitative and analytic skills that will allow them to handle these types of data in order to tackle relevant questions of interest. In this spirit, we describe here strategies (or lessons learned) for undergraduate educators with regard to better preparing undergraduate biology majors for the new types of challenges that lay ahead. The topics covered here span a broad range, from classroom approaches to the administrative level (e.g., fostering inter-departmental communication, student advising) and beyond. A key theme here is the need for an attitude shift with regard to mathematics education by both students and faculty alike. Such a shift will facilitate the development and implementation of new teaching strategies with regard to improving integration of mathematics and biology pedagogy.

Teaching the process of science: faculty perceptions and an effective methodology

Most scientific endeavors require science process skills such as data interpretation, problem solving, experimental design, scientific writing, oral communication, collaborative work, and critical analysis of primary literature. These are the fundamental skills upon which the conceptual framework of scientific expertise is built. Unfortunately, most college science departments lack a formalized curriculum for teaching undergraduates science process skills. However, evidence strongly suggests that explicitly teaching undergraduates skills early in their education may enhance their understanding of science content. Our research reveals that faculty overwhelming support teaching undergraduates science process skills but typically do not spend enough time teaching skills due to the perceived need to cover content. To encourage faculty to address this issue, we provide our pedagogical philosophies, methods, and materials for teaching science process skills to freshman pursuing life science majors. We build upon previous work, showing student learning gains in both reading primary literature and scientific writing, and share student perspectives about a course where teaching the process of science, not content, was the focus. We recommend a wider implementation of courses that teach undergraduates science process skills early in their studies with the goals of improving student success and retention in the sciences and enhancing general science literacy.