The structure of observed learning outcome (SOLO) taxonomy: a model to promote dental students' learning

Using ChatGPT for Writing Articles for Patients' Education for Dermatological Diseases: A Pilot Study

Background

Patients' education is a vital strategy for understanding a disease by patients and proper management of the condition. Physicians and academicians frequently make customized education materials for their patients. An artificial intelligence (AI)-based writer can help them write an article. Chat Generative Pre-Trained Transformer (ChatGPT) is a conversational language model developed by OpenAI (openai.com). The model can generate human-like responses.

Objective

We aimed to evaluate the generated text from ChatGPT for its suitability in patients' education.

Materials and Methods

We asked the ChatGPT to list common dermatological diseases. It provided a list of 14 diseases. We used the disease names to converse with the application with disease-specific input (e.g., write a patient education guide on acne). The text was copied for checking the number of words, readability, and text similarity by software. The text's accuracy was checked by a dermatologist following the structure of observed learning outcomes (SOLO) taxonomy. For the readability ease score, we compared the observed value with a score of 30. For the similarity index, we compared the observed value with 15% and tested it with a one-sample t-test.

Results

The ChatGPT generated a paragraph of text of 377.43 ± 60.85 words for a patient education guide on skin diseases. The average text reading ease score was 46.94 ± 8.23 (P < 0.0001), and it indicates that this level of text can easily be understood by a high-school student to a newly joined college student. The text similarity index was higher (27.07 ± 11.46%, P = 0.002) than the expected limit of 15%. The text had a "relational" level of accuracy according to the SOLO taxonomy.

Conclusion

In its current form, ChatGPT can generate a paragraph of text for patients' educational purposes that can be easily understood. However, the similarity index is high. Hence, doctors should be cautious when using the text generated by ChatGPT and must check for text similarity before using it.

Applicability of ChatGPT in Assisting to Solve Higher Order Problems in Pathology

Background Artificial intelligence (AI) is evolving for healthcare services. Higher cognitive thinking in AI refers to the ability of the system to perform advanced cognitive processes, such as problem-solving, decision-making, reasoning, and perception. This type of thinking goes beyond simple data processing and involves the ability to understand and manipulate abstract concepts, interpret, and use information in a contextually relevant way, and generate new insights based on past experiences and accumulated knowledge. Natural language processing models like ChatGPT is a conversational program that can interact with humans to provide answers to queries. Objective We aimed to ascertain the capability of ChatGPT in solving higher-order reasoning in the subject of pathology. Methods This cross-sectional study was conducted on the internet using an AI-based chat program that provides free service for research purposes. The current version of ChatGPT (January 30 version) was used to converse with a total of 100 higher-order reasoning queries. These questions were randomly selected from the question bank of the institution and categorized according to different systems. The responses to each question were collected and stored for further analysis. The responses were evaluated by three expert pathologists on a zero to five scale and categorized into the structure of the observed learning outcome (SOLO) taxonomy categories. The score was compared by a one-sample median test with hypothetical values to find its accuracy. Result A total of 100 higher-order reasoning questions were solved by the program in an average of 45.31±7.14 seconds for an answer. The overall median score was 4.08 (Q1-Q3: 4-4.33) which was below the hypothetical maximum value of five (one-test median test p <0.0001) and similar to four (one-test median test p = 0.14). The majority (86%) of the responses were in the "relational" category in the SOLO taxonomy. There was no difference in the scores of the responses for questions asked from various organ systems in the subject of Pathology (Kruskal Wallis p = 0.55). The scores rated by three pathologists had an excellent level of inter-rater reliability (ICC = 0.975 [95% CI: 0.965-0.983]; F = 40.26; p < 0.0001). Conclusion The capability of ChatGPT to solve higher-order reasoning questions in pathology had a relational level of accuracy. Hence, the text output had connections among its parts to provide a meaningful response. The answers from the program can score approximately 80%. Hence, academicians or students can get help from the program for solving reasoning-type questions also. As the program is evolving, further studies are needed to find its accuracy level in any further versions.

Personalization of the Learning Path within an Augmented Reality Spatial Ability Training Application Based on Fuzzy Weights

Adaptive systems and Augmented Reality are among the most promising technologies in teaching and learning processes, as they can be an effective tool for training engineering students' spatial skills. Prior work has investigated the integration of AR technology in engineering education, and more specifically, in spatial ability training. However, the modeling of user knowledge in order to personalize the training has been neither sufficiently explored nor exploited in this task. There is a lot of space for research in this area. In this work, we introduce a novel personalization of the learning path within an AR spatial ability training application. The aim of the research is the integration of Augmented Reality, specifically in engineering evaluation and fuzzy logic technology. During one academic semester, three engineering undergraduate courses related to the domain of spatial skills were supported by a developed adaptive training system named PARSAT. Using the technology of fuzzy weights in a rule-based decision-making module and the learning theory of the Structure of the Observed Learning Outcomes for the design of the learning material, PARSAT offers adaptive learning activities for the students' cognitive skills. Students' data were gathered at the end of the academic semester, and a thorough analysis was delivered. The findings demonstrated that the proposed training method outperformed the traditional method that lacked adaptability, in terms of domain expertise and learning theories, considerably enhancing student learning outcomes.

Rethinking Assessment: The Future of Examinations in Higher Education

The global higher education landscape is significantly impacted as a result of the COVID-19 pandemic and the majority of the universities now follow an online or hybrid mode of delivery. This presents substantial challenges for universities, particularly to conduct examinations, as traditionally most exams were conducted physically on campus. During the first wave of the pandemic, many universities had no option and were forced to move online in a very short period of time, causing universities also to conduct exams online without transforming pedagogy and the structure/s of closed-book exams. Inevitably, in non-proctored and unregulated examinations, this allowed room for students to collaborate and share material during online exams without being noticed by an invigilator as in the case of physical exams. Online exams, also leave room for students to find information online which made preventing plagiarism a significant challenge. This paper investigates the practices used in both closed-book and open-book exams and identifies the challenges associated with the transition to online exams. It also identifies potential ways forward for future online exams, while minimizing opportunities for students to collaborate, plagiarise and use online material. The findings of this study reveal that online examinations affect teachers and students differently: while teachers have mixed feelings about online exams, students are anxious about their grades and the technical hassle they experience in online exams. While viva has emerged as a popular form of alternative assessment, students still feel the need of returning to physical exams. None of the teachers who participated in this study discussed a psychosocial approach to education and exams in this pandemic. We conclude this study on the note that there is a need for the collaboration of social scientists, psychologists, psychosocial specialists, educationists, and humanities scholars/humanists for better educational policy and pedagogical practices during the pandemic.