The Computer Will See You Now: Overcoming Barriers to Adoption of Computer-Assisted History Taking (CAHT) in Primary Care

Completion Rate and Satisfaction With Online Computer-Assisted History Taking Questionnaires in Orthopedics: Multicenter Implementation Report

Background

Collecting the medical history during a first outpatient consultation plays an important role in making a diagnosis. However, it is a time-consuming process, and time is scarce in today's health care environment. The computer-assisted history taking (CAHT) systems allow patients to share their medical history electronically before their visit. Although multiple advantages of CAHT have been demonstrated, adoption in everyday medical practice remains low, which has been attributed to various barriers.

Objective

This study aimed to implement a CAHT questionnaire for orthopedic patients in preparation for their first outpatient consultation and analyze its completion rate and added value.

Methods

A multicenter implementation study was conducted in which all patients who were referred to the orthopedic department were invited to self-complete the CAHT questionnaire. The primary outcome of the study is the completion rate of the questionnaire. Secondary outcomes included patient and physician satisfaction. These were assessed via surveys and semistructured interviews.

Unlabelled

In total, 5321 patients were invited, and 4932 (92.7%) fully completed the CAHT questionnaire between April 2022 and July 2022. On average, participants (n=224) rated the easiness of completing the questionnaire at 8.0 (SD 1.9; 0-10 scale) and the satisfaction of the consult at 8.0 (SD 1.7; 0-10 scale). Satisfaction with the outpatient consultation was higher in cases where the given answers were used by the orthopedic surgeon during this consultation (median 8.3, IQR 8.0-9.1 vs median 8.0, IQR 7.0-8.5; P<.001). Physicians (n=15) scored the average added value as 7.8 (SD 1.7; 0-10 scale) and unanimously recognized increased efficiency, better patient engagement, and better medical record completeness. Implementing the patient's answers into the electronic health record was deemed necessary.

Conclusions

In this study, we have shown that previously recognized barriers to implementing and adapting CAHT can now be effectively overcome. We demonstrated that almost all patients completed the CAHT questionnaire. This results in reported improvements in both the efficiency and personalization of outpatient consultations. Given the pressing need for personalized health care delivery in today's time-constrained medical environment, we recommend implementing CAHT systems in routine medical practice.

Person-based design and evaluation of MIA, a digital medical interview assistant for radiology

Introduction

Radiologists frequently lack direct patient contact due to time constraints. Digital medical interview assistants aim to facilitate the collection of health information. In this paper, we propose leveraging conversational agents to realize a medical interview assistant to facilitate medical history taking, while at the same time offering patients the opportunity to ask questions on the examination.

Methods

MIA, the digital medical interview assistant, was developed using a person-based design approach, involving patient opinions and expert knowledge during the design and development with a specific use case in collecting information before a mammography examination. MIA consists of two modules: the interview module and the question answering module (Q&A). To ensure interoperability with clinical information systems, we use HL7 FHIR to store and exchange the results collected by MIA during the patient interaction. The system was evaluated according to an existing evaluation framework that covers a broad range of aspects related to the technical quality of a conversational agent including usability, but also accessibility and security.

Results

Thirty-six patients recruited from two Swiss hospitals (Lindenhof group and Inselspital, Bern) and two patient organizations conducted the usability test. MIA was favorably received by the participants, who particularly noted the clarity of communication. However, there is room for improvement in the perceived quality of the conversation, the information provided, and the protection of privacy. The Q&A module achieved a precision of 0.51, a recall of 0.87 and an F-Score of 0.64 based on 114 questions asked by the participants. Security and accessibility also require improvements.

Conclusion

The applied person-based process described in this paper can provide best practices for future development of medical interview assistants. The application of a standardized evaluation framework helped in saving time and ensures comparability of results.

Primary care physicians' perceptions of artificial intelligence systems in the care of adolescents' mental health

BACKGROUND

Given that mental health problems in adolescence may have lifelong impacts, the role of primary care physicians (PCPs) in identifying and managing these issues is important. Artificial Intelligence (AI) may offer solutions to the current challenges involved in mental health care. We therefore explored PCPs' challenges in addressing adolescents' mental health, along with their attitudes towards using AI to assist them in their tasks.

METHODS

We used purposeful sampling to recruit PCPs for a virtual Focus Group (FG). The virtual FG lasted 75 minutes and was moderated by two facilitators. A life transcription was produced by an online meeting software. Transcribed data was cleaned, followed by a priori and inductive coding and thematic analysis.

RESULTS

We reached out to 35 potential participants via email. Seven agreed to participate, and ultimately four took part in the FG. PCPs perceived that AI systems have the potential to be cost-effective, credible, and useful in collecting large amounts of patients' data, and relatively credible. They envisioned AI assisting with tasks such as diagnoses and establishing treatment plans. However, they feared that reliance on AI might result in a loss of clinical competency. PCPs wanted AI systems to be user-friendly, and they were willing to assist in achieving this goal if it was within their scope of practice and they were compensated for their contribution. They stressed a need for regulatory bodies to deal with medicolegal and ethical aspects of AI and clear guidelines to reduce or eliminate the potential of patient harm.

CONCLUSION

This study provides the groundwork for assessing PCPs' perceptions of AI systems' features and characteristics, potential applications, possible negative aspects, and requirements for using them. A future study of adolescents' perspectives on integrating AI into mental healthcare might contribute a fuller understanding of the potential of AI for this population.

Evaluation and significance of a digital assistant for patient history-taking and physical examination in telemedicine

Introduction

Assisted history-taking systems can be used in provider-to-provider teleconsultations to task-shift the collection of evidence-based medical history and physical exam information to a frontline health worker. We developed such a task-shifting digital assistant, called 'Ayu', for nurses in rural India to collect clinical information from a patient and share it with a remote doctor to arrive at an accurate diagnosis and triage decision.

Materials & Methods

We evaluated the ability of the task-shifting digital assistant to collect a comprehensive patient history by using 190 standardized patient case studies and evaluating the information recall of the assistant by a skilled clinician. Following this, we tested the ability of nurses to use the system by training and evaluating the system's accuracy when used by 19 nurses in rural Gujarat, India. We also measured the diagnostic and triage accuracy based on the generated history note. Finally, we evaluated the system's acceptability by using the Technology Acceptance Model framework.

Results

Ayu could capture 65% of patient history information and 42% of physical exam information from patient case studies. When used by nurses, the mean accuracy of the generated clinical note was 7.71 ± 2.42. Using the information collected by a nurse using Ayu, a primary care physician could arrive at the correct diagnosis in 74% of cases, and correct triage decision in 88% of cases. Overall, we saw a high acceptability from nurses to use the system.

Conclusions

Ayu can capture an acceptable proportion of clinical information and can aid in collecting an evidence-based medical history by task-shifting some of the early investigational steps. Further development of Ayu to increase its information retrieval ability and ease of use by health workers is needed.

Development and benchmarking of a Korean audio speech recognition model for Clinician-Patient conversations in radiation oncology clinics

BACKGROUND

The purpose of this study is to develop an audio speech recognition (ASR) deep learning model for transcribing clinician-patient conversations in radiation oncology clinics.

METHODS

We finetuned the pre-trained English QuartzNet 15x5 model for the Korean language using a publicly available dataset of simulated situations between clinicians and patients. Subsequently, real conversations between a radiation oncologist and 115 patients in actual clinics were then prospectively collected, transcribed, and divided into training (30.26 h) and testing (0.79 h) sets. These datasets were used to develop the ASR model for clinics, which was benchmarked against other ASR models, including the 'Whisper large,' the 'Riva Citrinet-1024 Korean model,' and the 'Riva Conformer Korean model.'

RESULTS

The pre-trained English ASR model was successfully fine-tuned and converted to recognize the Korean language, resulting in a character error rate (CER) of 0.17. However, we found that this performance was not sustained on the real conversation dataset. To address this, we further fine-tuned the model, resulting in an improved CER of 0.26. Other developed ASR models, including 'Whisper large,' the 'Riva Citrinet-1024 Korean model,' and the 'Riva Conformer Korean model.', showed a CER of 0.31, 0.28, and 0.25, respectively. On the general Korean conversation dataset, 'zeroth-korean,' our model showed a CER of 0.44, while the 'Whisper large,' the 'Riva Citrinet-1024 Korean model,' and the 'Riva Conformer Korean model' resulted in CERs of 0.26, 0.98, and 0.99, respectively.

CONCLUSION

In conclusion, we developed a Korean ASR model to transcribe real conversations between a radiation oncologist and patients. The performance of the model was deemed acceptable for both specific and general purposes, compared to other models. We anticipate that this model will reduce the time required for clinicians to document the patient's chief complaints or side effects.

SWOT Analysis and Expert Assessment of the Effectiveness of the Introduction of Healthcare Information Systems in Polyclinics in Aktobe, Kazakhstan

OBJECTIVES

The purpose of this study was to assess the organizational effectiveness of the introduction of a healthcare information system (electronic medical records and databases) in healthcare in Kazakhstan.

METHODS

The authors used a combination of 2.

METHODS

expert assessment and strengths, weaknesses, opportunities, and threats (SWOT) analysis. SWOT analysis is a necessary element of research, constituting a mandatory preliminary stage both when drawing up strategic plans and for taking corrective measures in the future. The expert survey was conducted using 2 questionnaires.

RESULTS

The study involved 40 experts drawn from specialists in primary healthcare in Aktobe: 15 representatives of administrative and managerial personnel (chief doctors and their deputies, heads of medical statistics offices, organizational and methodological offices, and internal audit services) and 25 general practitioners.

CONCLUSIONS

The following functional indicators of the medical and organizational effectiveness of the introduction of information systems in polyclinics were highlighted: first, improvement of administrative control, followed in descending order by registration and movement of medical documentation, statistical reporting and process results, and the cost of employees' working time. There has been no reduction in financial costs, namely in terms of the costs of copying, delivery of information in paper form, technical equipment, and paper.

Accuracy of Cloud-Based Speech Recognition Open Application Programming Interface for Medical Terms of Korean

BACKGROUND

There are limited data on the accuracy of cloud-based speech recognition (SR) open application programming interfaces (APIs) for medical terminology. This study aimed to evaluate the medical term recognition accuracy of current available cloud-based SR open APIs in Korean.

METHODS

We analyzed the SR accuracy of currently available cloud-based SR open APIs using real doctor-patient conversation recordings collected from an outpatient clinic at a large tertiary medical center in Korea. For each original and SR transcription, we analyzed the accuracy rate of each cloud-based SR open API (i.e., the number of medical terms in the SR transcription per number of medical terms in the original transcription).

RESULTS

A total of 112 doctor-patient conversation recordings were converted with three cloud-based SR open APIs (Naver Clova SR from Naver Corporation; Google Speech-to-Text from Alphabet Inc.; and Amazon Transcribe from Amazon), and each transcription was compared. Naver Clova SR (75.1%) showed the highest accuracy with the recognition of medical terms compared to the other open APIs (Google Speech-to-Text, 50.9%, P < 0.001; Amazon Transcribe, 57.9%, P < 0.001), and Amazon Transcribe demonstrated higher recognition accuracy compared to Google Speech-to-Text (P < 0.001). In the sub-analysis, Naver Clova SR showed the highest accuracy in all areas according to word classes, but the accuracy of words longer than five characters showed no statistical differences (Naver Clova SR, 52.6%; Google Speech-to-Text, 56.3%; Amazon Transcribe, 36.6%).

CONCLUSION

Among three current cloud-based SR open APIs, Naver Clova SR which manufactured by Korean company showed highest accuracy of medical terms in Korean, compared to Google Speech-to-Text and Amazon Transcribe. Although limitations are existing in the recognition of medical terminology, there is a lot of rooms for improvement of this promising technology by combining strengths of each SR engines.