Appraising the performance of ChatGPT in psychiatry using 100 clinical case vignettes

BACKGROUND

ChatGPT has emerged as the most advanced and rapidly developing large language chatbot system. With its immense potential ranging from answering a simple query to cracking highly competitive medical exams, ChatGPT continues to impress the scientists and researchers worldwide giving room for more discussions regarding its utility in various fields. One such field of attention is Psychiatry. With suboptimal diagnosis and treatment, assuring mental health and well-being is a challenge in many countries, particularly developing nations. To this regard, we conducted an evaluation to assess the performance of ChatGPT 3.5 in Psychiatry using clinical cases to provide evidence-based information regarding the implication of ChatGPT 3.5 in enhancing mental health and well-being.

METHODS

ChatGPT 3.5 was used in this experimental study to initiate the conversations and collect responses to clinical vignettes in Psychiatry. Using 100 clinical case vignettes, the replies were assessed by expert faculties from the Department of Psychiatry. There were 100 different psychiatric illnesses represented in the cases. We recorded and assessed the initial ChatGPT 3.5 responses. The evaluation was conducted using the objective of questions that were put forth at the conclusion of the case, and the aim of the questions was divided into 10 categories. The grading was completed by taking the mean value of the scores provided by the evaluators. Graphs and tables were used to represent the grades.

RESULTS

The evaluation report suggests that ChatGPT 3.5 fared extremely well in Psychiatry by receiving "Grade A" ratings in 61 out of 100 cases, "Grade B" ratings in 31, and "Grade C" ratings in 8. Majority of the queries were concerned with the management strategies, which were followed by diagnosis, differential diagnosis, assessment, investigation, counselling, clinical reasoning, ethical reasoning, prognosis, and request acceptance. ChatGPT 3.5 performed extremely well, especially in generating management strategies followed by diagnoses for different psychiatric conditions. There were no responses which were graded "D" indicating that there were no errors in the diagnosis or response for clinical care. Only a few discrepancies and additional details were missed in a few responses that received a "Grade C" CONCLUSION: It is evident from our study that ChatGPT 3.5 has appreciable knowledge and interpretation skills in Psychiatry. Thus, ChatGPT 3.5 undoubtedly has the potential to transform the field of Medicine and we emphasize its utility in Psychiatry through the finding of our study. However, for any AI model to be successful, assuring the reliability, validation of information, proper guidelines and implementation framework are necessary.

Mental health experiences and coping strategies of BAME care workers who worked in nursing and residential care homes during the COVID-19 pandemic in Luton, England

BACKGROUND

The COVID-19 pandemic intensified the risk factors for poor mental health among care workers in the UK. However, there is inadequate evidence on the mental health impact of COVID-19 on Black, Asian, and minority ethnic (BAME) care workers in particular. This study seeks to explore mental health experiences and coping strategies of BAME care workers who worked in nursing and residential care homes during the COVID-19 pandemic.

METHOD

This is a qualitative study conducted between February and May, 2021 in Luton, England. A sample of n = 15 care workers from BAME background working in nursing and residential care homes were recruited purposively using the snowball sampling technique. In-depth interviews were conducted around topics such as views on COVID-19, the impact of the COVID-19 pandemic on mental health and coping during the COVID-19 pandemic. Data from the interviews was analysed using the Framework Analysis Approach.

RESULTS

The COVID-19 pandemic had a negative impact on the participants' mental health as they experienced stress, depression, anxiety, trauma and paranoia. The majority of the participants explained that they managed their mental health by belief in God and religious practices, by keeping themselves busy doing activities they were passionate about, following government guidelines on the prevention of COVID-19, seeing the service users happy and some participants managed through support that was offered by the government. However, some participants did not have any support for their mental health.

CONCLUSION

Issues such as increased workload associated with COVID-19 restrictions engendered mental health problems among BAME care workers, however, the workload only further increased during the pandemic, but the health and social care sector was already affected by heavy workload due to staff shortages and this needs to be addressed through increasing their wages to encourage more people to work in the health and social care sector. In addition, some BAME care workers never received any support for their mental health during the pandemic. Hence, integrating mental health services such as counselling, supportive psychotherapy and recreational therapies in care homes could help to support the mental health of care workers in the COVID-19 era.