Developing an artificial intelligence-based headache diagnostic model and its utility for non-specialists' diagnostic accuracy

The Ethical Stewardship of Artificial Intelligence in Chronic Pain and Headache: A Narrative Review

PURPOSE OF REVIEW

As artificial intelligence (AI) and machine learning (ML) are becoming more pervasive in medicine, understanding their ethical considerations for chronic pain and headache management is crucial for optimizing their safety.

RECENT FINDINGS

We reviewed thirty-eight editorial and original research articles published between 2018 and 2023, focusing on the application of AI and ML to chronic pain or headache. The core medical principles of beneficence, non-maleficence, autonomy, and justice constituted the evaluation framework. The AI applications addressed topics such as pain intensity prediction, diagnostic aides, risk assessment for medication misuse, empowering patients to self-manage their conditions, and optimizing access to care. Virtually all AI applications aligned both positively and negatively with specific medical ethics principles. This review highlights the potential of AI to enhance patient outcomes and physicians' experiences in managing chronic pain and headache. We emphasize the importance of carefully considering the advantages, disadvantages, and unintended consequences of utilizing AI tools in chronic pain and headache, and propose the four core principles of medical ethics as an evaluation framework.

[Digitalization in Migraine Therapy]

Digitalization is a commonly used keyword in medicine and also in the area of migraine and its therapy. However, digitalization should not be an end in itself, but should improve the treatment of patients and make the work of practitioners easier. This article summarizes the use of e-health applications for migraine medicine, today and in the future.

A large language model-based generative natural language processing framework fine-tuned on clinical notes accurately extracts headache frequency from electronic health records

OBJECTIVE

To develop a natural language processing (NLP) algorithm that can accurately extract headache frequency from free-text clinical notes.

BACKGROUND

Headache frequency, defined as the number of days with any headache in a month (or 4 weeks), remains a key parameter in the evaluation of treatment response to migraine preventive medications. However, due to the variations and inconsistencies in documentation by clinicians, significant challenges exist to accurately extract headache frequency from the electronic health record (EHR) by traditional NLP algorithms.

METHODS

This was a retrospective cross-sectional study with patients identified from two tertiary headache referral centers, Mayo Clinic Arizona and Mayo Clinic Rochester. All neurology consultation notes written by 15 specialized clinicians (11 headache specialists and 4 nurse practitioners) between 2012 and 2022 were extracted and 1915 notes were used for model fine-tuning (90%) and testing (10%). We employed four different NLP frameworks: (1) ClinicalBERT (Bidirectional Encoder Representations from Transformers) regression model, (2) Generative Pre-Trained Transformer-2 (GPT-2) Question Answering (QA) model zero-shot, (3) GPT-2 QA model few-shot training fine-tuned on clinical notes, and (4) GPT-2 generative model few-shot training fine-tuned on clinical notes to generate the answer by considering the context of included text.

RESULTS

The mean (standard deviation) headache frequency of our training and testing datasets were 13.4 (10.9) and 14.4 (11.2), respectively. The GPT-2 generative model was the best-performing model with an accuracy of 0.92 (0.91, 0.93, 95% confidence interval [CI]) and R2 score of 0.89 (0.87, 0.90, 95% CI), and all GPT-2-based models outperformed the ClinicalBERT model in terms of exact matching accuracy. Although the ClinicalBERT regression model had the lowest accuracy of 0.27 (0.26, 0.28), it demonstrated a high R2 score of 0.88 (0.85, 0.89), suggesting the ClinicalBERT model can reasonably predict the headache frequency within a range of ≤ ± 3 days, and the R2 score was higher than the GPT-2 QA zero-shot model or GPT-2 QA model few-shot training fine-tuned model.

CONCLUSION

We developed a robust information extraction model based on a state-of-the-art large language model, a GPT-2 generative model that can extract headache frequency from EHR free-text clinical notes with high accuracy and R2 score. It overcame several challenges related to different ways clinicians document headache frequency that were not easily achieved by traditional NLP models. We also showed that GPT-2-based frameworks outperformed ClinicalBERT in terms of accuracy in extracting headache frequency from clinical notes. To facilitate research in the field, we released the GPT-2 generative model and inference code with open-source license of community use in GitHub. Additional fine-tuning of the algorithm might be required when applied to different health-care systems for various clinical use cases.

Predicting interindividual response to theta burst stimulation in the lower limb motor cortex using machine learning

Using theta burst stimulation (TBS) to induce neural plasticity has played an important role in improving the treatment of neurological disorders. However, the variability of TBS-induced synaptic plasticity in the primary motor cortex prevents its clinical application. Thus, factors associated with this variability should be explored to enable the creation of a predictive model. Statistical approaches, such as regression analysis, have been used to predict the effects of TBS. Machine learning may potentially uncover previously unexplored predictive factors due to its increased capacity for capturing nonlinear changes. In this study, we used our prior dataset (Katagiri et al., 2020) to determine the factors that predict variability in TBS-induced synaptic plasticity in the lower limb motor cortex for both intermittent (iTBS) and continuous (cTBS) TBS using machine learning. Validation of the created model showed an area under the curve (AUC) of 0.85 and 0.69 and positive predictive values of 77.7 and 70.0% for iTBS and cTBS, respectively; the negative predictive value was 75.5% for both patterns. Additionally, the accuracy was 0.76 and 0.72, precision was 0.82 and 0.67, recall was 0.82 and 0.67, and F1 scores were 0.82 and 0.67 for iTBS and cTBS, respectively. The most important predictor of iTBS was the motor evoked potential amplitude, whereas it was the intracortical facilitation for cTBS. Our results provide additional insights into the prediction of the effects of TBS variability according to baseline neurophysiological factors.

The Clinical Relevance of Artificial Intelligence in Migraine

Migraine is a burdensome neurological disorder that still lacks clear and easily accessible diagnostic biomarkers. Furthermore, a straightforward pathway is hard to find for migraineurs' management, so the search for response predictors has become urgent. Nowadays, artificial intelligence (AI) has pervaded almost every aspect of our lives, and medicine has not been missed. Its applications are nearly limitless, and the ability to use machine learning approaches has given researchers a chance to give huge amounts of data new insights. When it comes to migraine, AI may play a fundamental role, helping clinicians and patients in many ways. For example, AI-based models can increase diagnostic accuracy, especially for non-headache specialists, and may help in correctly classifying the different groups of patients. Moreover, AI models analysing brain imaging studies reveal promising results in identifying disease biomarkers. Regarding migraine management, AI applications showed value in identifying outcome measures, the best treatment choices, and therapy response prediction. In the present review, the authors introduce the various and most recent clinical applications of AI regarding migraine.

Treatment Patterns for and Characteristics of Headache in Children and Adolescents Aged 6-17 Years in Japan: A Retrospective Cross-Sectional and Longitudinal Analysis of Health Insurance Claims Data

OBJECTIVE

To investigate the prescription patterns for patients aged 6-17 years with headaches in the REZULT database.

METHODS

We cross-sectionally investigated (Study 1) the pattern of prescription and the proportion of triptan overprescription (≥30 tablets/90 d of triptans) among patients diagnosed with headaches in 2020. Next, we longitudinally studied patients (Study 2) for more than two years from the initial headache diagnosis (July 2010 to April 2022). The number of prescribed tablets was counted every 90 days.

RESULTS

In Study 1, headache diagnoses were assigned to 62,568 of 543,628 (11.51%) patients, and 1524 of 62,568 (2.44%) patients received acute medication. Single nonsteroidal anti-inflammatory drugs and triptans were prescribed to 620/624 (99.36%) and 5/624 (0.80%) of patients aged 6-11 years, respectively, and 827/900 (91.89%) and 91/900 (10.11%) of patients aged 12-17 years, respectively. Triptan overprescription was observed in 11/96 (11.46%) patients, and 5/11 (45.45%) of those patients received prophylactic medication. In Study 2, 80,756/845,470 (9.55%) patients aged 6-17 years were diagnosed with headaches that persisted for at least two years. Over two years, 44/80,756 (0.05%) patients were overprescribed triptans, and 3408/80,756 (4.22%) patients were prescribed prophylaxis on at least one occasion.

CONCLUSIONS

Based on real-world data, the appropriate use of prophylactic treatment is still problematic. Overprescription of triptans was observed, although the number of patients was small.

Treatment patterns and characteristics of headache in patients in Japan: A retrospective cross-sectional and longitudinal analysis of health insurance claims data

BACKGROUND

The present study aimed to investigate prescription patterns for patients aged over 17 years with headaches in the REZULT database.

METHODS

We conducted a cross-sectional study (Study 1) of the proportion of over-prescription of acute medications (≥30 tablets/90 days for triptans, combination non-steroidal anti-inflammatory drugs (NSAIDs) and multiple types; ≥45 tablets/90 days for single NSAIDs) among patients with headache diagnosed in 2020. We longitudinally studied (Study 2) patients for >2 years from initial headache diagnosis (July 2010 to April 2022). The number of prescribed tablets was counted every 90 days.

RESULTS

In Study 1, headache was diagnosed in 200,055 of 3,638,125 (5.5%) patients: 13,651/200,055 (6.8%) received acute medication. Single NSAIDs were prescribed to 12,297/13,651 (90.1%) patients and triptans to 1710/13,651 (12.5%). Over-prescription was found in 2262/13,651 (16.6%) patients and 1200/13,651 (8.8%) patients received prophylactic medication. In Study 2, 408,183/6,840,618 (6.0%) patients were first diagnosed with headaches, which persisted for ≥2 years. Over time, the proportion of patients over-prescribed acute medications increased. Over 2 years, 37,617/408,183 (9.2%) patients were over-prescribed acute medications and 29,313/408,183 (7.2%) patients were prescribed prophylaxis at least once.

CONCLUSIONS

According to real-world data, prophylaxis remains poorly prescribed, and both acute and prophylactic treatment rates for headaches have increased over time.

New management strategies for primary headache disorders: Insights from P4 medicine

Primary headache disorder is the main cause of headache attacks, leading to significant disability and impaired quality of life. This disorder is increasingly recognized as a heterogeneous condition with a complex network of genetic, environmental, and lifestyle factors. However, the timely diagnosis and effective treatment of these headaches remain challenging. Precision medicine is a potential strategy based on P4 (predictive, preventive, personalized, and participatory) medicine that may bring new insights for headache care. Recent machine learning advances and widely available molecular biology and imaging data have increased the usefulness of this medical strategy. Precision medicine emphasizes classifying headaches according to their risk factors, clinical presentation, and therapy responsiveness to provide individualized headache management. Furthermore, early preventive strategies, mainly utilizing predictive tools, are critical in reducing headache attacks and improving the quality of life of individuals with headaches. The current review comprehensively discusses the potential application value of P4 medicine in headache management.

Developing an artificial intelligence-based diagnostic model of headaches from a dataset of clinic patients' records

OBJECTIVE

We developed an artificial intelligence (AI)-based headache diagnosis model using a large questionnaire database from a headache-specializing clinic.

BACKGROUND

Misdiagnosis of headache disorders is a serious issue and AI-based headache diagnosis models are scarce.

METHODS

We developed an AI-based headache diagnosis model and conducted internal validation based on a retrospective investigation of 6058 patients (4240 training dataset for model development and 1818 test dataset for internal validation) diagnosed by a headache specialist. The ground truth was the diagnosis by the headache specialist. The diagnostic performance of the AI model was evaluated.

RESULTS

The dataset included 4829/6058 (79.7%) patients with migraine, 834/6058 (13.8%) with tension-type headache, 78/6058 (1.3%) with trigeminal autonomic cephalalgias, 38/6058 (0.6%) with other primary headache disorders, and 279/6058 (4.6%) with other headaches. The mean (standard deviation) age was 34.7 (14.5) years, and 3986/6058 (65.8%) were female. The model's micro-average accuracy, sensitivity (recall), specificity, precision, and F-values for the test dataset were 93.7%, 84.2%, 84.2%, 96.1%, and 84.2%, respectively. The diagnostic performance for migraine was high, with a sensitivity of 88.8% and c-statistics of 0.89 (95% confidence interval 0.87-0.91).

CONCLUSIONS

Our AI model demonstrated high diagnostic performance for migraine. If secondary headaches can be ruled out, the model can be a powerful tool for diagnosing migraine; however, further data collection and external validation are required to strengthen the performance, ensure the generalizability in other outpatients, and demonstrate its utility in real-world settings.

Developing an Artificial Intelligence-Based Pediatric and Adolescent Migraine Diagnostic Model

Introduction Misdiagnosis of pediatric and adolescent migraine is a significant problem. The first artificial intelligence (AI)-based pediatric migraine diagnosis model was made utilizing a database of questionnaires obtained from a previous epidemiological study, the Itoigawa Benizuwaigani Study. Methods The AI-based headache diagnosis model was created based on the internal validation based on a retrospective investigation of 909 patients (636 training dataset for model development and 273 test dataset for internal validation) aged six to 17 years diagnosed based on the International Classification of Headache Disorders 3rd edition. The diagnostic performance of the AI model was evaluated. Results The dataset included 234/909 (25.7%) pediatric or adolescent patients with migraine. The mean age was 11.3 (standard deviation 3.17) years. The model's accuracy, sensitivity (recall), specificity, precision, and F-values for the test dataset were 94.5%, 88.7%, 96.5%, 90.0%, and 89.4%, respectively. Conclusions The AI model exhibited high diagnostic performance for pediatric and adolescent migraine. It holds great potential as a powerful tool for diagnosing these conditions, especially when secondary headaches are ruled out. Nonetheless, further data collection and external validation are necessary to enhance the model's performance and ensure its applicability in real-world settings.