Using quality improvement to implement the CNS/AAN quality measure on rescue medication for seizures

Implications of Large Language Models for Quality and Efficiency of Neurologic Care: Emerging Issues in Neurology

Large language models (LLMs) are advanced artificial intelligence (AI) systems that excel in recognizing and generating human-like language, possibly serving as valuable tools for neurology-related information tasks. Although LLMs have shown remarkable potential in various areas, their performance in the dynamic environment of daily clinical practice remains uncertain. This article outlines multiple limitations and challenges of using LLMs in clinical settings that need to be addressed, including limited clinical reasoning, variable reliability and accuracy, reproducibility bias, self-serving bias, sponsorship bias, and potential for exacerbating health care disparities. These challenges are further compounded by practical business considerations and infrastructure requirements, including associated costs. To overcome these hurdles and harness the potential of LLMs effectively, this article includes considerations for health care organizations, researchers, and neurologists contemplating the use of LLMs in clinical practice. It is essential for health care organizations to cultivate a culture that welcomes AI solutions and aligns them seamlessly with health care operations. Clear objectives and business plans should guide the selection of AI solutions, ensuring they meet organizational needs and budget considerations. Engaging both clinical and nonclinical stakeholders can help secure necessary resources, foster trust, and ensure the long-term sustainability of AI implementations. Testing, validation, training, and ongoing monitoring are pivotal for successful integration. For neurologists, safeguarding patient data privacy is paramount. Seeking guidance from institutional information technology resources for informed, compliant decisions, and remaining vigilant against biases in LLM outputs are essential practices in responsible and unbiased utilization of AI tools. In research, obtaining institutional review board approval is crucial when dealing with patient data, even if deidentified, to ensure ethical use. Compliance with established guidelines like SPIRIT-AI, MI-CLAIM, and CONSORT-AI is necessary to maintain consistency and mitigate biases in AI research. In summary, the integration of LLMs into clinical neurology offers immense promise while presenting formidable challenges. Awareness of these considerations is vital for harnessing the potential of AI in neurologic care effectively and enhancing patient care quality and safety. The article serves as a guide for health care organizations, researchers, and neurologists navigating this transformative landscape.

Implementing American Academy of Neurology Quality Measures in Antigua Using Quality Improvement Methodology

Background and Objectives

The American Academy of Neurology has developed quality measures related to various neurologic disorders. A gap exists in the implementation of these measures in the different health care systems. To date, there has been no electronic health care record nor implementation of quality measures in Antigua. Therefore, we aimed to increase the percent of patients who have epilepsy quality measures documented using standardized common data elements in the outpatient neurology clinic at Sir Lester Bird Medical Center from 0% to 80% per week by June 1, 2022 and sustain for 6 months.

Methods

We used the Institute for Health care Improvement Model for Improvement methodology. A data use agreement was implemented. Data were displayed using statistical process control charts and the American Society for Quality criteria to determine statistical significance and centerline shifts.

Results

Current and future state process maps were developed to determine areas of opportunity for interventions. Interventions were developed following a "Plan-Do-Study-Act cycle." One intervention was the creation of a RedCap survey and database to be used by health care providers during clinical patient encounters. Because of multiple interventions, we achieved a 100% utilization of the survey for clinical care.

Discussion

Quality improvement (QI) methodology can be used for implementation of quality measures in various settings to improve patient care outcomes without use of significant resources. Implementation of quality measures can increase efficiency in clinical delivery. Similar QI methodology could be implemented in other resource-limited countries of the Caribbean and globally.

A replicable, open-source, data integration method to support national practice-based research & quality improvement systems

OBJECTIVES

The Epilepsy Learning Healthcare System (ELHS) was created in 2018 to address measurable improvements in outcomes for people with epilepsy. However, fragmentation of data systems has been a major barrier for reporting and participation. In this study, we aimed to test the feasibility of an open-source Data Integration (DI) method that connects real-life clinical data to national research and quality improvement (QI) systems.

METHODS

The ELHS case report forms were programmed as EPIC SmartPhrases at Mass General Brigham (MGB) in December 2018 and subsequently as EPIC SmartForms in June 2021 to collect actionable, standardized, structured epilepsy data in the electronic health record (EHR) for subsequent pull into the external national registry of the ELHS. Following the QI methodology in the Chronic Care Model, 39 providers, epileptologists and neurologists, incorporated the ELHS SmartPhrase into their clinical workflow, focusing on collecting diagnosis of epilepsy, seizure type according to the International League Against Epilepsy, seizure frequency, date of last seizure, medication adherence and side effects. The collected data was stored in the Enterprise Data Warehouse (EDW) without integration with external systems. We developed and validated a DI method that extracted the data from EDW using structured query language and later preprocessed using text mining. We used the ELHS data dictionary to match fields in the preprocessed notes to obtain the final structured dataset with seizure control information. For illustration, we described the data curated from the care period of 12/2018-12/2021.

RESULTS

The cohort comprised a total of 1806 patients with a mean age of 43 years old (SD: 17.0), where 57% were female, 80% were white, and 84% were non-Hispanic/Latino. Using our DI method, we automated the data mining, preprocessing, and exporting of the structured dataset into a local database, to be weekly accessible to clinicians and quality improvers. During the period of SmartPhrase implementation, there were 5168 clinic visits logged by providers documenting each patient's seizure type and frequency. During this period, providers documented 59% patients having focal seizures, 35% having generalized seizures and 6% patients having another type. Of the cohort, 45% patients had private insurance. The resulting structured dataset was bulk uploaded via web interface into the external national registry of the ELHS.

CONCLUSIONS

Structured data can be feasibly extracted from text notes of epilepsy patients for weekly reporting to a national learning healthcare system.

Predicting Patient No-Shows in an Academic Pediatric Neurology Clinic

Background: No-shows can negatively affect patient care. Efforts to predict high-risk patients are needed. Previously, our epilepsy clinic identified patients with 2 or more no-shows or late cancelations in the past 18 months as being at high risk for no-shows. Our objective was to develop a model to accurately predict the risk of no-shows among patients with epilepsy seen at our neurology clinic. Methods: Using electronic health record data, we developed a least absolute shrinkage and selection operator (LASSO)-regularized logistic regression model to predict no-shows and compared its performance with our neurology clinic's above-mentioned ad hoc rule. Results: The ad hoc rule identified 13% of patients seen at our neurology clinic as high-risk patients for no-shows and resulted in a positive predictive value of 38%. In comparison, our LASSO model resulted in a positive predictive value of 48%. Our LASSO model identified that lack of private insurance, inactive Epic MyChart, greater past no-show rates, fewer appointment changes before the appointment date, and follow-up appointments were more likely to result in no-shows. Conclusions: Our LASSO model outperformed the ad hoc rule used by our neurology clinic in predicting patients at high risk for no-shows. Social workers can use the no-show risk scores generated by our LASSO model to prioritize high-risk patients for targeted intervention to reduce no-shows at our neurology clinic.