A Comparative Study of Large Language Models, Human Experts, and Expert-Edited Large Language Models to Neuro-Ophthalmology Questions

BACKGROUND

While large language models (LLMs) are increasingly used in medicine, their effectiveness compared with human experts remains unclear. This study evaluates the quality and empathy of Expert + AI, human experts, and LLM responses in neuro-ophthalmology.

METHODS

This randomized, masked, multicenter cross-sectional study was conducted from June to July 2023. We randomly assigned 21 neuro-ophthalmology questions to 13 experts. Each expert provided an answer and then edited a ChatGPT-4-generated response, timing both tasks. In addition, 5 LLMs (ChatGPT-3.5, ChatGPT-4, Claude 2, Bing, Bard) generated responses. Anonymized and randomized responses from Expert + AI, human experts, and LLMs were evaluated by the remaining 12 experts. The main outcome was the mean score for quality and empathy, rated on a 1-5 scale.

RESULTS

Significant differences existed between response types for both quality and empathy (P < 0.0001, P < 0.0001). For quality, Expert + AI (4.16 ± 0.81) performed the best, followed by GPT-4 (4.04 ± 0.92), GPT-3.5 (3.99 ± 0.87), Claude (3.6 ± 1.09), Expert (3.56 ± 1.01), Bard (3.5 ± 1.15), and Bing (3.04 ± 1.12). For empathy, Expert + AI (3.63 ± 0.87) had the highest score, followed by GPT-4 (3.6 ± 0.88), Bard (3.54 ± 0.89), GPT-3.5 (3.5 ± 0.83), Bing (3.27 ± 1.03), Expert (3.26 ± 1.08), and Claude (3.11 ± 0.78). For quality (P < 0.0001) and empathy (P = 0.002), Expert + AI performed better than Expert. Time taken for expert-created and expert-edited LLM responses was similar (P = 0.75).

CONCLUSIONS

Expert-edited LLM responses had the highest expert-determined ratings of quality and empathy warranting further exploration of their potential benefits in clinical settings.

Optic Neuropathy Associated with POLG Mutations: A Case Series and Literature Review

BACKGROUND

The clinical characteristics of patients with polymerase gamma (POLG) mutation-associated optic neuropathy remain incompletely characterized.

METHODS

We describe the clinical characteristics of 3 patients with POLG-associated optic neuropathy. We performed a literature review of optic neuropathy cases associated with POLG mutations and compared them with our cohort.

RESULTS

Many published cases of POLG-associated optic neuropathy in our literature review lacked details regarding severity of vision loss, visual field defects, and optical coherence tomography analysis. The clinical presentation of POLG mutations remains widely variable in age (from pediatric cases to adults) and associated systemic findings. All patients in our literature review presented with systemic symptoms, most commonly muscle weakness, ptosis, and ophthalmoplegia, whereas many young patients had severe systemic symptoms. In our case series, all 3 cases had isolated optic neuropathy affecting the papillomacular bundle, with signs such as reduced visual acuity and color vision, central visual field defects, temporal retinal nerve fiber layer loss with temporal optic disc pallor, and retinal ganglion cell complex loss. In addition, 2 of the 3 cases had added mitochondrial stressors in addition to the POLG mutation.

CONCLUSIONS

Clinicians should be aware that POLG mutations can present as isolated optic neuropathy primarily affecting the papillomacular bundle. With mitochondrial failure being the likely underlying pathogenic mechanism in POLG-associated optic neuropathy, helping affected patients eliminate mitochondrial stressors may be important in reducing the risk for progressive vision loss in this otherwise currently untreatable disorder.

Spastic Paraplegia Type 7-Associated Optic Neuropathy: A Case Series

BACKGROUND

Hereditary optic neuropathies comprise a group of clinically and genetically heterogeneous disorders. Optic neuropathy has been previously reported in families with spastic paraplegia type 7 (SPG7) gene mutations. However, the typical time course and clinical presentation of SPG7-associated optic neuropathy is poorly understood. We report a series of 5 patients harboring pathogenic SPG7 mutations who originally presented to a neuro-ophthalmology clinic with symptoms of optic neuropathy.

METHODS

Retrospective case series of 5 patients with pathogenic SPG7 mutations and optic atrophy from 3 neuro-ophthalmology clinics. Demographic, clinical, diagnostic, and treatment data were collected and reported by the clinician authors.

RESULTS

Five patients ranging in age from 8 to 48 years were evaluated in the neuro-ophthalmology clinic. Although there were variable clinical presentations for each subject, all noted progressive vision loss, typically bilateral, and several also had previous diagnoses of peripheral neuropathy (e.g., Guillain-Barré Syndrome). Patients underwent neuro-ophthalmic examinations and testing with visual fields and optic coherence tomography of the retinal nerve fiber layer. Genetic testing revealed pathogenic variants in the SPG7 gene.

CONCLUSIONS

Five patients presented to the neuro-ophthalmology clinic with progressive vision loss and were diagnosed with optic atrophy. Although each patient harbored an SPG7 mutation, this cohort was phenotypically and genotypically heterogeneous. Three patients carried the Ala510Val variant. The patients demonstrated varying degrees of visual acuity and visual field loss, although evaluations were completed during different stages of disease progression. Four patients had a previous diagnosis of peripheral neuropathy. This raises the prospect that a single pathogenic variant of SPG7 may be associated with peripheral neuropathy in addition to optic neuropathy. These results support the consideration of SPG7 testing in patients with high suspicion for genetic optic neuropathy, as manifested by symmetric papillomacular bundle damage without clear etiology on initial workup. Applied judiciously, genetic testing, including for SPG7, may help clarify the cause of unexplained progressive optic neuropathies.

Association of Brain Volume and Retinal Thickness in the Early Stages of Alzheimer's Disease

BACKGROUND

The eye has been considered a 'window to the brain,' and several neurological diseases including neurodegenerative conditions like Alzheimer's disease (AD) also show changes in the retina.

OBJECTIVE

To investigate retinal nerve fiber layer (RNFL) thickness and its association with brain volume via magnetic resonance imaging (MRI) in older adults with subjective or objective cognitive decline.

METHODS

75 participants underwent ophthalmological and neurological evaluation including optical coherence tomography and MRI (28 cognitively normal subjects, 26 with subjective cognitive decline, 17 patients diagnosed with mild cognitive impairment, and 4 with AD). Differences in demographics, thickness of RNFL, and brain volume were assessed using ANCOVA, while partial Pearson correlations, covaried for age and sex, were used to compare thickness of the peripapillary RNFL with brain volumes, with p < 0.05 considered statistically significant.

RESULTS

Mean RNFL thickness was significantly correlated with brain volumes, including global volume (right eye r = 0.235 p = 0.046, left eye r = 0.244, p = 0.037), temporal lobe (right eye r = 0.242 p = 0.039, left eye r = 0.290, p = 0.013), hippocampal (right eye r = 0.320 p = 0.005, left eye r = 0.306, p = 0.008), amygdala (left eye r = 0.332, p = 0.004), and occipital lobe (right eye r = 0.264 p = 0.024) volumes.

CONCLUSION

RNFL thickness in both eyes was positively associated with brain volumes in subjects with subjective and objective cognitive decline. The RNFL, however, did not correlate with the disease, but the small sample number makes it important to conduct larger studies. RNFL thickness may be a useful non-invasive and inexpensive tool for detection of brain neurodegeneration and may assist with diagnosis and monitoring of progression and treatment in AD.

The Impact of COVID-19 on Neuro-Ophthalmology Office Visits and Adoption of Telemedicine Services

BACKGROUND

The COVID-19 public health emergency (PHE) has significantly changed medical practice in the United States, including an increase in the utilization of telemedicine. Here, we characterize change in neuro-ophthalmic care delivery during the early COVID-19 PHE, including a comparison of care delivered via telemedicine and in office.

METHODS

Neuro-ophthalmology outpatient encounters from 3 practices in the United States (4 providers) were studied during the early COVID-19 PHE (March 15, 2020-June 15, 2020) and during the same dates 1 year prior. For unique patient visits, patient demographics, visit types, visit format, and diagnosis were compared between years and between synchronous telehealth and in-office formats for 2020.

RESULTS

There were 1,276 encounters for 1,167 patients. There were 30% fewer unique patient visits in 2020 vs 2019 (477 vs 670) and 55% fewer in-office visits (299 vs 670). Compared with 2019, encounters in 2020 were more likely to be established, to occur via telemedicine and to relate to an efferent diagnosis. In 2020, synchronous telehealth visits were more likely to be established compared with in-office encounters.

CONCLUSIONS

In the practices studied, a lower volume of neuro-ophthalmic care was delivered during the early COVID-19 public health emergency than in the same period in 2019. The type of care shifted toward established patients with efferent diagnoses and the modality of care shifted toward telemedicine.

Patient Harm Due to Diagnostic Error of Neuro-Ophthalmologic Conditions

PURPOSE

To prospectively examine diagnostic error of neuro-ophthalmic conditions and resultant harm at multiple sites.

DESIGN

Prospective, cross-sectional study.

PARTICIPANTS

A total of 496 consecutive adult new patients seen at 3 university-based neuro-ophthalmology clinics in the United States in 2019 to 2020.

METHODS

Collected data regarding demographics, prior care, referral diagnosis, final diagnosis, diagnostic testing, treatment, patient disposition, and impact of the neuro-ophthalmologic encounter. For misdiagnosed patients, we identified the cause of error using the Diagnosis Error Evaluation and Research (DEER) taxonomy tool and whether the patient experienced harm due to the misdiagnosis.

MAIN OUTCOME MEASURES

The primary outcome was whether patients who were misdiagnosed before neuro-ophthalmology referral experienced harm as a result of the misdiagnosis. Secondary outcomes included appropriateness of referrals, misdiagnosis rate, interventions undergone before referral, and the primary type of diagnostic error.

RESULTS

Referral diagnosis was incorrect in 49% of cases. A total of 26% of misdiagnosed patients experienced harm, which could have been prevented by earlier referral to neuro-ophthalmology in 97%. Patients experienced inappropriate laboratory testing, diagnostic imaging, or treatment before referral in 23%, with higher rates for patients misdiagnosed before referral (34% of patients vs. 13% with a correct referral diagnosis, P < 0.0001). Seventy-six percent of inappropriate referrals were misdiagnosed, compared with 45% of appropriate referrals (P < 0.0001). The most common reasons for referral were optic neuritis or optic neuropathy (21%), papilledema (18%), diplopia or cranial nerve palsies (16%), and unspecified vision loss (11%). The most common sources of diagnostic error were the physical examination (36%), generation of a complete differential diagnosis (24%), history taking (24%), and use or interpretation of diagnostic testing (13%). In 489 of 496 patients (99%), neuro-ophthalmology consultation (NOC) affected patient care. In 2% of cases, neuro-ophthalmology directly saved the patient's life or vision; in an additional 10%, harmful treatment was avoided or appropriate urgent referral was provided; and in an additional 48%, neuro-ophthalmology provided a diagnosis and direction to the patient's care.

CONCLUSIONS

Misdiagnosis of neuro-ophthalmic conditions, mismanagement before referral, and preventable harm are common. Early appropriate referral to neuro-ophthalmology may prevent patient harm.

UNILATERAL OPTIC NEURITIS AND CENTRAL RETINAL VASCULITIS DUE TO OCULAR SYPHILIS

PURPOSE

Report a case of concurrent unilateral optic neuritis and central retinal artery occlusion as the presenting signs of syphilis.

METHODS

A case report of a 22-year-old man with progressive unilateral vision loss.

RESULTS

With no known previous history of syphilis, genital lesions, or other extraocular manifestations, the patient presented with pain with eye movements and decreased color vision. His vision dramatically worsened after a course of oral steroids. Examination was remarkable for severe right optic disk edema with a macular cherry-red spot and mild posterior uveitis. Magnetic resonance imaging of the orbits with contrast revealed enhancement and enlargement of the distal right optic nerve. Fluorescein angiography demonstrated delayed filling of the right central retinal artery, suggestive of impending central retinal artery occlusion. Syphilis serologies were positive from the serum, and cerebrospinal fluid Venereal Disease Research Laboratory test was reactive, consistent with neurosyphilis. Oral steroids were discontinued and vision improved with 2 weeks of intravenous penicillin.

CONCLUSION

This unusual case highlights one of the possible initial presentations of syphilis: unilateral optic neuritis and central retinal artery vasculitis with mild posterior uveitis. The worsening of vision after administration of oral steroids also highlights a potential complication of oral steroid use in the absence of a known etiology of vision loss. A thorough history and examination may be helpful in identifying risk factors for infectious causes, including syphilis, and should prompt additional evaluation.

Ocular Fundus Photography as an Educational Tool

The proficiency of nonophthalmologists with direct ophthalmoscopy is poor, which has prompted a search for alternative technologies to examine the ocular fundus. Although ocular fundus photography has existed for decades, its use has been traditionally restricted to ophthalmology clinical care settings and textbooks. Recent research has shown a role for nonmydriatic fundus photography in nonophthalmic settings, encouraging more widespread adoption of fundus photography technology. Recent studies have also affirmed the role of fundus photography as an adjunct or alternative to direct ophthalmoscopy in undergraduate medical education. In this review, the authors examine the use of ocular fundus photography as an educational tool and suggest future applications for this important technology. Novel applications of fundus photography as an educational tool have the potential to resurrect the dying art of funduscopy.

The demise of direct ophthalmoscopy: A modern clinical challenge

Ocular funduscopy appears to be a dying art. Physicians and medical students alike lack confidence in the use of an ophthalmoscope. As a result, few clinicians perform ophthalmoscopy, and many who do are unable to reliably detect abnormalities of the ocular fundus. Approaches to remediation in undergraduate medical education have included simulators, longitudinal skill reinforcement, Web-based teaching, and other techniques. Preservation of the ophthalmoscopic art has been hindered by technical difficulty, waning enthusiasm for ophthalmoscopy, and even discouragement from preceptors in medical education. Ocular fundus photography may serve a role in medical education to help improve student confidence in interpretation of ocular fundus findings and improve awareness of the importance of examination of the ocular fundus. Because neurology clerkships and clinical practices remain an important forum for honing ocular funduscopy skills, the neurologist should be familiar with novel alternative techniques that facilitate examination of the ocular fundus.

Central nervous system distribution of the transcription factor Phox2b in the adult rat

Phox2b is required for development of the peripheral autonomic nervous system and a subset of cranial nerves and lower brainstem nuclei. Phox2b mutations in man cause diffuse autonomic dysfunction and deficits in the automatic control of breathing. Here we study the distribution of Phox2b in the adult rat hindbrain to determine whether this protein is selectively expressed by neurons involved in respiratory and autonomic control. In the medulla oblongata, Phox2b-immunoreactive nuclei were present in the dorsal vagal complex, intermediate reticular nucleus, dorsomedial spinal trigeminal nucleus, nucleus ambiguus, catecholaminergic neurons, and retrotrapezoid nucleus (RTN). Phox2b was expressed by both central excitatory relays of the sympathetic baroreflex (nucleus of the solitary tract and C1 neurons) but not by the inhibitory relay of this reflex. Phox2b was absent from the ventral respiratory column (VRC) caudal to RTN and rare within the parabrachial nuclei. In the pons, Phox2b was confined to cholinergic efferent neurons (salivary, vestibulocochlear) and noncholinergic peritrigeminal neurons. Rostral to the pons, Phox2b was detected only in the oculomotor complex. In adult rats, Phox2b is neither a comprehensive nor a selective marker of hindbrain autonomic pathways. This marker identifies a subset of hindbrain neurons that control orofacial movements (dorsomedial spinal trigeminal nucleus, pontine peritrigeminal neurons), balance and auditory function (vestibulocochlear efferents), the eyes, and both divisions of the autonomic efferent system. Phox2b is virtually absent from the respiratory rhythm and pattern generator (VRC and dorsolateral pons) but is highly expressed by neurons involved in the chemical drive and reflex regulation of this oscillator.

Inspiratory bursts in the preBötzinger complex depend on a calcium-activated non-specific cation current linked to glutamate receptors in neonatal mice

Inspiratory neurons of the preBötzinger complex (preBötC) form local excitatory networks and display 10-30 mV transient depolarizations, dubbed inspiratory drive potentials, with superimposed spiking. AMPA receptors are critical for rhythmogenesis under normal conditions in vitro but whether other postsynaptic mechanisms contribute to drive potential generation remains unknown. We examined synaptic and intrinsic membrane properties that generate inspiratory drive potentials in preBötC neurons using neonatal mouse medullary slice preparations that generate respiratory rhythm. We found that NMDA receptors, group I metabotropic glutamate receptors (mGluRs), but not group II mGluRs, contributed to inspiratory drive potentials. Subtype 1 of the group I mGluR family (mGluR1) probably regulates a K+ channel, whereas mGluR5 operates via an inositol 1,4,5-trisphosphate (IP3) receptor-dependent mechanism to augment drive potential generation. We tested for and verified the presence of a Ca2+-activated non-specific cation current (I(CAN)) in preBötC neurons. We also found that high concentrations of intracellular BAPTA, a high-affinity Ca2+ chelator, and the I(CAN) antagonist flufenamic acid (FFA) decreased the magnitude of drive potentials. We conclude that I(CAN) underlies robust inspiratory drive potentials in preBötC neurons, and is only fully evoked by ionotropic and metabotropic glutamatergic synaptic inputs, i.e. by network activity.

Role of persistent sodium current in mouse preBötzinger Complex neurons and respiratory rhythm generation

Breathing movements in mammals depend on respiratory neurons in the preBötzinger Complex (preBötC), which comprise a rhythmic network and generate robust bursts that form the basis for inspiration. Persistent Na(+) current (I(NaP)) is widespread in the preBötC and is hypothesized to play a critical role in rhythm generation because of its subthreshold activation and slow inactivation properties that putatively promote long-lasting burst depolarizations. In neonatal mouse slice preparations that retain the preBötC and generate a respiratory-related rhythm, we tested the role of I(NaP) with multiple Na(+) channel antagonists: tetrodotoxin (TTX; 20 nM), riluzole (RIL; 10 microM), and the intracellular Na(+) channel antagonist QX-314 (2 mM). Here we show that I(NaP) promotes intraburst spiking in preBötC neurons but surprisingly does not contribute to the depolarization that underlies inspiratory bursts, i.e. the inspiratory drive potential. Local microinjection in the preBötC of 10 microM RIL or 20 nM TTX does not perturb respiratory frequency, even in the presence of bath-applied 100 microM flufenamic acid (FFA), which attenuates a Ca(2+)-activated non-specific cation current (I(CAN)) that may also have burst-generating functionality. These data contradict the hypothesis that I(NaP) in preBötC neurons is obligatory for rhythmogenesis. However, in the presence of FFA, local microinjection of 10 microM RIL in the raphe obscurus causes rhythm cessation, which suggests that I(NaP) regulates the excitability of neurons outside the preBötC, including serotonergic raphe neurons that project to, and help maintain, rhythmic preBötC function.

Sodium and calcium current-mediated pacemaker neurons and respiratory rhythm generation

The breathing motor pattern in mammals originates in brainstem networks. Whether pacemaker neurons play an obligatory role remains a key unanswered question. We performed whole-cell recordings in the preBotzinger Complex in slice preparations from neonatal rodents and tested for pacemaker activity. We observed persistent Na+ current (I(NaP))-mediated bursting in approximately 5% of inspiratory neurons in postnatal day 0 (P0)-P5 and in P8-P10 slices. I(NaP)-mediated bursting was voltage dependent and blocked by 20 mum riluzole (RIL). We found Ca2+ current (I(Ca))-dependent bursting in 7.5% of inspiratory neurons in P8-P10 slices, but in P0-P5 slices these cells were exceedingly rare (0.6%). This bursting was voltage independent and blocked by 100 microm Cd2+ or flufenamic acid (FFA) (10-200 microm), which suggests that a Ca2+-activated inward cationic current (I(CAN)) underlies burst generation. These data substantiate our observation that P0-P5 slices exposed to RIL contain few (if any) pacemaker neurons, yet maintain respiratory rhythm. We also show that 20 nm TTX or coapplication of 20 microm RIL + FFA (100-200 microm) stops the respiratory rhythm, but that adding 2 mum substance P restarts it. We conclude that I(NaP) and I(CAN) enhance neuronal excitability and promote rhythmogenesis, even if their magnitude is insufficient to support bursting-pacemaker activity in individual neurons. When I(NaP) and I(CAN) are removed pharmacologically, the rhythm can be maintained by boosting neural excitability, which is inconsistent with a pacemaker-essential mechanism of respiratory rhythmogenesis by the preBotzinger complex.