Natural language processing to automate a web-based model of care and modernize skin cancer multidisciplinary team meetings

BACKGROUND

Cancer multidisciplinary team (MDT) meetings are under intense pressure to reform given the rapidly rising incidence of cancer and national mandates for protocolized streaming of cases. The aim of this study was to validate a natural language processing (NLP)-based web platform to automate evidence-based MDT decisions for skin cancer with basal cell carcinoma as a use case.

METHODS

A novel and validated NLP information extraction model was used to extract perioperative tumour and surgical factors from histopathology reports. A web application with a bespoke application programming interface used data from this model to provide an automated clinical decision support system, mapped to national guidelines and generating a patient letter to communicate ongoing management. Performance was assessed against retrospectively derived recommendations by two independent and blinded expert clinicians.

RESULTS

There were 893 patients (1045 lesions) used to internally validate the model. High accuracy was observed when compared against human predictions, with an overall value of 0.92. Across all classifiers the virtual skin MDT was highly specific (0.96), while sensitivity was lower (0.72).

CONCLUSION

This study demonstrates the feasibility of a fully automated, virtual, web-based service model to host the skin MDT with good system performance. This platform could be used to support clinical decision-making during MDTs as 'human in the loop' approach to aid protocolized streaming. Future prospective studies are needed to validate the model in tumour types where guidelines are more complex.

Genetic influences on epilepsy outcomes: A whole-exome sequencing and health care records data linkage study

OBJECTIVE

This study was undertaken to develop a novel pathway linking genetic data with routinely collected data for people with epilepsy, and to analyze the influence of rare, deleterious genetic variants on epilepsy outcomes.

METHODS

We linked whole-exome sequencing (WES) data with routinely collected primary and secondary care data and natural language processing (NLP)-derived seizure frequency information for people with epilepsy within the Secure Anonymised Information Linkage Databank. The study participants were adults who had consented to participate in the Swansea Neurology Biobank, Wales, between 2016 and 2018. DNA sequencing was carried out as part of the Epi25 collaboration. For each individual, we calculated the total number and cumulative burden of rare and predicted deleterious genetic variants and the total of rare and deleterious variants in epilepsy and drug metabolism genes. We compared these measures with the following outcomes: (1) no unscheduled hospital admissions versus unscheduled admissions for epilepsy, (2) antiseizure medication (ASM) monotherapy versus polytherapy, and (3) at least 1 year of seizure freedom versus <1 year of seizure freedom.

RESULTS

We linked genetic data for 107 individuals with epilepsy (52% female) to electronic health records. Twenty-six percent had unscheduled hospital admissions, and 70% were prescribed ASM polytherapy. Seizure frequency information was linked for 100 individuals, and 10 were seizure-free. There was no significant difference between the outcome groups in terms of the exome-wide and gene-based burden of rare and deleterious genetic variants.

SIGNIFICANCE

We successfully uploaded, annotated, and linked genetic sequence data and NLP-derived seizure frequency data to anonymized health care records in this proof-of-concept study. We did not detect a genetic influence on real-world epilepsy outcomes, but our study was limited by a small sample size. Future studies will require larger (WES) data to establish genetic variant contribution to epilepsy outcomes.

GWAS meta-analysis of over 29,000 people with epilepsy identifies 26 risk loci and subtype-specific genetic architecture

Epilepsy is a highly heritable disorder affecting over 50 million people worldwide, of which about one-third are resistant to current treatments. Here we report a multi-ancestry genome-wide association study including 29,944 cases, stratified into three broad categories and seven subtypes of epilepsy, and 52,538 controls. We identify 26 genome-wide significant loci, 19 of which are specific to genetic generalized epilepsy (GGE). We implicate 29 likely causal genes underlying these 26 loci. SNP-based heritability analyses show that common variants explain between 39.6% and 90% of genetic risk for GGE and its subtypes. Subtype analysis revealed markedly different genetic architectures between focal and generalized epilepsies. Gene-set analyses of GGE signals implicate synaptic processes in both excitatory and inhibitory neurons in the brain. Prioritized candidate genes overlap with monogenic epilepsy genes and with targets of current antiseizure medications. Finally, we leverage our results to identify alternate drugs with predicted efficacy if repurposed for epilepsy treatment.

Epilepsy mortality in Wales during COVID-19

Purpose

We aimed to compare mortality rates in people with epilepsy in Wales during the pandemic with pre-pandemic rates.

Methods

We performed a retrospective study using population-scale anonymised health records. We identified deaths in people with epilepsy (DPWE), those with a diagnosis of epilepsy, and deaths associ- ated with epilepsy (DAE), where epilepsy was recorded as a cause of death. We compared death rates in 2020 with average rates in 2015–2019 using Poisson models.

Results

There were 188 DAE and 628 DPWE in Wales in 2020 (death rates: 7.7/100,000/year and 25.7/100,000/year). The average rates for DAE and DPWE from 2015 to 2019 were 5.8/100,000/year and 23.8/100,000/year, respectively. Death rate ratios (2020 compared to 2015–2019) for DAE were 1.34 (95%CI 1.14–1.57, p<0.001) and for DPWE were 1.08 (0.99–1.17, p = 0.09). The death rate ratios for non-COVID deaths (deaths without COVID mentioned on death certificates) for DAE were 1.17 (0.99–1.39, p = 0.06) and for DPWE were 0.96 (0.87–1.05, p = 0.37).

Conclusions

The significant increase in DAE in Wales during 2020 could be explained by the direct effect of COVID-19 infection. Non-COVID-19 deaths have not increased significantly but further work is needed to assess the longer-term impact.

Development and validation of an automated basal cell carcinoma histopathology information extraction system using natural language processing

Introduction

Routinely collected healthcare data are a powerful research resource, but often lack detailed disease-specific information that is collected in clinical free text such as histopathology reports. We aim to use natural Language Processing (NLP) techniques to extract detailed clinical and pathological information from histopathology reports to enrich routinely collected data.

Methods

We used the general architecture for text engineering (GATE) framework to build an NLP information extraction system using rule-based techniques. During validation, we deployed our rule-based NLP pipeline on 200 previously unseen, de-identified and pseudonymised basal cell carcinoma (BCC) histopathological reports from Swansea Bay University Health Board, Wales, UK. The results of our algorithm were compared with gold standard human annotation by two independent and blinded expert clinicians involved in skin cancer care.

Results

We identified 11,224 items of information with a mean precision, recall, and F1 score of 86.0% (95% CI: 75.1-96.9), 84.2% (95% CI: 72.8-96.1), and 84.5% (95% CI: 73.0-95.1), respectively. The difference between clinician annotator F1 scores was 7.9% in comparison with 15.5% between the NLP pipeline and the gold standard corpus. Cohen's Kappa score on annotated tokens was 0.85.

Conclusion

Using an NLP rule-based approach for named entity recognition in BCC, we have been able to develop and validate a pipeline with a potential application in improving the quality of cancer registry data, supporting service planning, and enhancing the quality of routinely collected data for research.

Heterogeneity of resting-state EEG features in juvenile myoclonic epilepsy and controls

Abnormal EEG features are a hallmark of epilepsy, and abnormal frequency and network features are apparent in EEGs from people with idiopathic generalized epilepsy in both ictal and interictal states. Here, we characterize differences in the resting-state EEG of individuals with juvenile myoclonic epilepsy and assess factors influencing the heterogeneity of EEG features. We collected EEG data from 147 participants with juvenile myoclonic epilepsy through the Biology of Juvenile Myoclonic Epilepsy study. Ninety-five control EEGs were acquired from two independent studies [Chowdhury et al. (2014) and EU-AIMS Longitudinal European Autism Project]. We extracted frequency and functional network-based features from 10 to 20 s epochs of resting-state EEG, including relative power spectral density, peak alpha frequency, network topology measures and brain network ictogenicity: a computational measure of the propensity of networks to generate seizure dynamics. We tested for differences between epilepsy and control EEGs using univariate, multivariable and receiver operating curve analysis. In addition, we explored the heterogeneity of EEG features within and between cohorts by testing for associations with potentially influential factors such as age, sex, epoch length and time, as well as testing for associations with clinical phenotypes including anti-seizure medication, and seizure characteristics in the epilepsy cohort. P-values were corrected for multiple comparisons. Univariate analysis showed significant differences in power spectral density in delta (2-5 Hz) (P = 0.0007, hedges' g = 0.55) and low-alpha (6-9 Hz) (P = 2.9 × 10-8, g = 0.80) frequency bands, peak alpha frequency (P = 0.000007, g = 0.66), functional network mean degree (P = 0.0006, g = 0.48) and brain network ictogenicity (P = 0.00006, g = 0.56) between epilepsy and controls. Since age (P = 0.009) and epoch length (P = 1.7 × 10-8) differed between the two groups and were potential confounders, we controlled for these covariates in multivariable analysis where disparities in EEG features between epilepsy and controls remained. Receiver operating curve analysis showed low-alpha power spectral density was optimal at distinguishing epilepsy from controls, with an area under the curve of 0.72. Lower average normalized clustering coefficient and shorter average normalized path length were associated with poorer seizure control in epilepsy patients. To conclude, individuals with juvenile myoclonic epilepsy have increased power of neural oscillatory activity at low-alpha frequencies, and increased brain network ictogenicity compared with controls, supporting evidence from studies in other epilepsies with considerable external validity. In addition, the impact of confounders on different frequency-based and network-based EEG features observed in this study highlights the need for careful consideration and control of these factors in future EEG research in idiopathic generalized epilepsy particularly for their use as biomarkers.

204 Idiopathic intracranial hypertension in Wales: population characterisation, epidemiological trends and healthcare utilisation

Objective

To characterise the Welsh idiopathic intracranial hypertension (IIH) population, epidemiologi- cal trends and healthcare outcomes using routinely collected healthcare data.

Methods

We used primary and secondary care healthcare diagnostic codes within the Secure Anonymised Information Linkage databank to ascertain IIH cases and controls in a retrospective cohort study between 2003–2017. We validated IIH diagnosis codes using anonymised secondary care lists of IIH cases.

Results

We analysed 35 million patient years of data (2003–2017). There were 1765 cases of IIH in 2017 (85% female). The prevalence and incidence of IIH in 2017 was 76/100,000 and 7.8/100,000, significantly increased from 2003 (prevalence=12/100,000, incidence=2.3/100,000). IIH prevalence is associated with socio-economic deprivation and increasing body mass index (BMI). 9% of people with IIH had CSF shunts with less than 0.2% having bariatric surgery. Unscheduled hospital admissions were significantly higher in the IIH cohort compared to controls; and also in IIH patients with CSF shunts compared to those without.

Conclusions

IIH incidence and prevalence is increasing significantly, corresponding to population increases in BMI. This has important implications for healthcare professionals and policy makers given the comor- bidities, complications and increased healthcare utilisation and economic burden associated with IIH.

lotif_miah@hotmail.com

146 Epilepsy, deprivation and mortality in Wales 2005–2017

Background

Public Health England have recently reported that deaths associated with epilepsy are increasing and are associated with increased deprivation. We investigated comparable Welsh mortality trends and associations between epilepsy mortality and deprivation.

Method

We used routinely-collected health data within the Secure Anonymised Information Linkage (SAIL) Databank. We recorded deaths associated with epilepsy (DAE), epilepsy recorded on death certificates, and deaths in people with epilepsy (DPWE), people with diagnoses of epilepsy and epilepsy prescriptions before death. We compared death rates in different deprivation deciles adjusting for epilepsy prevalence.

Results

During 2005–2017 (41million patient-years) there were 2116 DAE and 7821 DPWE. DAE and DPWE increased from 4.3/100,000/yr and 17.2/100,000/yr in 2005–2007 to 5.7/100,000/yr and 20.9/100,000/yr in 2015–2017. The age-standardised mortality rates (ASMR) in 2006–2008 for DAE and DPWE were 5.3/100,000/yr and 20/100,000/yr respectively, in 2015–2017 they were 5.8/100,000/yr and 20/100,000/yr. DAE were not significantly associated with deprivation when adjusted for epilepsy prevalence.

Conclusion

When adjusting for age, deaths associated wtih epilepsy and deaths in people with epilepsy did not increase significantly in Wales between 2005–2007 and 2015–2017. The association between dep- rivation and deaths associated with epilepsy appears to be explained by higher epilepsy prevalence in areas of higher deprivation.

w.o.pickrell@swansea.ac.uk

Epilepsy mortality in Wales during COVID-19

PURPOSE

The COVID-19 pandemic has increased mortality worldwide and those with chronic conditions may have been disproportionally affected. However, it is unknown whether the pandemic has changed mortality rates for people with epilepsy. We aimed to compare mortality rates in people with epilepsy in Wales during the pandemic with pre-pandemic rates.

METHODS

We performed a retrospective study using individual-level linked population-scale anonymised electronic health records. We identified deaths in people with epilepsy (DPWE), i.e. those with a diagnosis of epilepsy, and deaths associated with epilepsy (DAE), where epilepsy was recorded as a cause of death on death certificates. We compared death rates in 2020 with average rates in 2015-2019 using Poisson models to calculate death rate ratios.

RESULTS

There were 188 DAE and 628 DPWE in Wales in 2020 (death rates: 7.7/100,000/year and 25.7/100,000/year). The average rates for DAE and DPWE from 2015 to 2019 were 5.8/100,000/year and 23.8/100,000/year, respectively. Death rate ratios (2020 compared to 2015-2019) for DAE were 1.34 (95%CI 1.14-1.57, p<0.001) and for DPWE were 1.08 (0.99-1.17, p = 0.09). The death rate ratios for non-COVID deaths (deaths without COVID mentioned on death certificates) for DAE were 1.17 (0.99-1.39, p = 0.06) and for DPWE were 0.96 (0.87-1.05, p = 0.37).

CONCLUSIONS

The significant increase in DAE in Wales during 2020 could be explained by the direct effect of COVID-19 infection. Non-COVID-19 deaths have not increased significantly but further work is needed to assess the longer-term impact.

Markup: A Web-Based Annotation Tool Powered by Active Learning

Across various domains, such as health and social care, law, news, and social media, there are increasing quantities of unstructured texts being produced. These potential data sources often contain rich information that could be used for domain-specific and research purposes. However, the unstructured nature of free-text data poses a significant challenge for its utilisation due to the necessity of substantial manual intervention from domain-experts to label embedded information. Annotation tools can assist with this process by providing functionality that enables the accurate capture and transformation of unstructured texts into structured annotations, which can be used individually, or as part of larger Natural Language Processing (NLP) pipelines. We present Markup (https://www.getmarkup.com/) an open-source, web-based annotation tool that is undergoing continued development for use across all domains. Markup incorporates NLP and Active Learning (AL) technologies to enable rapid and accurate annotation using custom user configurations, predictive annotation suggestions, and automated mapping suggestions to both domain-specific ontologies, such as the Unified Medical Language System (UMLS), and custom, user-defined ontologies. We demonstrate a real-world use case of how Markup has been used in a healthcare setting to annotate structured information from unstructured clinic letters, where captured annotations were used to build and test NLP applications.

Epilepsy, antiepileptic drugs, and the risk of major cardiovascular events

OBJECTIVE

This study was undertaken to determine whether epilepsy and antiepileptic drugs (including enzyme-inducing and non-enzyme-inducing drugs) are associated with major cardiovascular events using population-level, routinely collected data.

METHODS

Using anonymized, routinely collected, health care data in Wales, UK, we performed a retrospective matched cohort study (2003-2017) of adults with epilepsy prescribed an antiepileptic drug. Controls were matched with replacement on age, gender, deprivation quintile, and year of entry into the study. Participants were followed to the end of the study for the occurrence of a major cardiovascular event, and survival models were constructed to compare the time to a major cardiovascular event (cardiac arrest, myocardial infarction, stroke, ischemic heart disease, clinically significant arrhythmia, thromboembolism, onset of heart failure, or a cardiovascular death) for individuals in the case group versus the control group.

RESULTS

There were 10 241 cases (mean age = 49.6 years, 52.2% male, mean follow-up = 6.1 years) matched to 35 145 controls. A total of 3180 (31.1%) cases received enzyme-inducing antiepileptic drugs, and 7061 (68.9%) received non-enzyme-inducing antiepileptic drugs. Cases had an increased risk of experiencing a major cardiovascular event compared to controls (adjusted hazard ratio = 1.58, 95% confidence interval [CI] = 1.51-1.63, p < .001). There was no notable difference in major cardiovascular events between those treated with enzyme-inducing antiepileptic drugs and those treated with non-enzyme-inducing antiepileptic drugs (adjusted hazard ratio = .95, 95% CI = .86-1.05, p = .300).

SIGNIFICANCE

Individuals with epilepsy prescribed antiepileptic drugs are at an increased risk of major cardiovascular events compared with population controls. Being prescribed an enzyme-inducing antiepileptic drug is not associated with a greater risk of a major cardiovascular event compared to treatment with other antiepileptic drugs. Our data emphasize the importance of cardiovascular risk management in the clinical care of people with epilepsy.

Incidence, Prevalence, and Health Care Outcomes in Idiopathic Intracranial Hypertension: A Population Study

OBJECTIVE

To characterize trends in incidence, prevalence, and health care outcomes in the idiopathic intracranial hypertension (IIH) population in Wales using routinely collected health care data.

METHODS

We used and validated primary and secondary care IIH diagnosis codes within the Secure Anonymised Information Linkage databank to ascertain IIH cases and controls in a retrospective cohort study between 2003 and 2017. We recorded body mass index (BMI), deprivation quintile, CSF diversion surgery, and unscheduled hospital admissions in case and control cohorts.

RESULTS

We analyzed 35 million patient-years of data. There were 1,765 cases of IIH in 2017 (85% female). The prevalence and incidence of IIH in 2017 was 76/100,000 and 7.8/100,000/y, a significant increase from 2003 (corresponding figures = 12/100,000 and 2.3/100,000/y) (p < 0.001). IIH prevalence is associated with increasing BMI and increasing deprivation. The odds ratio for developing IIH in the least deprived quintile compared to the most deprived quintile, adjusted for sex and BMI, was 0.65 (95% confidence interval 0.55 to 0.76). Nine percent of IIH cases had CSF shunts with less than 0.2% having bariatric surgery. Unscheduled hospital admissions were higher in the IIH cohort compared to controls (rate ratio 5.28, p < 0.001) and in individuals with IIH and CSF shunts compared to those without shunts (rate ratio 2.02, p < 0.01).

CONCLUSIONS

IIH incidence and prevalence is increasing considerably, corresponding to population increases in BMI, and is associated with increased deprivation. This has important implications for health care professionals and policy makers given the comorbidities, complications, and increased health care utilization associated with IIH.

Idiopathic Intracranial Hypertension in Wales

IntroductionIdiopathic Intracranial Hypertension (IIH) is a condition of unknown aetiology that is strongly associated with obesity. IIH predominantly affects women of childbearing age and causes chronic disabling headaches, visual disturbance and, in a minority of patients, permanent visual loss. Objectives and ApproachWe characterised the IIH population, epidemiological trends and healthcare outcomes in Wales using routinely collected healthcare data. We used primary and secondary care healthcare diagnosis codes within the Secure Anonymised Information Linkage databank to ascertain IIH cases and controls in a retrospective cohort study between 2003 and 2017. We validated IIH diagnosis codes using anonymised secondary care lists of IIH cases. ResultsWe analysed 35 million patient years of data (2003–2017). There were 1765 cases of IIH in 2017 (85% female). The prevalence and incidence of IIH in 2017 was 76/100,000 and 7.8/100,000/year, a significant increase from 2003 (corresponding figures=12/100,000 and 2.3/100,000/year). The odds ratio for developing IIH in the least deprived quintile compared to the most deprived quintile, adjusted for gender and Body Mass Index (BMI), was 0.65 (95% CI=0.55–0.76). 9% of people with IIH had CSF shunts with less than 0.2% having bariatric surgery. Unscheduled hospital admissions were significantly higher in the IIH cohort compared to controls (rate ratio=5.28, p<0.001) and in the people with IIH with CSF shunts compared to those without (rate ratio=2.02, p<0.01). Conclusion / ImplicationsIIH incidence and prevalence is increasing significantly, corresponding to population increases in BMI, and is associated with increased deprivation. This has important implications for healthcare professionals and policy makers given the comorbidities, complications and increased healthcare utilisation associated with IIH.

Markup: A Web-Based Clinical Annotation Tool with Enhanced Ontology Mapping

IntroductionUnstructured free-text clinical notes often contain valuable information relating to patient symptoms, prescriptions and diagnoses. These can assist with better care for patients and novel healthcare research if transformed into accessible, structured clinical text. In particular, Natural Language Processing (NLP) algorithms can produce such structured outputs, but require gold standard data to train and validate their accuracy. While existing tools such as Brat and Webanno provide interfaces to manually annotate text, there is a lack of capability to efficiently annotate complex clinical information. Objectives and ApproachWe present Markup, an open-source, web-based annotation tool developed for use within clinical contexts by domain experts to produce gold standard annotations for NLP development. Markup incorporates NLP and Active Learning technologies to enable rapid and accurate annotation of unstructured documents. Markup supports custom user configurations, automated annotation suggestions, and automated mapping to existing clinical ontologies such as the Unified Medical Language System (UMLS), the Systematized Nomenclature of Medicine - Clinical Terms (SNOMED-CT), or custom, user-defined ontologies. ResultsMarkup has been tested on Epilepsy clinic letters, where captured annotations were used to build and test NLP applications. Markup allowed for inter-annotator statistics to be calculated in the case of multiple annotators. Re-annotation, following iterations of annotation definitions, was incorporated for flexibility. UMLS codes, certainty context, and multiple components from complex phrases were all able to be captured and exported in a structured format. Conclusions / ImplicationsMarkup allows gold standard annotations to be collected efficiently across unstructured text and is optimized to capture health-specific information. These annotations are important to develop and validate NLP algorithms that automate the capture of important information from clinic letters at scale.

A National Level Case-Control Study for Determining Risk of Major Cardiovascular Events in People with Epilepsy

IntroductionThe risk of cardiovascular events amongst people with epilepsy who are receiving enzyme-inducing anti-epileptic drugs (EIAEDs) seems to be higher than those on other medications and the general population. National-level record linkage enables development of case-control studies at a wider scope accounting for multiple factors. Objectives and ApproachPeople with epilepsy were identified between 2003-01-01 and 2017-12-31 and were matched to a control group on: age, gender, deprivation quintile and year of diagnosis, accounting for any changes in clinical therapeutic guidelines. Primary and secondary care population records were linked to capture relevant comorbidities and major cardiovascular events. Annual district birth and death extract were used in combination with the Welsh Demographic Service (WDS) dataset to capture demographic and cardiovascular related death records. The WDS dataset was used to identify eligible control groups for each case and a linkage approach between the control and case database was developed for matching cases and controls with replacement and randomization. Survival analysis was conducted to evaluate the difference in time to first major cardiovascular event in patients receiving EIAED versus Non-EIAEDs and controls. Results10,241 cases (mean age 49.6 years, 52.2% male) with diagnosis of epilepsy were matched to 35,145 controls. 3,180 (31.1%) cases received EIAEDs and 7,061 (68.9%) received non-EIAEDs. The risk of experiencing a major cardiovascular event was higher in cases compared to controls (adjusted hazard ratio 1.52,95%CI[1.50–1.55];p<0.001). There was no significant difference in cardiovascular events between those treated with non-EIAEDs and EIAEDs (adjusted hazard ratio 1.04,95%CI[0.95-1.12];p=0.407). Conclusion / ImplicationsData linkage provides a unique opportunity and insight into studying disease risk factors. We have shown that individuals with epilepsy prescribed antiepileptic drugs, re at an increased risk of a major cardiovascular events regardless of treatment type (EIAED,NEIAED) compared with a matched control population.

Using natural language processing to extract structured epilepsy data from unstructured clinic letters: development and validation of the ExECT (extraction of epilepsy clinical text) system

OBJECTIVE

Routinely collected healthcare data are a powerful research resource but often lack detailed disease-specific information that is collected in clinical free text, for example, clinic letters. We aim to use natural language processing techniques to extract detailed clinical information from epilepsy clinic letters to enrich routinely collected data.

DESIGN

We used the general architecture for text engineering (GATE) framework to build an information extraction system, ExECT (extraction of epilepsy clinical text), combining rule-based and statistical techniques. We extracted nine categories of epilepsy information in addition to clinic date and date of birth across 200 clinic letters. We compared the results of our algorithm with a manual review of the letters by an epilepsy clinician.

SETTING

De-identified and pseudonymised epilepsy clinic letters from a Health Board serving half a million residents in Wales, UK.

RESULTS

We identified 1925 items of information with overall precision, recall and F1 score of 91.4%, 81.4% and 86.1%, respectively. Precision and recall for epilepsy-specific categories were: epilepsy diagnosis (88.1%, 89.0%), epilepsy type (89.8%, 79.8%), focal seizures (96.2%, 69.7%), generalised seizures (88.8%, 52.3%), seizure frequency (86.3%-53.6%), medication (96.1%, 94.0%), CT (55.6%, 58.8%), MRI (82.4%, 68.8%) and electroencephalogram (81.5%, 75.3%).

CONCLUSIONS

We have built an automated clinical text extraction system that can accurately extract epilepsy information from free text in clinic letters. This can enhance routinely collected data for research in the UK. The information extracted with ExECT such as epilepsy type, seizure frequency and neurological investigations are often missing from routinely collected data. We propose that our algorithm can bridge this data gap enabling further epilepsy research opportunities. While many of the rules in our pipeline were tailored to extract epilepsy specific information, our methods can be applied to other diseases and also can be used in clinical practice to record patient information in a structured manner.

Validating epilepsy diagnoses in routinely collected data

IntroductionPrimary healthcare records are used for studies within large data repositories. One of the limitations of using these routinely collected data for epilepsy research is the possibility of including incorrectly recorded diagnoses. To our knowledge, the accuracy of UK GP diagnosis codes for epilepsy has only partially been validated. Objectives and ApproachWe aimed to validate the accuracy of case ascertainment algorithms in identifying people with epilepsy in routinely collected Welsh healthcare data. A reference population of 150 people with definite epilepsy and 150 people without epilepsy was ascertained from hospital records and linked to records held within the Secure Anonymised Information Linkage (SAIL) databank in Wales. We used three different algorithms to identify the reference population: a) individuals with an epilepsy diagnosis code and two consecutive AED prescription codes; b) individuals with an epilepsy diagnosis code only; c) individuals with two consecutive AED prescription codes only. ResultsWe applied the algorithms to all patients and to adults and children separately. For all patients, combining diagnosis and AED prescription codes had a sensitivity of 84% (95% ci 77–90) and specificity of 98% (95–100) in identifying people with epilepsy; diagnosis codes alone had a sensitivity of 86% (80–91) and a specificity of 97% (92–99); and AED prescription codes alone achieved a sensitivity of 92% (70–83) and a specificity of 73% (65–80). Using AED codes only was more accurate in children, achieving a sensitivity of 88% (75–95) and specificity of 98% (88–100). This can be explained by the widespread use of AEDs for indications other than epilepsy in adults, which is not the case for children. Conclusion/ImplicationsGP epilepsy diagnosis and AED prescription codes can be used to identify people with epilepsy using anonymised healthcare records in Wales. In children using AED prescription codes alone is an accurate way to identify epilepsy cases. These results are generalizable to other studies that use UK primary care records.

Using natural language processing to extract structured epilepsy data from unstructured clinic letters

IntroductionElectronic health records (EHR) are a powerful resource in enabling large-scale healthcare research. EHRs often lack detailed disease-specific information that is collected in free text within clinical settings. This challenge can be addressed by using Natural Language Processing (NLP) to derive and extract detailed clinical information from free text. Objectives and ApproachUsing a training sample of 40 letters, we used the General Architecture for Text Engineering (GATE) framework to build custom rule sets for nine categories of epilepsy information as well as clinic date and date of birth. We used a validation set of 200 clinic letters to compare the results of our algorithm to a separate manual review by a clinician, where we evaluated a “per item” and a “per letter” approach for each category. ResultsThe “per letter” approach identified 1,939 items of information with overall precision, recall and F1-score of 92.7%, 77.7% and 85.6%. Precision and recall for epilepsy specific categories were: diagnosis (85.3%,92.4%),  type (93.7%,83.2%), focal seizure (99.0%,68.3%), generalised seizure (92.5%,57.0%), seizure frequency (92.0%,52.3%), medication (96.1%,94.0%), CT (66.7%,47.1%), MRI (96.6%,51.4%) and EEG (95.8%,40.6%). By combining all items per category, per letter we were able to achieve higher precision, recall and F1-scores of 94.6%, 84.2% and 89.0% across all categories. Conclusion/ImplicationsOur results demonstrate that NLP techniques can be used to accurately extract rich phenotypic details from clinic letters that is often missing from routinely-collected data. Capturing these new data types provides a platform for conducting novel precision neurology research, in addition to potential applicability to other disease areas.

Validating epilepsy diagnoses in routinely collected data

•

Cases with and without epilepsy were linked with anonymised primary care data.

•

Primary care diagnosis and drug codes accurately identify the cases with epilepsy.

•

Drug codes alone can be used to identify children with epilepsy.

•

Combining drug and diagnosis codes for adults and children increases accuracy.

Purpose

Anonymised, routinely-collected healthcare data is increasingly being used for epilepsy research. We validated algorithms using general practitioner (GP) primary healthcare records to identify people with epilepsy from anonymised healthcare data within the Secure Anonymised Information Linkage (SAIL) databank in Wales, UK.

Method

A reference population of 150 people with definite epilepsy and 150 people without epilepsy was ascertained from hospital records and linked to records contained within SAIL (containing GP records for 2.4 million people). We used three different algorithms, using combinations of GP epilepsy diagnosis and anti-epileptic drug (AED) prescription codes, to identify the reference population.

Results

Combining diagnosis and AED prescription codes had a sensitivity of 84% (95% ci 77–90) and specificity of 98% (95–100) in identifying people with epilepsy; diagnosis codes alone had a sensitivity of 86% (80–91) and a specificity of 97% (92–99); and AED prescription codes alone achieved a sensitivity of 92% (70–83) and a specificity of 73% (65–80). Using AED codes only was more accurate in children achieving a sensitivity of 88% (75–95) and specificity of 98% (88–100).

Conclusion

GP epilepsy diagnosis and AED prescription codes can be confidently used to identify people with epilepsy using anonymised healthcare records in Wales, UK.

Obtaining structured clinical data from unstructured data using natural language processing software

ABSTRACT BackgroundFree text documents in healthcare settings contain a wealth of information not captured in electronic healthcare records (EHRs). Epilepsy clinic letters are an example of an unstructured data source containing a large amount of intricate disease information. Extracting meaningful and contextually correct clinical information from free text sources, to enhance EHRs, remains a significant challenge. SCANR (Swansea University Collaborative in the Analysis of NLP Research) was set up to use natural language processing (NLP) technology to extract structured data from unstructured sources. IBM Watson Content Analytics software (ICA) uses NLP technology. It enables users to define annotations based on dictionaries and language characteristics to create parsing rules that highlight relevant items. These include clinical details such as symptoms and diagnoses, medication and test results, as well as personal identifiers.   ApproachTo use ICA to build a pipeline to accurately extract detailed epilepsy information from clinic letters. MethodsWe used ICA to retrieve important epilepsy information from 41 pseudo-anonymized unstructured epilepsy clinic letters. The 41 letters consisted of 13 ‘new’ and 28 ‘follow-up’ letters (for 15 different patients) written by 12 different doctors in different styles. We designed dictionaries and annotators to enable ICA to extract epilepsy type (focal, generalized or unclassified), epilepsy cause, age of onset, investigation results (EEG, CT and MRI), medication, and clinic date. Epilepsy clinicians assessed the accuracy of the pipeline. ResultsThe accuracy (sensitivity, specificity) of each concept was: epilepsy diagnosis 98% (97%, 100%), focal epilepsy 100%, generalized epilepsy 98% (93%, 100%), medication 95% (93%, 100%), age of onset 100% and clinic date 95% (95%, 100%). Precision and recall for each concept were respectively, 98% and 97% for epilepsy diagnosis, 100% each for focal epilepsy, 100% and 93% for generalized epilepsy, 100% each for age of onset, 100% and 93% for medication, 100% and 96% for EEG results, 100% and 83% for MRI scan results, and 100% and 95% for clinic date. Conclusions ICA is capable of extracting detailed, structured epilepsy information from unstructured clinic letters to a high degree of accuracy. This data can be used to populate relational databases and be linked to EHRs. Researchers can build in custom rules to identify concepts of interest from letters and produce structured information. We plan to extend our work to hundreds and then thousands of clinic letters, to provide phenotypically rich epilepsy data to link with other anonymised, routinely collected data.

Codifying unstructured data: A Natural Language Processing approach to extract rich data from clinical letters

ABSTRACT ObjectivesElectronic healthcare records (EHR) are the main data sources that facilitate epidemiology research. Routinely collected data such as primary and secondary care are now easily linked to produce novel and high impact research. There are, however, rich data locked in the free text of clinical letters that are not otherwise translated into EHRs. It is highly desirable to be able to extract this information to strengthen the body of information in existing EHRs. The Swansea Collaborative in Analysis of NLP Research (SCANR) group at Swansea University has been established to evaluate the usage of Natural Language Processing platforms for obtaining new clinical data. To use Clix Enrich to extract SNOMED concepts from a variety of clinical free texts and produce EHRs from the extraction process. Approach SNOMED concepts contain common items of interest such as diagnosis, medication and symptoms, as well as contextual concepts such as historical reference and negation. Clix Enrich uses the SNOMED dictionary to encode clinical free text (pre-co-ordinated) and find contextually correct SNOMED concepts (post co-ordinated). We used Clix Enrich to extract meaningful clinical terms from MS and Epilepsy consultant letters, as well as presenting complaint fields from a Welsh Emergency Department (ED). ResultsWe tailored Clix Enrich to extract a wide variety of clinical terms from each source (fourty texts per source) and validated the extraction accuracy with clinical experts in each domain. Clix Enrich was able to accurately extract the correct diagnosis for MS, Epilepsy and ED attendance (100%, 95% and 80%), dosage and frequency of anti-epileptic medication and MS modifying therapy (90%, 100%) and EDDS score (94%). We note a probable source of discrepancy in extraction accuracy between letter sources in the frequency of abbreviated terms, particularly within the presenting complaint field of the ED sample. ConclusionClix Enrich can be used to accurately extract SNOMED concepts from clinical letters. The resulting datasets are readily available to link to existing EHRs, and can be linked to EHRs that adopt the SNOMED coding structure, or backward compatible hierarchies. Clix Enrich comes with out-of-the-box extraction methods but the optimum way to extract the correct information would be to build in custom queries, thus requiring clinical expertise to validate extraction.