Validity of Administrative Claims and Electronic Health Registry Data From a Single Practice for Eye Health Surveillance

Using Natural Language Processing to Identify Different Lens Pathology in Electronic Health Records

PURPOSE

Nearly all published ophthalmology-related Big Data studies rely exclusively on International Classification of Diseases (ICD) billing codes to identify patients with particular ocular conditions. However, inaccurate or nonspecific codes may be used. We assessed whether natural language processing (NLP), as an alternative approach, could more accurately identify lens pathology.

DESIGN

Database study comparing the accuracy of NLP versus ICD billing codes to properly identify lens pathology.

METHODS

We developed an NLP algorithm capable of searching free-text lens exam data in the electronic health record (EHR) to identify the type(s) of cataract present, cataract density, presence of intraocular lenses, and other lens pathology. We applied our algorithm to 17.5 million lens exam records in the Sight Outcomes Research Collaborative (SOURCE) repository. We selected 4314 unique lens-exam entries and asked 11 clinicians to assess whether all pathology present in the entries had been correctly identified in the NLP algorithm output. The algorithm's sensitivity at accurately identifying lens pathology was compared with that of the ICD codes.

RESULTS

The NLP algorithm correctly identified all lens pathology present in 4104 of the 4314 lens-exam entries (95.1%). For less common lens pathology, algorithm findings were corroborated by reviewing clinicians for 100% of mentions of pseudoexfoliation material and 99.7% for phimosis, subluxation, and synechia. Sensitivity at identifying lens pathology was better for NLP (0.98 [0.96-0.99] than for billing codes (0.49 [0.46-0.53]).

CONCLUSIONS

Our NLP algorithm identifies and classifies lens abnormalities routinely documented by eye-care professionals with high accuracy. Such algorithms will help researchers to properly identify and classify ocular pathology, broadening the scope of feasible research using real-world data.

The Impact of Documentation Workflow on the Accuracy of the Coded Diagnoses in the Electronic Health Record

Objective

To determine the impact of documentation workflow on the accuracy of coded diagnoses in electronic health records (EHRs).

Design

Cross-sectional study.

Participants

All patients who completed visits at the Casey Eye Institute Retina Division faculty clinic between April 7, 2022 and April 13, 2022.

Main Outcome Measures

Agreement between coded diagnoses and clinical notes.

Methods

We assessed the rate of agreement between the diagnoses in the clinical notes and the coded diagnosis in the EHR using manual review and examined the impact of the documentation workflow on the rate of agreement in an academic retina practice.

Results

In 202 visits by 8 physicians, 78% (range, 22%-100%) had an agreement between the coded diagnoses and the clinical notes. When physicians integrated the diagnosis code entry and note composition, the rate of agreement was 87.9% (range, 62%-100%). For those who entered the diagnosis codes separately from writing notes, the agreement was 44.4% (22%-50%, P < 0.0001).

Conclusion

The visit-specific agreement between the coded diagnosis and the progress note can vary widely by workflow. The workflow and EHR design may be an important part of understanding and improving the quality of EHR data.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.