Representation of ophthalmology concepts by electronic systems: adequacy of controlled medical terminologies

Concept Coverage Analysis of Ophthalmic Infections and Trauma among the Standardized Medical Terminologies SNOMED-CT, ICD-10-CM, and ICD-11

Purpose

Widespread electronic health record adoption has generated a large volume of data and emphasized the need for standardized terminology to describe clinical concepts. Here, we undertook a systematic concept coverage analysis to determine the representation of clinical concepts in ophthalmic infection and ophthalmic trauma among standardized medical terminologies, including the Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT), the International Classification of Diseases (ICD) version 10 with clinical modifications (ICD-10-CM), and ICD version 11 (ICD-11).

Design

Extraction of concepts related to ophthalmic infection and ophthalmic trauma and structured search in terminology browsers.

Data Sources

The American Academy of Ophthalmology Basic and Clinical Science Course (BCSC), SNOMED-CT, and ICD-10-CM terminologies from the Observational Health Data Sciences and Informatics Athena browser, and the ICD-11 terminology browser.

Methods

Concepts pertaining to ophthalmic infection and ophthalmic trauma were extracted from the 2022 BCSC free text and index terms. We searched terminology browsers to identify corresponding codes and classified the extent of semantic alignment as equal, wide, narrow, or unmatched in each terminology. The overlap of equal concepts in each terminology was represented in a Venn diagram.

Main Outcome Measures

Proportions of clinical concepts with corresponding codes at various levels of semantic alignment.

Results

A total of 443 concepts were identified: 304 concepts related to ophthalmic infection and 139 concepts related to ophthalmic trauma. The SNOMED-CT had the highest proportion of equal coverage, with 82.0% (249 of 304) among concepts related to ophthalmic infection and 82.0% (115 of 139) among concepts related to ophthalmic trauma. Across all concepts, 28% (124 of 443) were classified as equal in ICD-10-CM and 52.8% (234 of 443) were classified as equal in ICD-11.

Conclusions

The SNOMED-CT had significantly better semantic alignment than ICD-10-CM and ICD-11 for ophthalmic infections and ophthalmic trauma. This demonstrates opportunity for continuing advancement of representation of ophthalmic concepts in standardized medical terminologies.

A health terminological system for inherited retinal diseases: Content coverage evaluation and a proposed classification

PURPOSE

To present a classification of inherited retinal diseases (IRDs) and evaluate its content coverage in comparison with common standard terminology systems.

METHODS

In this comparative cross-sectional study, a panel of subject matter experts annotated a list of IRDs based on a comprehensive review of the literature. Then, they leveraged clinical terminologies from various reference sets including Unified Medical Language System (UMLS), Online Mendelian Inheritance in Man (OMIM), International Classification of Diseases (ICD-11), Systematized Nomenclature of Medicine (SNOMED-CT) and Orphanet Rare Disease Ontology (ORDO).

RESULTS

Initially, we generated a hierarchical classification of 62 IRD diagnosis concepts in six categories. Subsequently, the classification was extended to 164 IRD diagnoses after adding concepts from various standard terminologies. Finally, 158 concepts were selected to be classified into six categories and genetic subtypes of 412 cases were added to the related concepts. UMLS has the greatest content coverage of 90.51% followed respectively by SNOMED-CT (83.54%), ORDO (81.01%), OMIM (60.76%), and ICD-11 (60.13%). There were 53 IRD concepts (33.54%) that were covered by all five investigated systems. However, 2.53% of the IRD concepts in our classification were not covered by any of the standard terminologies.

CONCLUSIONS

This comprehensive classification system was established to organize IRD diseases based on phenotypic and genotypic specifications. It could potentially be used for IRD clinical documentation purposes and could also be considered a preliminary step forward to developing a more robust standard ontology for IRDs or updating available standard terminologies. In comparison, the greatest content coverage of our proposed classification was related to the UMLS Metathesaurus.

Barriers to electronic medical record implementation: a comparison between ophthalmology and other surgical specialties in Canada

OBJECTIVE

In the present study, the barriers limiting widespread adoption of electronic medical records (EMRs) among Canadian ophthalmologists were evaluated in comparison with physicians from other surgical specialities. The published literature regarding EMR use in ophthalmic practice was also reviewed.

DESIGN

Population-based, cross-sectional study.

PARTICIPANTS

A total of 1199 Canadian surgeons participating in the 2014 National Physician Survey (NPS).

METHODS

Data regarding speciality surgeons' adoption of EMR programs were extracted from the 2014 NPS, a nationwide survey of practicing physicians in Canada. The data were entered into a spreadsheet, and basic statistical analyses, including χ² analyses, were performed to compare the responses of ophthalmologists to other surgeons.

RESULTS

Compared with other surgeons, ophthalmologists surveyed were significantly more likely to identify the following barriers to EMR adoption: "no suitable product for my practice" (p = 0.01), "too costly" (p = 0.0006), "too time consuming" (p < 0.0001), and "planning to retire soon" (p = 0.001). No statistically detectable differences were found between ophthalmologists and other surgeons for the following barriers: privacy concerns, reliability concerns, and lack of training.

CONCLUSIONS

The barriers that limit increased EMR adoption among Canadian ophthalmologists are different from those of other surgeons. This may be attributed to unique features of the field, including heavy reliance on hand-drawn figures in documentation, high patient volume, and the high costs associated with independent practice. Given the well-established benefits of EMR technology, consideration should be given to implementing strategies to mitigate these barriers. Additional research may help determine which specific improvements can be made to increase the use of EMR systems by ophthalmologists.

An exploration of the properties of the CORE problem list subset and how it facilitates the implementation of SNOMED CT

Objective Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) is the emergent international health terminology standard for encoding clinical information in electronic health records. The CORE Problem List Subset was created to facilitate the terminology’s implementation. This study evaluates the CORE Subset’s coverage and examines its growth pattern as source datasets are being incorporated.

Methods Coverage of frequently used terms and the corresponding usage of the covered terms were assessed by “leave-one-out” analysis of the eight datasets constituting the current CORE Subset. The growth pattern was studied using a retrospective experiment, growing the Subset one dataset at a time and examining the relationship between the size of the starting subset and the coverage of frequently used terms in the incoming dataset. Linear regression was used to model that relationship.

Results On average, the CORE Subset covered 80.3% of the frequently used terms of the left-out dataset, and the covered terms accounted for 83.7% of term usage. There was a significant positive correlation between the CORE Subset’s size and the coverage of the frequently used terms in an incoming dataset. This implies that the CORE Subset will grow at a progressively slower pace as it gets bigger.

Conclusion The CORE Problem List Subset is a useful resource for the implementation of Systematized Nomenclature of Medicine Clinical Terms in electronic health records. It offers good coverage of frequently used terms, which account for a high proportion of term usage. If future datasets are incorporated into the CORE Subset, it is likely that its size will remain small and manageable.

Accuracy and speed of electronic health record versus paper-based ophthalmic documentation strategies

PURPOSE

To compare accuracy and speed of keyboard and mouse electronic health record (EHR) documentation strategies with those of a paper documentation strategy.

DESIGN

Prospective cohort study.

METHODS

Three documentation strategies were developed: (1) keyboard EHR, (2) mouse EHR, and (3) paper. Ophthalmology trainees recruited for the study were presented with 5 clinical cases and documented findings using each strategy. For each case-strategy pair, findings and documentation time were recorded. Accuracy of each strategy was calculated based on sensitivity (fraction of findings in actual case that were documented by subject) and positive ratio (fraction of findings identified by subject that were present in the actual case).

RESULTS

Twenty subjects were enrolled. A total of 258 findings were identified in the 5 cases, resulting in 300 case-strategy pairs and 77 400 possible total findings documented. Sensitivity was 89.1% for the keyboard EHR, 87.2% for mouse EHR, and 88.6% for the paper strategy (no statistically significant differences). The positive ratio was 99.4% for the keyboard EHR, 98.9% for mouse EHR, and 99.9% for the paper strategy (P < .001 for mouse EHR vs paper; no significant differences between other pairs). Mean ± standard deviation documentation speed was significantly slower for the keyboard (2.4 ± 1.1 seconds/finding) and mouse (2.2 ± 0.7 seconds/finding) EHR compared with the paper strategy (2.0 ± 0.8 seconds/finding). Documentation speed of the mouse EHR strategy worsened with repetition.

CONCLUSIONS

No documentation strategy was perfectly accurate in this study. Documentation speed for both EHR strategies was slower than with paper. Further studies involving total physician time requirements for ophthalmic EHRs are required.

Detection and characterization of usability problems in structured data entry interfaces in dentistry

BACKGROUND

Poor usability is one of the major barriers for optimally using electronic health records (EHRs). Dentists are increasingly adopting EHRs, and are using structured data entry interfaces to enter data such that the data can be easily retrieved and exchanged. Until recently, dentists have lacked a standardized terminology to consistently represent oral health diagnoses.

OBJECTIVES

In this study we evaluated the usability of a widely used EHR interface that allow the entry of diagnostic terms, using multi-faceted methods to identify problems and work with the vendor to correct them using an iterative design method.

METHODS

Fieldwork was undertaken at two clinical sites, and dental providers as subjects participated in user testing (n=32), interviews (n=36) and observations (n=24).

RESULTS

User testing revealed that only 22-41% of users were able to successfully complete a simple task of entering one diagnosis, while no user was able to complete a more complex task. We identified and characterized 24 high-level usability problems reducing efficiency and causing user errors. Interface-related problems included unexpected approaches for displaying diagnosis, lack of visibility, and inconsistent use of UI widgets. Terminology related issues included missing and mis-categorized concepts. Work domain issues involved both absent and superfluous functions. In collaboration with the vendor, each usability problem was prioritized and a timeline set to resolve the concerns.

DISCUSSION

Mixed methods evaluations identified a number of critical usability issues relating to the user interface, underlying terminology of the work domain. The usability challenges were found to prevent most users from successfully completing the tasks. Our further work we will determine if changes to the interface, terminology and work domain do result in improved usability.

Special requirements for electronic health record systems in ophthalmology

The field of ophthalmology has a number of unique features compared with other medical and surgical specialties regarding clinical workflow and data management. This has important implications for the design of electronic health record (EHR) systems that can be used intuitively and efficiently by ophthalmologists and that can promote improved quality of care. Ophthalmologists often lament the absence of these specialty-specific features in EHRs, particularly in systems that were developed originally for primary care physicians or other medical specialists. The purpose of this article is to summarize the special requirements of EHRs that are important for ophthalmology. The hope is that this will help ophthalmologists to identify important features when searching for EHR systems, to stimulate vendors to recognize and incorporate these functions into systems, and to assist federal agencies to develop future guidelines regarding meaningful use of EHRs. More broadly, the American Academy of Ophthalmology believes that these functions are elements of good system design that will improve access to relevant information at the point of care between the ophthalmologist and the patient, will enhance timely communications between primary care providers and ophthalmologists, will mitigate risk, and ultimately will improve the ability of physicians to deliver the highest-quality medical care.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial interest disclosure may be found after the references.

Migrating existing clinical content from ICD-9 to SNOMED

OBJECTIVE

To identify challenges in mapping internal International Classification of Disease, 9th edition, Clinical Modification (ICD-9-CM) encoded legacy data to Systematic Nomenclature of Medicine (SNOMED), using SNOMED-prescribed compositional approaches where appropriate, and to explore the mapping coverage provided by the US National Library of Medicine (NLM)'s SNOMED clinical core subset.

DESIGN

This study selected ICD-CM codes that occurred at least 100 times in the organization's problem list or diagnosis data in 2008. After eliminating codes whose exact mappings were already available in UMLS, the remainder were mapped manually with software assistance.

RESULTS

Of the 2194 codes, 784 (35.7%) required manual mapping. 435 of these represented concept types documented in SNOMED as deprecated: these included the qualifying phrases such as 'not elsewhere classified'. A third of the codes were composite, requiring multiple SNOMED code to map. Representing 45 composite concepts required introducing disjunction ('or') or set-difference ('without') operators, which are not currently defined in SNOMED. Only 47% of the concepts required for composition were present in the clinical core subset. Search of SNOMED for the correct concepts often required extensive application of knowledge of both English and medical synonymy.

CONCLUSION

Strategies to deal with legacy ICD data must address the issue of codes created by non-taxonomist users. The NLM core subset possibly needs augmentation with concepts from certain SNOMED hierarchies, notably qualifiers, body structures, substances/products and organisms. Concept-matching software needs to utilize query expansion strategies, but these may be effective in production settings only if a large but non-redundant SNOMED subset that minimizes the proportion of extensively pre-coordinated concepts is also available.

Determining correspondences between high-frequency MedDRA concepts and SNOMED: a case study

BACKGROUND

The Systematic Nomenclature of Medicine Clinical Terms (SNOMED CT) is being advocated as the foundation for encoding clinical documentation. While the electronic medical record is likely to play a critical role in pharmacovigilance - the detection of adverse events due to medications - classification and reporting of Adverse Events is currently based on the Medical Dictionary of Regulatory Activities (MedDRA). Complete and high-quality MedDRA-to-SNOMED CT mappings can therefore facilitate pharmacovigilance. The existing mappings, as determined through the Unified Medical Language System (UMLS), are partial, and record only one-to-one correspondences even though SNOMED CT can be used compositionally. Efforts to map previously unmapped MedDRA concepts would be most productive if focused on concepts that occur frequently in actual adverse event data. We aimed to identify aspects of MedDRA that complicate mapping to SNOMED CT, determine pattern in unmapped high-frequency MedDRA concepts, and to identify types of integration errors in the mapping of MedDRA to UMLS.

METHODS

Using one years' data from the US Federal Drug Administrations Adverse Event Reporting System, we identified MedDRA preferred terms that collectively accounted for 95% of both Adverse Events and Therapeutic Indications records. After eliminating those already mapping to SNOMED CT, we attempted to map the remaining 645 Adverse-Event and 141 Therapeutic-Indications preferred terms with software assistance.

RESULTS

All but 46 Adverse-Event and 7 Therapeutic-Indications preferred terms could be composed using SNOMED CT concepts: none of these required more than 3 SNOMED CT concepts to compose. We describe the common composition patterns in the paper. About 30% of both Adverse-Event and Therapeutic-Indications Preferred Terms corresponded to single SNOMED CT concepts: the correspondence was detectable by human inspection but had been missed during the integration process, which had created duplicated concepts in UMLS.

CONCLUSIONS

Identification of composite mapping patterns, and the types of errors that occur in the MedDRA content within UMLS, can focus larger-scale efforts on improving the quality of such mappings, which may assist in the creation of an adverse-events ontology.

A review of auditing methods applied to the content of controlled biomedical terminologies

Although controlled biomedical terminologies have been with us for centuries, it is only in the last couple of decades that close attention has been paid to the quality of these terminologies. The result of this attention has been the development of auditing methods that apply formal methods to assessing whether terminologies are complete and accurate. We have performed an extensive literature review to identify published descriptions of these methods and have created a framework for characterizing them. The framework considers manual, systematic and heuristic methods that use knowledge (within or external to the terminology) to measure quality factors of different aspects of the terminology content (terms, semantic classification, and semantic relationships). The quality factors examined included concept orientation, consistency, non-redundancy, soundness and comprehensive coverage. We reviewed 130 studies that were retrieved based on keyword search on publications in PubMed, and present our assessment of how they fit into our framework. We also identify which terminologies have been audited with the methods and provide examples to illustrate each part of the framework.

Adoption and perceptions of electronic health record systems by ophthalmologists: an American Academy of Ophthalmology survey

OBJECTIVE

To assess the current state of electronic health record (EHR) use by ophthalmologists, including adoption rate and user satisfaction.

DESIGN

Population-based, cross-sectional study.

PARTICIPANTS

A total of 592 members of the American Academy of Ophthalmology (AAO) participated.

METHODS

A total of 3796 AAO members were randomly selected on the basis of geography and solicited to participate in a study of EHR adoption. Among those solicited, 392 members completed a web-based version of the survey and 200 members completed a telephone-based version. The survey included sections assessing the current level of EHR adoption, the value of various EHR features, the practice demographics, and, for participants with an EHR, the details of their system. Responses were collected and analyzed using univariate statistical tests.

MAIN OUTCOME MEASURES

Current adoption rate of EHRs, user satisfaction with EHRs, and importance of various EHR features to both users and nonusers.

RESULTS

Overall, 12% of the practices surveyed had already implemented an EHR, 7% were in the process of doing so, and another 10% had plans to do so within 12 months. Both EHR users and nonusers rated the same EHR features as having the most value to their practices, and the 2 groups rated options for simplifying the EHR selection process similarly. Among those with an EHR in their practice, 69% were satisfied or extremely satisfied with their system, 64% reported increased or stable overall productivity, 51% reported decreased or stable overall costs, and 76% would recommend an EHR to a fellow ophthalmologist.

CONCLUSIONS

The adoption rate of EHRs by ophthalmology practices is low but comparable to that seen in other specialties. The satisfaction of those ophthalmologists already using an EHR is high. Because EHRs are part of the rapidly changing health information technology marketplace, the AAO Medical Information Technology Committee is planning to update these results on a regular basis.

Standardized terminology for aesthetic ophthalmic plastic surgery

PURPOSE

To determine whether existing Systemized Nomenclature of Medicine (SNOMED) terminology adequately describes aesthetic concepts commonly encountered in the oculoplastic and facial plastic surgery setting.

METHODS

This was a noncomparative case series. A panel of three oculoplastic surgeons compiled a list of unique concepts describing specialized aesthetic terminology commonly encountered in the oculoplastic and facial plastic surgery setting, with a specific focus on anatomic structures and descriptive findings. A standard electronic browser was used to manually search for the existence of equivalent matching concepts in SNOMED. A quality of match score from 1 to 3 was used with values of (1) no match, (2) partial match, and (3) equivalent match.

RESULTS

An assessment of the existing aesthetic terminology revealed that a majority of concepts were not represented. Of 62 total concepts, 68% had no match, 13% had a partial match, and 19% had a complete match.

CONCLUSIONS

SNOMED coverage of aesthetic terminology was less than in previous studies examining content representation for other medical topics. Such findings underscore a need for further development and refinement of aesthetic content.

Use of SNOMED CT to represent clinical research data: a semantic characterization of data items on case report forms in vasculitis research

OBJECTIVE

To estimate the coverage provided by SNOMED CT for clinical research concepts represented by the items on case report forms (CRFs), as well as the semantic nature of those concepts relevant to post-coordination methods.

DESIGN

Convenience samples from CRFs developed by rheumatologists conducting several longitudinal, observational studies of vasculitis were selected. A total of 17 CRFs were used as the basis of analysis for this study, from which a total set of 616 (unique) items were identified. Each unique data item was classified as either a clinical finding or procedure. The items were coded by the presence and nature of SNOMED CT coverage and classified into semantic types by 2 coders.

MEASUREMENTS

Basic frequency analysis was conducted to determine levels of coverage provided by SNOMED CT. Estimates of coverage by various semantic characterizations were estimated.

RESULTS

Most of the core clinical concepts (88%) from these clinical research data items were covered by SNOMED CT; however, far fewer of the concepts were fully covered (that is, where all aspects of the CRF item could be represented completely without post-coordination; 23%). In addition, a large majority of the concepts (83%) required post-coordination, either to clarify context (e.g., time) or to better capture complex clinical concepts (e.g., disease-related findings). For just over one third of the sampled CRF data items, both types of post-coordination were necessary to fully represent the meaning of the item.

CONCLUSION

SNOMED CT appears well-suited for representing a variety of clinical concepts, yet is less suited for representing the full amount of information collected on CRFs.

Representation of Ophthalmology Concepts by Electronic Systems

OBJECTIVE

To assess intercoder agreement for ophthalmology concepts by 3 physician coders using 5 controlled terminologies (International Classification of Diseases 9, Clinical Modification [ICD9CM]; Current Procedural Terminology, fourth edition; Logical Observation Identifiers, Names, and Codes [LOINC]; Systematized Nomenclature of Medicine, Clinical Terms [SNOMED-CT]; and Medical Entities Dictionary).

DESIGN

Noncomparative case series.

PARTICIPANTS

Five complete ophthalmology case presentations selected from a publicly available journal.

METHODS

Each case was parsed into discrete concepts. Electronic or paper browsers were used independently by 3 physician coders to assign a code for every concept in each terminology. A match score representing adequacy of assignment for each concept was assigned on a 3-point scale (0, no match; 1, partial match; 2, complete match). For every concept, the level of intercoder agreement was determined by 2 methods: (1) based on exact code matching with assignment of complete agreement when all coders assigned the same code, partial agreement when 2 coders assigned the same code, and no agreement when all coders assigned different codes, and (2) based on manual review for semantic equivalence of all assigned codes by an independent ophthalmologist to classify intercoder agreement for each concept as complete agreement, partial agreement, or no agreement. Subsequently, intercoder agreement was calculated in the same manner for the subset of concepts judged to have adequate coverage by each terminology, based on receiving a match score of 2 by at least 2 of the 3 coders.

MAIN OUTCOME MEASURES

Intercoder agreement in each controlled terminology: complete, partial, or none.

RESULTS

Cases were parsed into 242 unique concepts. When all concepts were analyzed by manual review, the proportion of complete intercoder agreement ranged from 12% (LOINC) to 44% (SNOMED-CT), and the difference in intercoder agreement between LOINC and all other terminologies was statistically significant (P<0.004). When only concepts with adequate terminology were analyzed by manual review, the proportion of complete intercoder agreement ranged from 33% (LOINC) to 64% (ICD9CM), and there were no statistically significant differences in intercoder agreement among any pairs of terminologies.

CONCLUSIONS

The level of intercoder agreement for ophthalmic concepts in existing controlled medical terminologies is imperfect. Intercoder reproducibility is essential for accurate and consistent electronic representation of medical data.