Health reform: is routinely collected electronic information fit for purpose?

Evaluation of venous thromboembolism risk assessment models for hospital inpatients: the VTEAM evidence synthesis

Background

Pharmacological prophylaxis during hospital admission can reduce the risk of acquired blood clots (venous thromboembolism) but may cause complications, such as bleeding. Using a risk assessment model to predict the risk of blood clots could facilitate selection of patients for prophylaxis and optimise the balance of benefits, risks and costs.

Objectives

We aimed to identify validated risk assessment models and estimate their prognostic accuracy, evaluate the cost-effectiveness of different strategies for selecting hospitalised patients for prophylaxis, assess the feasibility of using efficient research methods and estimate key parameters for future research.

Design

We undertook a systematic review, decision-analytic modelling and observational cohort study conducted in accordance with Enhancing the QUAlity and Transparency Of health Research (EQUATOR) guidelines.

Setting

NHS hospitals, with primary data collection at four sites.

Participants

Medical and surgical hospital inpatients, excluding paediatric, critical care and pregnancy-related admissions.

Interventions

Prophylaxis for all patients, none and according to selected risk assessment models.

Main outcome measures

Model accuracy for predicting blood clots, lifetime costs and quality-adjusted life-years associated with alternative strategies, accuracy of efficient methods for identifying key outcomes and proportion of inpatients recommended prophylaxis using different models.

Results

We identified 24 validated risk assessment models, but low-quality heterogeneous data suggested weak accuracy for prediction of blood clots and generally high risk of bias in all studies. Decision-analytic modelling showed that pharmacological prophylaxis for all eligible is generally more cost-effective than model-based strategies for both medical and surgical inpatients, when valuing a quality-adjusted life-year at £20,000. The findings were more sensitive to uncertainties in the surgical population; strategies using risk assessment models were more cost-effective if the model was assumed to have a very high sensitivity, or the long-term risks of post-thrombotic complications were lower. Efficient methods using routine data did not accurately identify blood clots or bleeding events and several pre-specified feasibility criteria were not met. Theoretical prophylaxis rates across an inpatient cohort based on existing risk assessment models ranged from 13% to 91%.

Limitations

Existing studies may underestimate the accuracy of risk assessment models, leading to underestimation of their cost-effectiveness. The cost-effectiveness findings do not apply to patients with an increased risk of bleeding. Mechanical thromboprophylaxis options were excluded from the modelling. Primary data collection was predominately retrospective, risking case ascertainment bias.

Conclusions

Thromboprophylaxis for all patients appears to be generally more cost-effective than using a risk assessment model, in hospitalised patients at low risk of bleeding. To be cost-effective, any risk assessment model would need to be highly sensitive. Current evidence on risk assessment models is at high risk of bias and our findings should be interpreted in this context. We were unable to demonstrate the feasibility of using efficient methods to accurately detect relevant outcomes for future research.

Future work

Further research should evaluate routine prophylaxis strategies for all eligible hospitalised patients. Models that could accurately identify individuals at very low risk of blood clots (who could discontinue prophylaxis) warrant further evaluation.

Study registration

This study is registered as PROSPERO CRD42020165778 and Researchregistry5216.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: NIHR127454) and will be published in full in Health Technology Assessment; Vol. 28, No. 20. See the NIHR Funding and Awards website for further award information.

The suitability of government health information assets for secondary use in research: A fit-for-purpose analysis

BACKGROUND

Governments have responsibility for ensuring the quality and fitness-for-purpose of personal health data provided to them. While these health information assets are used widely for research, this secondary usage has received minimal research attention.

OBJECTIVE

This study aimed to investigate the secondary uses, in research, of population health and administrative datasets (information assets) of the Department of Health (DoH), Victoria, Australia. The objectives were to (i) identify research based on these datasets published between 2008 and 2020; (ii) describe the data quality studies published between 2008 and 2020 for each dataset and (iii) evaluate "fitness-for-purpose" of the published research.

METHOD

Using a modified scoping review, research publications from 2008 to 2020 based on information assets related to health service provision and containing person-level data were reviewed. Publications were summarised by data quality and purpose-categories based on a taxonomy of data use. Fitness-for-purpose was evaluated by comparing the publicly stated purpose(s) for which each information asset was collected, with the purpose(s) assigned to the published research.

RESULTS

Of the >1000 information assets, 28 were utilised in 756 publications: 54% were utilised for general research purposes, 14% for patient safety, 10% for quality of care and 39% included data quality-related publications. Almost 85% of publications used information assets that were fit-for-purpose.

CONCLUSION

The DoH information assets were used widely for secondary purposes, with the majority identified as fit-for-purpose. We recommend that data custodians, including governments, provide information on data quality and transparency on data use of their health information assets.

Digital Health Data Quality Issues: Systematic Review

BACKGROUND

The promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making. However, the ability to effectively harness data has proven elusive, largely because of the quality of the data captured. Despite the importance of data quality (DQ), an agreed-upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships among the dimensions or their resultant impact.

OBJECTIVE

The aim of this study was to develop a consolidated digital health DQ dimension and outcome (DQ-DO) framework to provide insights into 3 research questions: What are the dimensions of digital health DQ? How are the dimensions of digital health DQ related? and What are the impacts of digital health DQ?

METHODS

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focusing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved and inductively analyzed to identify digital health DQ dimensions and outcomes. The inductive analysis was performed through open coding, constant comparison, and card sorting with subject matter experts to identify digital health DQ dimensions and digital health DQ outcomes. Subsequently, a computer-assisted analysis was performed and verified by DQ experts to identify the interrelationships among the DQ dimensions and relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ-DO framework.

RESULTS

The digital health DQ-DO framework consists of 6 dimensions of DQ, namely accessibility, accuracy, completeness, consistency, contextual validity, and currency; interrelationships among the dimensions of digital health DQ, with consistency being the most influential dimension impacting all other digital health DQ dimensions; 5 digital health DQ outcomes, namely clinical, clinician, research-related, business process, and organizational outcomes; and relationships between the digital health DQ dimensions and DQ outcomes, with the consistency and accessibility dimensions impacting all DQ outcomes.

CONCLUSIONS

The DQ-DO framework developed in this study demonstrates the complexity of digital health DQ and the necessity for reducing digital health DQ issues. The framework further provides health care executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritize which DQ-related problems to tackle first.

Accuracy of efficient data methods to determine the incidence of hospital-acquired thrombosis and major bleeding in medical and surgical inpatients: a multicentre observational cohort study in four UK hospitals

OBJECTIVES

We evaluated the accuracy of using routine health service data to identify hospital-acquired thrombosis (HAT) and major bleeding events (MBE) compared with a reference standard of case note review.

DESIGN

A multicentre observational cohort study.

SETTING

Four acute hospitals in the UK.

PARTICIPANTS

A consecutive unselective cohort of general medical and surgical patients requiring hospitalisation for a period of >24 hours during the calendar year 2021. We excluded paediatric, obstetric and critical care patients due to differential risk profiles.

INTERVENTIONS

We compared preidentified sources of routinely collected information (using hospital coding data and local contractually mandated thrombosis datasets) to data extracted from case notes using a predesigned workflow methodology.

PRIMARY AND SECONDARY OUTCOME MEASURES

We defined HAT as objectively confirmed venous thromboembolism occurring during hospital stay or within 90 days of discharge and MBE as per international consensus.

RESULTS

We were able to source all necessary routinely collected outcome data for 87% of 2008 case episodes reviewed. The sensitivity of hospital coding data (International Classification of Diseases 10th Revision, ICD-10) for the diagnosis of HAT and MBE was 62% (95% CI, 54 to 69) and 38% (95% CI, 27 to 50), respectively. Sensitivity improved to 81% (95% CI, 75 to 87) when using local thrombosis data sets.

CONCLUSIONS

Using routinely collected data appeared to miss a substantial proportion of outcome events, when compared with case note review. Our study suggests that currently available routine data collection methods in the UK are inadequate to support efficient study designs in venous thromboembolism research.

TRIAL REGISTRATION NUMBER

NIHR127454.

Data quality assessment in emergency medical services: an objective approach

BACKGROUND

In emergency medical services, high quality data are of great importance for patient care. Due to the unique nature of this type of services, the purpose of this study was to assess data quality in emergency medical services using an objective approach.

METHODS

This was a retrospective quantitative study conducted in 2019. The research sample included the emergency medical records of patients who referred to three emergency departments by the pre-hospital emergency care services (n = 384). Initially a checklist was designed based on the data elements of the triage form, pre-hospital emergency care form, and emergency medical records. Then, data completeness, accuracy and timeliness were assessed.

RESULTS

Data completeness in the triage form, pre-hospital emergency care form, and emergency medical records was 52.3%, 70% and 57.3%, respectively. Regarding data accuracy, most of the data elements were consistent. Measuring data timeliness showed that in some cases, paper-based ordering and computer-based data entry was not sequential.

CONCLUSION

Data quality in emergency medical services was not satisfactory and there were some weaknesses in the documentation processes. The results of this study can inform the clinical and administrative staff to pay more attentions to these weaknesses and plan for data quality improvement.

Consultation analysis: use of free text versus coded text

General practice in the United Kingdom has been using electronic health records for over two decades, but coding clinical information remains poor. Lack of interest and training are considerable barriers preventing code use levels improvement. Tailored training could be the way forward, to break barriers in the uptake of coding; to do so it is paramount to understand coding use of the particular clinicians, to recognise their needs. It should be possible to easily assess text quantity and quality in medical consultations. A tool to measure these parameters, which could be used to tailor training needs and assess change, is demonstrated. The tool is presented and a preliminary study using a randomised sample of five recent consultations from thirteen different clinicians is used as an example. The tool, based on using a word processor and a spread-sheet, allowed quantitative analysis among clinicians while word clouds permitted a qualitative comparison between coded and free text. The average amount of free text per consultation was 68.2 words, (ranging from 25.4 and 130.2 among clinicians); an average of 6% of the text was coded (ranging from 0 to 13%). Patterns among clinicians could be identified. Using Word cloud, a different text use was demonstrated depending on its purpose. Some free text could be turned into code but nomenclature probably prevented some of the codings, like the expression of time. This proof of concept demonstrated that it is possible to calculate what percentage of consultations are coded and what codes are used. This allowed understanding clinicians’ preferences; training needs and gaps in nomenclature.

Extending an open-source tool to measure data quality: case report on Observational Health Data Science and Informatics (OHDSI)

Introduction

As the health system seeks to leverage large-scale data to inform population outcomes, the informatics community is developing tools for analysing these data. To support data quality assessment within such a tool, we extended the open-source software Observational Health Data Sciences and Informatics (OHDSI) to incorporate new functions useful for population health.

Methods

We developed and tested methods to measure the completeness, timeliness and entropy of information. The new data quality methods were applied to over 100 million clinical messages received from emergency department information systems for use in public health syndromic surveillance systems.

Discussion

While completeness and entropy methods were implemented by the OHDSI community, timeliness was not adopted as its context did not fit with the existing OHDSI domains. The case report examines the process and reasons for acceptance and rejection of ideas proposed to an open-source community like OHDSI.

Linking observational data from general practice, hospital admissions and diabetes clinic databases: can it be used to predict hospital admission?

BACKGROUND

Linking process of care data from general practice (GP) and hospital data may provide more information about the risk of hospital admission and re-admission for people with type-2 diabetes mellitus (T2DM). This study aimed to extract and link data from a hospital, a diabetes clinic (DC). A second aim was to determine whether the data could be used to predict hospital admission for people with T2DM.

METHODS

Data were extracted using the GRHANITE™ extraction and linkage tool. The data from nine GPs and the DC included data from the two years prior to the hospital admission. The date of the first hospital admission for patients with one or more admissions was the index admission. For those patients without an admission, the census date 31/03/2014 was used in all outputs requiring results prior to an admission. Readmission was any admission following the index admission. The data were summarised to provide a comparison between two groups of patients: 1) Patients with a diagnosis of T2DM who had been treated at a GP and had a hospital admission and 2) Patients with a diagnosis of T2DM who had been treated at a GP and did not have a hospital admission.

RESULTS

Data were extracted for 161,575 patients from the three data sources, 644 patients with T2DM had data linked between the GPs and the hospital. Of these, 170 also had data linked with the DC. Combining the data from the different data sources improved the overall data quality for some attributes particularly those attributes that were recorded consistently in the hospital admission data. The results from the modelling to predict hospital admission were plausible given the issues with data completeness.

CONCLUSION

This project has established the methodology (tools and processes) to extract, link, aggregate and analyse data from general practices, hospital admission data and DC data. This study methodology involved the establishment of a comparator/control group from the same sites to compare and contrast the predictors of admission, addressing a limitation of most published risk stratification and admission prediction studies. Data completeness needs to be improved for this to be useful to predict hospital admissions.

A Review of Data Quality Assessment in Emergency Medical Services

INTRODUCTION

Data quality is an important issue in emergency medicine. The unique characteristics of emergency care services, such as high turn-over and the speed of work may increase the possibility of making errors in the related settings. Therefore, regular data quality assessment is necessary to avoid the consequences of low quality data. This study aimed to identify the main dimensions of data quality which had been assessed, the assessment approaches, and generally, the status of data quality in the emergency medical services.

METHODS

The review was conducted in 2016. Related articles were identified by searching databases, including Scopus, Science Direct, PubMed and Web of Science. All of the review and research papers related to data quality assessment in the emergency care services and published between 2000 and 2015 (n=34) were included in the study.

RESULTS

The findings showed that the five dimensions of data quality; namely, data completeness, accuracy, consistency, accessibility, and timeliness had been investigated in the field of emergency medical services. Regarding the assessment methods, quantitative research methods were used more than the qualitative or the mixed methods. Overall, the results of these studies showed that data completeness and data accuracy requires more attention to be improved.

CONCLUSION

In the future studies, choosing a clear and a consistent definition of data quality is required. Moreover, the use of qualitative research methods or the mixed methods is suggested, as data users' perspectives can provide a broader picture of the reasons for poor quality data.

Clinical Informatics Researcher's Desiderata for the Data Content of the Next Generation Electronic Health Record

OBJECTIVE

Clinical informatics researchers depend on the availability of high-quality data from the electronic health record (EHR) to design and implement new methods and systems for clinical practice and research. However, these data are frequently unavailable or present in a format that requires substantial revision. This article reports the results of a review of informatics literature published from 2010 to 2016 that addresses these issues by identifying categories of data content that might be included or revised in the EHR.

MATERIALS AND METHODS

We used an iterative review process on 1,215 biomedical informatics research articles. We placed them into generic categories, reviewed and refined the categories, and then assigned additional articles, for a total of three iterations.

RESULTS

Our process identified eight categories of data content issues: Adverse Events, Clinician Cognitive Processes, Data Standards Creation and Data Communication, Genomics, Medication List Data Capture, Patient Preferences, Patient-reported Data, and Phenotyping.

DISCUSSION

These categories summarize discussions in biomedical informatics literature that concern data content issues restricting clinical informatics research. These barriers to research result from data that are either absent from the EHR or are inadequate (e.g., in narrative text form) for the downstream applications of the data. In light of these categories, we discuss changes to EHR data storage that should be considered in the redesign of EHRs, to promote continued innovation in clinical informatics.

CONCLUSION

Based on published literature of clinical informaticians' reuse of EHR data, we characterize eight types of data content that, if included in the next generation of EHRs, would find immediate application in advanced informatics tools and techniques.

A review of medical terminology standards and structured reporting

Much effort has been invested in standardizing medical terminology for representation of medical knowledge, storage in electronic medical records, retrieval, reuse for evidence-based decision making, and for efficient messaging between users. We only focus on those efforts related to the representation of clinical medical knowledge required for capturing diagnoses and findings from a wide range of general to specialty clinical perspectives (e.g., internists to pathologists). Standardized medical terminology and the usage of structured reporting have been shown to improve the usage of medical information in secondary activities, such as research, public health, and case studies. The impact of standardization and structured reporting is not limited to secondary activities; standardization has been shown to have a direct impact on patient healthcare.

Making electronic health records support quality management: A narrative review

BACKGROUND

Since the 1990s many hospitals in the OECD countries have introduced electronic health record (EHR) systems. A number of studies have examined the factors impinging on EHR implementation. Others have studied the clinical efficacy of EHR. However, only few studies have explored the (intermediary) factors that make EHR systems conducive to quality management (QM).

OBJECTIVE

Undertake a narrative review of existing studies in order to identify and discuss the factors conducive to making EHR support three dimensions of QM: clinical outcomes, managerial monitoring and cost-effectiveness.

METHOD

A narrative review of Web of Science, Cochrane, EBSCO, ProQuest, Scopus and three Nordic research databases.

LIMITATION

most studies do not specify the type of EHR examined.

RESULTS

39 studies were identified for analysis. 10 factors were found to be conducive to make EHR support QM. However, the contribution of EHR to the three specific dimensions of QM varied substantially. Most studies (29) included clinical outcomes. However, only half of these reported EHR to have a positive impact. Almost all the studies (36) dealt with the ability of EHR to enhance managerial monitoring of clinical activities, the far majority of which showed a positive relationship. Finally, only five dealt with cost-effectiveness of which two found positive effects.

DISCUSSION AND CONCLUSION

The findings resonates well with previous reviews, though two factors making EHR support QM seem new, namely: political goals and strategies, and integration of guidelines for clinical conduct. Lacking EHR type specification and diversity in study method imply that there is a strong need for further research on the factors that may make EHR may support QM.

Integrated and consumer-directed care: a necessary paradigm shift for rural chronic ill health

Chronic ill health has recently emerged as the most important health issue on a global scale. Rural communities are disproportionally affected by chronic ill health. Many health systems are centred on the management of acute conditions and are often poorly equipped to deal with chronic ill health. Cardiovascular disease (CVD) is one of the most prominent chronic ill health conditions and the principal cause of mortality worldwide. In this paper, CVD is used as an example to demonstrate the disparity between rural and urban experience of chronic ill health, access to medical care and clinical outcomes. Advances have been made to address chronic ill health through improving self-management strategies, health literacy and access to medical services. However, given the higher incidence of chronic health conditions and poorer clinical outcomes in rural communities, it is imperative that integrated health care emphasises greater collaboration between services. It is also vital that rural GPs are better supported to work with their patients, and that they use consumer-directed approaches to empower patients to direct and coordinate their own care.

Data quality of electronic medical records in Manitoba: do problem lists accurately reflect chronic disease billing diagnoses?

OBJECTIVE

To determine problem list completeness related to chronic diseases in electronic medical records (EMRs) and explore clinic and physician factors influencing completeness.

METHODS

A retrospective analysis of primary care EMR data quality related to seven chronic diseases (hypertension, diabetes, asthma, congestive heart failure, coronary artery disease, hypothyroidism, and chronic obstructive pulmonary disorder) in Manitoba, Canada. We included 119 practices in 18 primary care clinics across urban and rural Manitoba. The main outcome measure was EMR problem list completeness. Completeness was measured by comparing the number of EMR-documented diagnoses to the number of billings associated with each disease. We calculated odds ratios for the effect of clinic patient load and salary type on EMR problem list completeness of the 7 chronic diseases.

RESULTS

Completeness of EMR problem list for each disease varied widely among clinics. Factors that significantly affected EMR problem list completeness included the primary care provider, the patient load, and the clinic's funding and organization model (ie, salaried, fee-for-service, or residency training clinics). Average rates of completeness were: hypertension, 72%; diabetes, 80%; hypothyroidism, 63%; asthma, 56%; chronic obstructive pulmonary disorder, 43%; congestive heart failure, 54%; and coronary artery disease, 64%.

CONCLUSION

This study demonstrates the high variability but generally low quality of problem lists (health condition records) related to 7 common chronic diseases in EMRs. There are systematic physician- and clinic-level factors associated with low data quality completeness. This information may be useful to support improvement in EMR data quality in primary care.

Shared Longitudinal Health Records for Clinical and Population Health

The ability of a health information exchange to consolidate information, collected in multiple, disparate information systems, into a single, person-centric health record can provide a comprehensive and longitudinal representation of an individual’s medical history. Shared, longitudinal health records can be leveraged to enhance the delivery of individual clinical care and provide opportunities to improve health outcomes at the population level. This chapter will describe the clinical benefits imparted by the shared health record (SHR) component of the OpenHIE infrastructure. It will also characterize the potential population health benefits of the aggregate level data contained and distributed by the Health Management Information System component of OpenHIE. The chapter will further discuss the implementation of these systems.

By the end of the chapter, the reader should be able to:●

Identify and describe the differences among an electronic medical record, electronic health record, and a shared heath record.

●

Explain the role of a shared health record in a health information exchange.

●

List and describe the components of a shared health record.

●

Discuss the role and benefits of a health management information system within a health information exchange.

●

Define a population health indicator.

●

Identify and describe application domains for a health management information system.

●

Define a database management system.

●

Compare the implications of implementing a shared health record using an electronic health record system versus a database management system.

●

Discuss emerging trends likely to shape the evolution of shared health records and health management information systems.

Are family physicians comprehensively using electronic medical records such that the data can be used for secondary purposes? A Canadian perspective

Background

With the introduction and implementation of a variety of government programs and policies to encourage adoption of electronic medical records (EMRs), EMRs are being increasingly adopted in North America. We sought to evaluate the completeness of a variety of EMR fields to determine if family physicians were comprehensively using their EMRs and the suitability of use of the data for secondary purposes in Ontario, Canada.

Methods

We examined EMR data from a convenience sample of family physicians distributed throughout Ontario within the Electronic Medical Record Administrative data Linked Database (EMRALD) as extracted in the summer of 2012. We identified all physicians with at least one year of EMR use. Measures were developed and rates of physician documentation of clinical encounters, electronic prescriptions, laboratory tests, blood pressure and weight, referrals, consultation letters, and all fields in the cumulative patient profile were calculated as a function of physician and patient time since starting on the EMR.

Results

Of the 167 physicians with at least one year of EMR use, we identified 186,237 patients. Overall, the fields with the highest level of completeness were for visit documentations and prescriptions (>70 %). Improvements were observed with increasing trends of completeness overtime for almost all EMR fields according to increasing physician time on EMR. Assessment of the influence of patient time on EMR demonstrated an increasing likelihood of the population of EMR fields overtime, with the largest improvements occurring between the first and second years.

Conclusions

All of the data fields examined appear to be reasonably complete within the first year of adoption with the biggest increase occurring the first to second year. Using all of the basic functions of the EMR appears to be occurring in the current environment of EMR adoption in Ontario. Thus the data appears to be suitable for secondary use.

Ontological specification of quality of chronic disease data in EHRs to support decision analytics: a realist review

This systematic review examined the current state of conceptualization and specification of data quality and the role of ontology based approaches to develop data quality based on "fitness for purpose" within the health context. A literature review was conducted of all English language studies, from January 2000-March 2013, which addressed data/information quality, fitness for purpose of data, used and implemented ontology-based approaches. Included papers were critically appraised with a "context-mechanism-impacts/outcomes" overlay. We screened 315 papers, excluded 36 duplicates, 182 on abstract review and 46 on full-text review; leaving 52 papers for critical appraisal. Six papers conceptualized data quality within the "fitness for purpose" definition. While most agree with a multidimensional definition of DQ, there is little consensus on a conceptual framework. We found no reports of systematic and comprehensive ontological approaches to DQ based on fitness for purpose or use. However, 16 papers used ontology-specified implementations in DQ improvement, with most of them focusing on some dimensions of DQ such as completeness, accuracy, correctness, consistency and timeliness. The majority of papers described the processes of the development of DQ in various information systems. There were few evaluative studies, including any comparing ontological with non-ontological approaches, on the assessment of clinical data quality and the performance of the application.

Literature review of SNOMED CT use

OBJECTIVE

The aim of this paper is to report on the use of the systematised nomenclature of medicine clinical terms (SNOMED CT) by providing an overview of published papers.

METHODS

Published papers on SNOMED CT between 2001 and 2012 were identified using PubMed and Embase databases using the keywords 'systematised nomenclature of medicine' and 'SNOMED CT'. For each paper the following characteristics were retrieved: SNOMED CT focus category (ie, indeterminate, theoretical, pre-development/design, implementation and evaluation/commodity), usage category (eg, prospective content coverage, used to classify or code in a study), medical domain and country.

RESULTS

Our search strategy identified 488 papers. A comparison between the papers published between 2001-6 and 2007-12 showed an increase in every SNOMED CT focus category. The number of papers classified as 'theoretical' increased from 46 to 78, 'pre-development/design' increased from 61 to 173 and 'implementation' increased from 10 to 34. Papers classified as 'evaluation/commodity' only started to appear from 2010.

CONCLUSIONS

The majority of studies focused on 'theoretical' and 'pre-development/design'. This is still encouraging as SNOMED CT is being harmonized with other standardized terminologies and is being evaluated to determine the content coverage of local terms, which is usually one of the first steps towards adoption. Most implementations are not published in the scientific literature, requiring a look beyond the scientific literature to gain insights into SNOMED CT implementations.