Data quality and completeness in a web stroke registry as the basis for data and process mining

Design of a bi-directional methodology for automated assessment of compliance to continuous application of clinical guidelines, and its evaluation in the type 2 diabetes domain

We introduce a new approach for automated guideline-based-care quality assessment, the bidirectional knowledge-based assessment of compliance (BiKBAC) method, and the DiscovErr system, which implements it. Our methodology compares the guideline's Asbru-based formal representation, including its intentions, with the longitudinal medical record, using a top-down and bottom-up approach. Partial matches are resolved using fuzzy temporal logic. The system was evaluated in the type 2 Diabetes management domain, comparing it to three expert clinicians, including two diabetes experts. The system and the experts commented on the management of 10 patients, randomly selected from 2,000 diabetes patients. On average, each record spanned 5.23 years; the data included 1,584 medical transactions. The system provided 279 comments. The experts made 181 different unique comments. The completeness (recall) of the system was 91% when the gold standard was comments made by at least two of the three experts, and 98%, compared to comments made by all three experts. The experts also assessed all of the 114 medication-therapy-related comments, and a random 35% of the 165 tests-and-monitoring-related comments. The system's correctness (precision) was 81%, compared to comments judged as correct by both diabetes experts, and 91%, compared to comments judged as correct by one diabetes expert and at least as partially correct by the other. 89% of the comments were judged as important by both diabetes experts, 8% were judged as important by one expert, and 3% were judged as less important by both experts. Adding the validated system comments to the experts' comments, the completeness scores of the experts were 75%, 60%, and 55%; the expert correctness scores were respectively 99%, 91%, and 88%. Thus, the system could be ranked first in completeness and second in correctness. We conclude that systems such as DiscovErr can effectively assess the quality of continuous guideline-based care.

Process mining and customer journey mapping in healthcare: Enhancing patient-centred care in stroke rehabilitation

Background

Patient-centred care and enhancing patient experience is a priority across Australia. Stroke rehabilitation has multiple consumer touchpoints that would benefit from a better understanding of customer journeys, subsequently impacting better patient-centred care, and contributing to process improvements and better patient outcomes. Customer journey mapping through process mining extracts process data from event logs in existing information systems discovering patient journeys, which can be utilized to monitor guideline compliance and uncover nonconformance.

Methodology

Utilizing process mining and variant analysis, customer journey maps were developed for 130 stroke rehabilitation patients from referral to discharge. In total, 168 cases from the Australasian Rehabilitation Outcomes Centre dataset were matched with 6291 cases from inpatient stroke data. Variants were explored for age, gender, outcome measures, length of stay and functional independence measure (FIM) change.

Results

The study illustrated the process, process variants and patient journey map in stroke rehabilitation. Process characteristics of stroke rehabilitation patients were extracted and represented utilizing process mining and results highlighted process variation, attributes, touchpoints and timestamps across stroke rehabilitation patient journeys categorized by patient demographics and outcome variables. Patients demonstrated a mean and median duration of 49.5 days and 44 days, respectively, across the patient journeys. Nine variants were discovered, with 78.46% (n = 102) of patients following the expected sequence of activities in their stroke rehabilitation patient journey. Relationships involving age, gender, length of stay and FIM change along the patient journeys were evident, with four cases experiencing stroke rehabilitation journeys of more than 100 days, warranting further investigation.

Conclusion

Process mining can be utilized to visualize and analyse patient journeys and identify gaps in service quality, thus contributing to better patient-centred care and improved patient outcomes and experiences in stroke rehabilitation.

Digital Health Data Quality Issues: Systematic Review

BACKGROUND

The promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making. However, the ability to effectively harness data has proven elusive, largely because of the quality of the data captured. Despite the importance of data quality (DQ), an agreed-upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships among the dimensions or their resultant impact.

OBJECTIVE

The aim of this study was to develop a consolidated digital health DQ dimension and outcome (DQ-DO) framework to provide insights into 3 research questions: What are the dimensions of digital health DQ? How are the dimensions of digital health DQ related? and What are the impacts of digital health DQ?

METHODS

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focusing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved and inductively analyzed to identify digital health DQ dimensions and outcomes. The inductive analysis was performed through open coding, constant comparison, and card sorting with subject matter experts to identify digital health DQ dimensions and digital health DQ outcomes. Subsequently, a computer-assisted analysis was performed and verified by DQ experts to identify the interrelationships among the DQ dimensions and relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ-DO framework.

RESULTS

The digital health DQ-DO framework consists of 6 dimensions of DQ, namely accessibility, accuracy, completeness, consistency, contextual validity, and currency; interrelationships among the dimensions of digital health DQ, with consistency being the most influential dimension impacting all other digital health DQ dimensions; 5 digital health DQ outcomes, namely clinical, clinician, research-related, business process, and organizational outcomes; and relationships between the digital health DQ dimensions and DQ outcomes, with the consistency and accessibility dimensions impacting all DQ outcomes.

CONCLUSIONS

The DQ-DO framework developed in this study demonstrates the complexity of digital health DQ and the necessity for reducing digital health DQ issues. The framework further provides health care executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritize which DQ-related problems to tackle first.

The impact of data quality assurance and control solutions on the completeness, accuracy, and consistency of data in a national spinal cord injury registry of Iran (NSCIR-IR)

STUDY DESIGN

Descriptive study.

OBJECTIVE

This study aimed to develop and evaluate a systematic arrangement for improvement and monitoring of data quality of the National Spinal Cord (and Column) Injury Registry of Iran (NSCIR-IR)-a multicenter hospital-based registry.

SETTING

SCI community in Iran.

METHODS

Quality assurance and quality control were the primary objectives in improving overall quality of data that were considered in designing a paper-based and computerized case report. To prevent incorrect data entry, we implemented several validation algorithms, including 70 semantic rules, 18 syntactic rules, seven temporal rules, and 13 rules for acceptable value range. Qualified and trained staff members were also employed to review and identify any defect, inaccuracy, or inconsistency in the data to improve data quality. A set of functions were implemented in the software to cross-validate, and feedback on data was provided by reviewers and registrars.

RESULTS

Socio-demographic data items were 100% complete, except for national ID and education level, which were 97% and 92.3% complete, respectively. Completeness of admission data and emergency medical services data were 100% except for arrival and transfer time (99.4%) and oxygen saturation (48.9%). Evaluation of data received from two centers located in Tehran proved to be 100% accurate following validation by quality reviewers. All data was also found to be 100% consistent.

CONCLUSIONS

This approach to quality assurance and consistency validation proved to be effective. Our solutions resulted in a significant decrease in the number of missing data.

Electronic healthcare records and data quality

This paper is based on Victoria Charnock's MA dissertation carried out as part of her Masters in Leadership and Management in Healthcare at the University of Salford and supervised by Professor Hardiker. A review of current literature was conducted to provide a robust and dimensional definition of data quality in the field of health care. This was used as the basis on which to assess the effect that electronic health care records has had in practice, specifically on data quality and according to the dimensions of accuracy, completeness and use of data. All of the papers reviewed referred to the importance of accuracy and completeness, identifying the advantages of electronic health records in their use of standardized data entry controls. Drawing on the third dimension in the definition, use of data, the impact that system design may have on data quality and implications for staff training is further discussed and recommendations made. F.J.

A Holistic Clustering Methodology for Liver Transplantation Survival

BACKGROUND

Liver transplants account for a high number of procedures with major investments from all stakeholders involved; however, limited studies address liver transplant population heterogeneity pretransplant predictive of posttransplant survival.

OBJECTIVE

The aim of the study was to identify novel and meaningful patient clusters predictive of mortality that explains the heterogeneity of liver transplant population, taking a holistic approach.

METHODS

A retrospective cohort study of 344 adult patients who underwent liver transplantation between 2008 through 2014. Predictors were summarized severity scores for comorbidities and other suboptimal health states grouped into 11 body systems, the primary reason for transplantation, demographics/environmental factors, and Model for End Liver Disease score. Logistic regression was used to compute the severity scores, hierarchical clustering with weighted Euclidean distance for clustering, Lasso-penalized regression for characterizing the clusters, and Kaplan-Meier analysis to compare survival across the clusters.

RESULTS

Cluster 1 included patients with more severe circulatory problems. Cluster 2 represented older patients with more severe primary disease, whereas Cluster 3 contained healthiest patients. Clusters 4 and 5 represented patients with musculoskeletal (e.g., pain) and endocrine problems (e.g., malnutrition), respectively. There was a statistically significant difference for mortality between clusters (p < .001).

CONCLUSIONS

This study developed a novel methodology to address heterogeneous and high-dimensional liver transplant population characteristics in a single study predictive of survival. A holistic approach for data modeling and additional psychosocial risk factors has the potential to address holistically nursing challenges on liver transplant care and research.

Process Mining for Quality Improvement: Propositions for Practice and Research

OBJECTIVE

Process mining offers ways to discover patient flow, check how actual processes conform to a standard, and use data to enhance or improve processes. Process mining has been used in health care for about a decade, however, with limited focus on quality improvement. Hence, the aim of the article is to present how process mining can be used to support quality improvement, thereby bridging the gap between process mining and quality improvement.

METHOD

We have analyzed current literature to perform a comparison between process mining and process mapping.

RESULT

To better understand how process mining can be used for quality improvement we provide 2 examples. We have noted 4 limitations that must be overcome, which have been formulated as propositions for practice. We have also formulated 3 propositions for future research.

CONCLUSION

In summary, although process mapping is still valuable in quality improvement, we suggest increased focus on process mining. Process mining adds to quality improvement by providing a better understanding of processes in terms of uncovering (un)wanted variations as to obtain better system results.

The assessment of data quality issues for process mining in healthcare using Medical Information Mart for Intensive Care III, a freely available e-health record database

There is a growing body of literature on process mining in healthcare. Process mining of electronic health record systems could give benefit into better understanding of the actual processes happened in the patient treatment, from the event log of the hospital information system. Researchers report issues of data access approval, anonymisation constraints, and data quality. One solution to progress methodology development is to use a high-quality, freely available research dataset such as Medical Information Mart for Intensive Care III, a critical care database which contains the records of 46,520 intensive care unit patients over 12 years. Our article aims to (1) explore data quality issues for healthcare process mining using Medical Information Mart for Intensive Care III, (2) provide a structured assessment of Medical Information Mart for Intensive Care III data quality and challenge for process mining, and (3) provide a worked example of cancer treatment as a case study of process mining using Medical Information Mart for Intensive Care III to illustrate an approach and solution to data quality challenges. The electronic health record software was upgraded partway through the period over which data was collected and we use this event to explore the link between electronic health record system design and resulting process models.

Developing a Rural, Community-Based Registry for Cardiovascular Quality Improvement

BACKGROUND

Cardiovascular disease is one of the leading causes of death, yet most evidence is collected from small clinical trials or individual hospital providers. Achieving scalable data to enable quality improvements (QIs) remains a challenge. We investigate whether a registry that is shared by multiple providers and integrates data longitudinally could help drive QIs across a large rural geographic region.

METHODS

We describe a case study involving the development of an informatics infrastructure across the entire state of Wyoming. This rural, regional, community-based cardiovascular system of care involved all interventional hospitals in the state as well as all surrounding states. Data exchange was initiated between 36 hospitals, and 56 ambulance agencies, to a centralized registry for clinical analytics and QI for patients with acute myocardial infarction.

RESULTS

After 3 years, the registry maintained all documented acute myocardial infarctions across Wyoming. Median total ischemic time (time from patient's symptom onset to definitive treatment) had a 36.7% improvement during the program. Changes in quality for the rural community included reduction in overall treatment times, as well as enhanced training, standardized protocols, and community awareness. We also share key lessons learned.

CONCLUSIONS

Collaborative data registries for emergency cardiovascular care can help providers and communities measure and improve the quality of the care across regions.

An Environment for Guidelinebased Decision Support Systems for Outpatients Monitoring

Objectives: We propose an architecture for monitoring outpatients that relies on mobile technologies for acquiring data. The goal is to better control the onset of possible side effects between the scheduled visits at the clinic.

Methods: We analyze the architectural components required to ensure a high level of abstraction from data. Clinical practice guidelines were formalized with Alium, an authoring tool based on the PROforma language, using SNOMED-CT as a terminology standard. The Alium engine is accessible through a set of APIs that may be leveraged for implementing an application based on standard web technologies to be used by doctors at the clinic. Data sent by patients using mobile devices need to be complemented with those already available in the Electronic Health Record to generate personalized recommendations. Thus a middleware pursuing data abstraction is required. To comply with current standards, we adopted the HL7 Virtual Medical Record for Clinical Decision Support Logical Model, Release 2.

Results: The developed architecture for monitoring outpatients includes: (1) a guideline-based Decision Support System accessible through a web application that helps the doctors with prevention, diagnosis and treatment of therapy side effects; (2) an application for mobile devices, which allows patients to regularly send data to the clinic. In order to tailor the monitoring procedures to the specific patient, the Decision Support System also helps physicians with the configuration of the mobile application, suggesting the data to be collected and the associated collection frequency that may change over time, according to the individual patient’s conditions. A proof of concept has been developed with a system for monitoring the side effects of chemo-radiotherapy in head and neck cancer patients.

Conclusions: Our environment introduces two main innovation elements with respect to similar works available in the literature. First, in order to meet the specific patients’ needs, in our work the Decision Support System also helps the physicians in properly configuring the mobile application. Then the Decision Support System is also continuously fed by patient-reported outcomes.

Multiple Overimputation to Address Missing Data and Measurement Error: Application to HIV Treatment During Pregnancy and Pregnancy Outcomes

BACKGROUND

Investigations of the association of combination antiretroviral therapy (ART) with pregnancy outcomes often rely on routinely collected clinical data, which are prone to missing data and measurement error. Measurement error in gestational age may bias the relation between combination ART and gestational age-based outcomes.

METHODS

We demonstrate the use of multiple overimputation to address missing data and measurement error in gestational age. Using routinely collected clinical data from public health facilities in Lusaka, Zambia, we multiply imputed missing data and multiply overimputed observed values of gestational age. Poisson models with robust variance estimators were used to estimate risk ratios (RRs) for the associations of duration of combination ART with small for gestational age (SGA) and preterm birth. We compared results from a complete-case analysis, using multiple imputation to address missing data only and using multiple overimputation to address missing data and measurement error.

RESULTS

In the complete-case analysis, there was no evidence of an association between duration of combination ART and SGA or preterm birth. When we performed multiple overimputation, RRs for SGA moved past the null, but remained imprecise. For preterm birth, RRs for 9-32 weeks of combination ART moved away from the null as the variance due to measurement error increased.

CONCLUSION

When we used multiple overimputation to account for measurement error and missing data, we observed an increased risk of preterm birth with longer duration of combination ART. Future analyses examining associations between combination ART and pregnancy outcomes should consider using multiple overimputation to address measurement error in gestational age.

A Comprehensive Assessment Tool for Patient Registry Software Systems: The CIPROS Checklist

Background: Patient registries are an important instrument in medical research. Often their structure is complex and their implementation uses composite software systems to meet the wide spectrum of challenges.

Objectives: For the implementation of a registry, there is a wide range of commercial, open source, and self-developed systems available and a minimal standard for the critical appraisal of their architecture is needed.

Methods: We performed a systematic review of the literature to define a catalogue of relevant criteria to construct a minimal appraisal standard.

Results: The CIPROS list is developed based on 64 papers which were found by our systematic review. The list covers twelve sections and contains 72 items.

Conclusions: The CIPROS list supports developers to assess requirements on existing systems and strengthens the reporting of patient registry software system descriptions. It can be a first step to create standards for patient registry software system assessments.

Combining multivariate statistics and the think-aloud protocol to assess Human-Computer Interaction barriers in symptom checkers

Symptom checkers are software tools that allow users to submit a set of symptoms and receive advice related to them in the form of a diagnosis list, health information or triage. The heterogeneity of their potential users and the number of different components in their user interfaces can make testing with end-users unaffordable. We designed and executed a two-phase method to test the respiratory diseases module of the symptom checker Erdusyk. Phase I consisted of an online test with a large sample of users (n=53). In Phase I, users evaluated the system remotely and completed a questionnaire based on the Technology Acceptance Model. Principal Component Analysis was used to correlate each section of the interface with the questionnaire responses, thus identifying which areas of the user interface presented significant contributions to the technology acceptance. In the second phase, the think-aloud procedure was executed with a small number of samples (n=15), focusing on the areas with significant contributions to analyze the reasons for such contributions. Our method was used effectively to optimize the testing of symptom checker user interfaces. The method allowed kept the cost of testing at reasonable levels by restricting the use of the think-aloud procedure while still assuring a high amount of coverage. The main barriers detected in Erdusyk were related to problems understanding time repetition patterns, the selection of levels in scales to record intensities, navigation, the quantification of some symptom attributes, and the characteristics of the symptoms.

Personalization and Patient Involvement in Decision Support Systems: Current Trends

OBJECTIVES

This survey aims at highlighting the latest trends (2012-2014) on the development, use, and evaluation of Information and Communication Technologies (ICT) based decision support systems (DSSs) in medicine, with a particular focus on patient-centered and personalized care.

METHODS

We considered papers published on scientific journals, by querying PubMed and Web of ScienceTM. Included studies focused on the implementation or evaluation of ICT-based tools used in clinical practice. A separate search was performed on computerized physician order entry systems (CPOEs), since they are increasingly embedding patient-tailored decision support.

RESULTS

We found 73 papers on DSSs (53 on specific ICT tools) and 72 papers on CPOEs. Although decision support through the delivery of recommendations is frequent (28/53 papers), our review highlighted also DSSs only based on efficient information presentation (25/53). Patient participation in making decisions is still limited (9/53), and mostly focused on risk communication. The most represented medical area is cancer (12%). Policy makers are beginning to be included among stakeholders (6/73), but integration with hospital information systems is still low. Concerning knowledge representation/management issues, we identified a trend towards building inference engines on top of standard data models. Most of the tools (57%) underwent a formal assessment study, even if half of them aimed at evaluating usability and not effectiveness.

CONCLUSIONS

Overall, we have noticed interesting evolutions of medical DSSs to improve communication with the patient, consider the economic and organizational impact, and use standard models for knowledge representation. However, systems focusing on patient-centered care still do not seem to be available at large.

Safely Addressing Patients with Atrial Fibrillation to Early Anticoagulation after Acute Stroke

BACKGROUND

It has been widely reported that anticoagulants (ACs) are underused for primary and secondary prevention of ischemic stroke in patients with atrial fibrillation (AFib). Furthermore, precise evidence-based guidelines about the best timing for AC initiation after acute stroke are currently lacking.

METHODS AND RESULTS

In this retrospective, observational study, we analyzed prescription trends in AFib patients with acute ischemic stroke who were hospitalized in four neurologic stroke units of our region (Lombardia, Italy). In-hospital antithrombotic prescription was performed in highly heterogeneous patterns. A prestroke treatment with AC was the leading factor enhancing AC prescription during hospitalization. The other factors promoting AC were male gender, younger age, lower prestroke disability and stroke severity, and smaller stroke volumes. AFib subtype influenced AC prescription only in AC-naïve patients. Interestingly, Congestive heart failure, Hypertension, Age higher than 75 years, Diabetes, previous Stroke or TIA or thromboembolism, Vascular disease, Age 64-75 years, female Sex (CHA₂DS₂-VASc) and Hypertension, Abnormal renal and liver function, Stroke, Bleeding, Labile INRs, Elderly, Drugs and alcohol (HAS-BLED) scores were not associated with AC prescription. However, patients who were treated with AC, including early treatment (<48 hours), showed a low rate of bleeding.

CONCLUSIONS

Our findings potentially suggest that, although apparently neglecting the common risk stratification tools, our neurologists were able to select the more suitable candidates for prompt AC treatment. Further studies are needed to develop new scoring systems to aid ischemic and hemorrhagic risk estimation in the secondary prevention of stroke.

A New Paradigm to Analyze Data Completeness of Patient Data

BACKGROUND

There is a need to develop a tool that will measure data completeness of patient records using sophisticated statistical metrics. Patient data integrity is important in providing timely and appropriate care. Completeness is an important step, with an emphasis on understanding the complex relationships between data fields and their relative importance in delivering care. This tool will not only help understand where data problems are but also help uncover the underlying issues behind them.

OBJECTIVES

Develop a tool that can be used alongside a variety of health care database software packages to determine the completeness of individual patient records as well as aggregate patient records across health care centers and subpopulations.

METHODS

The methodology of this project is encapsulated within the Data Completeness Analysis Package (DCAP) tool, with the major components including concept mapping, CSV parsing, and statistical analysis.

RESULTS

The results from testing DCAP with Healthcare Cost and Utilization Project (HCUP) State Inpatient Database (SID) data show that this tool is successful in identifying relative data completeness at the patient, subpopulation, and database levels. These results also solidify a need for further analysis and call for hypothesis driven research to find underlying causes for data incompleteness.

CONCLUSION

DCAP examines patient records and generates statistics that can be used to determine the completeness of individual patient data as well as the general thoroughness of record keeping in a medical database. DCAP uses a component that is customized to the settings of the software package used for storing patient data as well as a Comma Separated Values (CSV) file parser to determine the appropriate measurements. DCAP itself is assessed through a proof of concept exercise using hypothetical data as well as available HCUP SID patient data.

Development of trigger-based semi-automated surveillance of ventilator-associated pneumonia and central line-associated bloodstream infections in a Dutch intensive care

Background

Availability of a patient data management system (PDMS) has created the opportunity to develop trigger-based electronic surveillance systems (ESSs). The aim was to evaluate a semi-automated trigger-based ESS for the detection of ventilator-associated pneumonia (VAP) and central line-associated blood stream infections (CLABSIs) in the intensive care.

Methods

Prospective comparison of surveillance was based on a semi-automated ESS with and without trigger. Components of the VAP/CLABSI definition served as triggers. These included the use of VAP/CLABSI-related antibiotics, the presence of mechanical ventilation or an intravenous central line, and the presence of specific clinical symptoms. Triggers were automatically fired by the PDMS. Chest X-rays and microbiology culture results were checked only on patient days with a positive trigger signal from the ESS. In traditional screening, no triggers were used; therefore, chest X-rays and culture results had to be screened for all patient days of all included patients. Patients with pneumonia at admission were excluded.

Results

A total of 553 patients were screened for VAP and CLABSI. The incidence of VAP was 3.3/1,000 ventilation days (13 VAP/3,927 mechanical ventilation days), and the incidence of CLABSI was 1.7/1,000 central line days (24 CLABSI/13.887 central line days). For VAP, the trigger-based screening had a sensitivity of 92.3%, a specificity of 100%, and a negative predictive value of 99.8% compared to traditional screening of all patients. For CLABSI, sensitivity was 91.3%, specificity 100%, and negative predictive value 99.6%.

Conclusions

Pre-selection of patients to be checked for signs and symptoms of VAP and CLABSI by a computer-generated automated trigger system was time saving but slightly less accurate than conventional surveillance. However, this after-the-fact surveillance was mainly designed as a quality indicator over time rather than for precise determination of infection rates. Therefore, surveillance of VAP and CLABSI with a trigger-based ESS is feasible and effective.