Estimating One-Year Risk of Incident Chronic Kidney Disease: Retrospective Development and Validation Study Using Electronic Medical Record Data From the State of Maine

Optimizing Care Delivery in Patients with Chronic Kidney Disease in the United States: Proceedings of a Multidisciplinary Roundtable Discussion and Literature Review

BACKGROUND

Approximately 37 million individuals in the United States (US) have chronic kidney disease (CKD). Patients with CKD have a substantial morbidity and mortality, which contributes to a huge economic burden to the healthcare system. A limited number of clinical pathways or defined workflows exist for CKD care delivery in the US, primarily due to a lower prioritization of CKD care within health systems compared with other areas (e.g., cardiovascular disease [CVD], cancer screening). CKD is a public health crisis and by the year 2040, CKD will become the fifth leading cause of years of life lost. It is therefore critical to address these challenges to improve outcomes in patients with CKD.

METHODS

The CKD Leaders Network conducted a virtual, 3 h, multidisciplinary roundtable discussion with eight subject-matter experts to better understand key factors impacting CKD care delivery and barriers across the US. A premeeting survey identified topics for discussion covering the screening, diagnosis, risk stratification, and management of CKD across the care continuum. Findings from this roundtable are summarized and presented herein.

RESULTS

Universal challenges exist across health systems, including a lack of awareness amongst providers and patients, constrained care team bandwidth, inadequate financial incentives for early CKD identification, non-standardized diagnostic classification and triage processes, and non-centralized patient information. Proposed solutions include highlighting immediate and long-term financial implications linked with failure to identify and address at-risk individuals, identifying and managing early-stage CKD, enhancing efforts to support guideline-based education for providers and patients, and capitalizing on next-generation solutions.

CONCLUSIONS

Payers and other industry stakeholders have opportunities to contribute to optimal CKD care delivery. Beyond addressing the inadequacies that currently exist, actionable tactics can be implemented into clinical practice to improve clinical outcomes in patients at risk for or diagnosed with CKD in the US.

Risk Prediction Score for Chronic Kidney Disease in Healthy Adults and Adults With Type 2 Diabetes: Systematic Review

INTRODUCTION

Chronic kidney disease (CKD) is an important public health problem. In 2017, the global prevalence was estimated at 9.1%. Appropriate tools to predict the risk of developing CKD are necessary to prevent its progression. Type 2 diabetes is a leading cause of CKD; screening the population living with the disease is a cost-effective solution to prevent CKD. The aim of our study was to identify the existing prediction scores and their diagnostic accuracy for detecting CKD in apparently healthy populations and populations with type 2 diabetes.

METHODS

We conducted an electronic search in databases, including Medline/PubMed, Embase, Health Evidence, and others. For the inclusion criteria we considered studies with a risk predictive score in healthy populations and populations with type 2 diabetes. We extracted information about the models, variables, and diagnostic accuracy, such as area under the receiver operating characteristic curve (AUC), C statistic, or sensitivity and specificity.

RESULTS

We screened 2,359 records and included 13 studies for healthy population, 7 studies for patients with type 2 diabetes, and 1 for both populations. We identified 12 models for patients with type 2 diabetes; the range of C statistic was from 0.56 to 0.81, and the range of AUC was from 0.71 to 0.83. For healthy populations, we identified 36 models with the range of C statistics from 0.65 to 0.91, and the range of AUC from 0.63 to 0.91.

CONCLUSION

This review identified models with good discriminatory performance and methodologic quality, but they need more validation in populations other than those studied. This review did not identify risk models with variables comparable between them to enable conducting a meta-analysis.

The application of machine learning to imaging in hematological oncology: A scoping review

Background

Here, we conducted a scoping review to (i) establish which machine learning (ML) methods have been applied to hematological malignancy imaging; (ii) establish how ML is being applied to hematological cancer radiology; and (iii) identify addressable research gaps.

Methods

The review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews guidelines. The inclusion criteria were (i) pediatric and adult patients with suspected or confirmed hematological malignancy undergoing imaging (population); (ii) any study using ML techniques to derive models using radiological images to apply to the clinical management of these patients (concept); and (iii) original research articles conducted in any setting globally (context). Quality Assessment of Diagnostic Accuracy Studies 2 criteria were used to assess diagnostic and segmentation studies, while the Newcastle-Ottawa scale was used to assess the quality of observational studies.

Results

Of 53 eligible studies, 33 applied diverse ML techniques to diagnose hematological malignancies or to differentiate them from other diseases, especially discriminating gliomas from primary central nervous system lymphomas (n=18); 11 applied ML to segmentation tasks, while 9 applied ML to prognostication or predicting therapeutic responses, especially for diffuse large B-cell lymphoma. All studies reported discrimination statistics, but no study calculated calibration statistics. Every diagnostic/segmentation study had a high risk of bias due to their case-control design; many studies failed to provide adequate details of the reference standard; and only a few studies used independent validation.

Conclusion

To deliver validated ML-based models to radiologists managing hematological malignancies, future studies should (i) adhere to standardized, high-quality reporting guidelines such as the Checklist for Artificial Intelligence in Medical Imaging; (ii) validate models in independent cohorts; (ii) standardize volume segmentation methods for segmentation tasks; (iv) establish comprehensive prospective studies that include different tumor grades, comparisons with radiologists, optimal imaging modalities, sequences, and planes; (v) include side-by-side comparisons of different methods; and (vi) include low- and middle-income countries in multicentric studies to enhance generalizability and reduce inequity.

Single center blind testing of a US multi-center validated diagnostic algorithm for Kawasaki disease in Taiwan

Background

Kawasaki disease (KD) is the leading cause of acquired heart disease in children. The major challenge in KD diagnosis is that it shares clinical signs with other childhood febrile control (FC) subjects. We sought to determine if our algorithmic approach applied to a Taiwan cohort.

Methods

A single center (Chang Gung Memorial Hospital in Taiwan) cohort of patients suspected with acute KD were prospectively enrolled by local KD specialists for KD analysis. Our previously single-center developed computer-based two-step algorithm was further tested by a five-center validation in US. This first blinded multi-center trial validated our approach, with sufficient sensitivity and positive predictive value, to identify most patients with KD diagnosed at centers across the US. This study involved 418 KDs and 259 FCs from the Chang Gung Memorial Hospital in Taiwan.

Findings

Our diagnostic algorithm retained sensitivity (379 of 418; 90.7%), specificity (223 of 259; 86.1%), PPV (379 of 409; 92.7%), and NPV (223 of 247; 90.3%) comparable to previous US 2016 single center and US 2020 fiver center results. Only 4.7% (15 of 418) of KD and 2.3% (6 of 259) of FC patients were identified as indeterminate. The algorithm identified 18 of 50 (36%) KD patients who presented 2 or 3 principal criteria. Of 418 KD patients, 157 were infants younger than one year and 89.2% (140 of 157) were classified correctly. Of the 44 patients with KD who had coronary artery abnormalities, our diagnostic algorithm correctly identified 43 (97.7%) including all patients with dilated coronary artery but one who found to resolve in 8 weeks.

Interpretation

This work demonstrates the applicability of our algorithmic approach and diagnostic portability in Taiwan.

A Primer on the Present State and Future Prospects for Machine Learning and Artificial Intelligence Applications in Cardiology

The artificial intelligence (AI) revolution is well underway, including in the medical field, and has dramatically transformed our lives. An understanding of the basics of AI applications, their development, and challenges to their clinical implementation is important for clinicians to fully appreciate the possibilities of AI. Such a foundation would ensure that clinicians have a good grasp and realistic expectations for AI in medicine and prevent discrepancies between the promised and real-world impact. When quantifying the track record for AI applications in cardiology, we found that a substantial number of AI systems are never deployed in clinical practice, although there certainly are many success stories. Successful implementations shared the following: they came from clinical areas where large amount of training data was available; were deployable into a single diagnostic modality; prediction models generally had high performance on external validation; and most were developed as part of collaborations with medical device manufacturers who had substantial experience with implementation of new technology. When looking into the current processes used for developing AI-based systems, we suggest that expanding the analytic framework to address potential deployment and implementation issues at project outset will improve the rate of successful implementation, and will be a necessary next step for AI to achieve its full potential in cardiovascular medicine.

Predicting the risk of cancer in adults using supervised machine learning: a scoping review

OBJECTIVES

The purpose of this scoping review is to: (1) identify existing supervised machine learning (ML) approaches on the prediction of cancer in asymptomatic adults; (2) to compare the performance of ML models with each other and (3) to identify potential gaps in research.

DESIGN

Scoping review using the population, concept and context approach.

SEARCH STRATEGY

PubMed search engine was used from inception to 10 November 2020 to identify literature meeting following inclusion criteria: (1) a general adult (≥18 years) population, either sex, asymptomatic (population); (2) any study using ML techniques to derive predictive models for future cancer risk using clinical and/or demographic and/or basic laboratory data (concept) and (3) original research articles conducted in all settings in any region of the world (context).

RESULTS

The search returned 627 unique articles, of which 580 articles were excluded because they did not meet the inclusion criteria, were duplicates or were related to benign neoplasm. Full-text reviews were conducted for 47 articles and a final set of 10 articles were included in this scoping review. These 10 very heterogeneous studies used ML to predict future cancer risk in asymptomatic individuals. All studies reported area under the receiver operating characteristics curve (AUC) values as metrics of model performance, but no study reported measures of model calibration.

CONCLUSIONS

Research gaps that must be addressed in order to deliver validated ML-based models to assist clinical decision-making include: (1) establishing model generalisability through validation in independent cohorts, including those from low-income and middle-income countries; (2) establishing models for all cancer types; (3) thorough comparisons of ML models with best available clinical tools to ensure transparency of their potential clinical utility; (4) reporting of model calibration performance and (5) comparisons of different methods on the same cohort to reveal important information about model generalisability and performance.

Electronic Medical Record-Based Case Phenotyping for the Charlson Conditions: Scoping Review

BACKGROUND

Electronic medical records (EMRs) contain large amounts of rich clinical information. Developing EMR-based case definitions, also known as EMR phenotyping, is an active area of research that has implications for epidemiology, clinical care, and health services research.

OBJECTIVE

This review aims to describe and assess the present landscape of EMR-based case phenotyping for the Charlson conditions.

METHODS

A scoping review of EMR-based algorithms for defining the Charlson comorbidity index conditions was completed. This study covered articles published between January 2000 and April 2020, both inclusive. Embase (Excerpta Medica database) and MEDLINE (Medical Literature Analysis and Retrieval System Online) were searched using keywords developed in the following 3 domains: terms related to EMR, terms related to case finding, and disease-specific terms. The manuscript follows the Preferred Reporting Items for Systematic reviews and Meta-analyses extension for Scoping Reviews (PRISMA) guidelines.

RESULTS

A total of 274 articles representing 299 algorithms were assessed and summarized. Most studies were undertaken in the United States (181/299, 60.5%), followed by the United Kingdom (42/299, 14.0%) and Canada (15/299, 5.0%). These algorithms were mostly developed either in primary care (103/299, 34.4%) or inpatient (168/299, 56.2%) settings. Diabetes, congestive heart failure, myocardial infarction, and rheumatology had the highest number of developed algorithms. Data-driven and clinical rule-based approaches have been identified. EMR-based phenotype and algorithm development reflect the data access allowed by respective health systems, and algorithms vary in their performance.

CONCLUSIONS

Recognizing similarities and differences in health systems, data collection strategies, extraction, data release protocols, and existing clinical pathways is critical to algorithm development strategies. Several strategies to assist with phenotype-based case definitions have been proposed.

Assessment of Deep Learning Using Nonimaging Information and Sequential Medical Records to Develop a Prediction Model for Nonmelanoma Skin Cancer

Importance

A prediction model for new-onset nonmelanoma skin cancer could enhance prevention measures, but few patient data-driven tools exist for more accurate prediction.

Objective

To use machine learning to develop a prediction model for incident nonmelanoma skin cancer based on large-scale, multidimensional, nonimaging medical information.

Design, Setting, and Participants

This study used a database comprising 2 million randomly sampled patients from the Taiwan National Health Insurance Research Database from January 1, 1999, to December 31, 2013. A total of 1829 patients with nonmelanoma skin cancer as their first diagnosed cancer and 7665 random controls without cancer were included in the analysis. A convolutional neural network, a deep learning approach, was used to develop a risk prediction model. This risk prediction model used 3-year clinical diagnostic information, medical records, and temporal-sequential information to predict the skin cancer risk of a given patient within the next year. Stepwise feature selection was also performed to investigate important and determining factors of the model. Statistical analysis was performed from November 1, 2016, to October 31, 2018.

Main Outcomes and Measures

Sensitivity, specificity, and area under the receiver operating characteristic (AUROC) curve were used to evaluate the performance of the models.

Results

A total of 1829 patients (923 women [50.5%] and 906 men [49.5%]; mean [SD] age, 65.3 [15.7] years) with nonmelanoma skin cancer and 7665 random controls without cancer (3951 women [51.5%] and 3714 men [48.4%]; mean [SD] age, 47.5 [17.3] years) were included in the analysis. The 1-year incident nonmelanoma skin cancer risk prediction model using sequential diagnostic information and drug prescription information as a time-incorporated feature matrix could attain an AUROC of 0.89 (95% CI, 0.87-0.91), with a mean (SD) sensitivity of 83.1% (3.5%) and mean (SD) specificity of 82.3% (4.1%). Carcinoma in situ of skin (AUROC, 0.867; -2.80% loss) and other chronic comorbidities (eg, degenerative osteopathy [AUROC, 0.872; -2.32% loss], hypertension [AUROC, 0.879; -1.53% loss], and chronic kidney insufficiency [AUROC, 0.879; -1.52% loss]) served as more discriminative factors for the prediction. Medications such as trazodone, acarbose, systemic antifungal agents, statins, nonsteroidal anti-inflammatory drugs, and thiazide diuretics were the top-ranking discriminative features in the model; each led to more than a 1% decrease of the AUROC when eliminated individually (eg, trazodone AUROC, 0.868; -2.67% reduction; acarbose AUROC, 0.870; -2.50 reduction; and systemic antifungal agents AUROC, 0.875; -1.99 reduction).

Conclusions and Relevance

The findings of this study suggest that a risk prediction model may have potential predictive factors for nonmelanoma skin cancer. This model may help health care professionals target high-risk populations for more intensive skin cancer preventive methods.

Prediction of the 1-Year Risk of Incident Lung Cancer: Prospective Study Using Electronic Health Records from the State of Maine

BACKGROUND

Lung cancer is the leading cause of cancer death worldwide. Early detection of individuals at risk of lung cancer is critical to reduce the mortality rate.

OBJECTIVE

The aim of this study was to develop and validate a prospective risk prediction model to identify patients at risk of new incident lung cancer within the next 1 year in the general population.

METHODS

Data from individual patient electronic health records (EHRs) were extracted from the Maine Health Information Exchange network. The study population consisted of patients with at least one EHR between April 1, 2016, and March 31, 2018, who had no history of lung cancer. A retrospective cohort (N=873,598) and a prospective cohort (N=836,659) were formed for model construction and validation. An Extreme Gradient Boosting (XGBoost) algorithm was adopted to build the model. It assigned a score to each individual to quantify the probability of a new incident lung cancer diagnosis from October 1, 2016, to September 31, 2017. The model was trained with the clinical profile in the retrospective cohort from the preceding 6 months and validated with the prospective cohort to predict the risk of incident lung cancer from April 1, 2017, to March 31, 2018.

RESULTS

The model had an area under the curve (AUC) of 0.881 (95% CI 0.873-0.889) in the prospective cohort. Two thresholds of 0.0045 and 0.01 were applied to the predictive scores to stratify the population into low-, medium-, and high-risk categories. The incidence of lung cancer in the high-risk category (579/53,922, 1.07%) was 7.7 times higher than that in the overall cohort (1167/836,659, 0.14%). Age, a history of pulmonary diseases and other chronic diseases, medications for mental disorders, and social disparities were found to be associated with new incident lung cancer.

CONCLUSIONS

We retrospectively developed and prospectively validated an accurate risk prediction model of new incident lung cancer occurring in the next 1 year. Through statistical learning from the statewide EHR data in the preceding 6 months, our model was able to identify statewide high-risk patients, which will benefit the population health through establishment of preventive interventions or more intensive surveillance.

Using Predictive Analytics to Identify Children at High Risk of Defaulting From a Routine Immunization Program: Feasibility Study

BACKGROUND

Despite the availability of free routine immunizations in low- and middle-income countries, many children are not completely vaccinated, vaccinated late for age, or drop out from the course of the immunization schedule. Without the technology to model and visualize risk of large datasets, vaccinators and policy makers are unable to identify target groups and individuals at high risk of dropping out; thus default rates remain high, preventing universal immunization coverage. Predictive analytics algorithm leverages artificial intelligence and uses statistical modeling, machine learning, and multidimensional data mining to accurately identify children who are most likely to delay or miss their follow-up immunization visits.

OBJECTIVE

This study aimed to conduct feasibility testing and validation of a predictive analytics algorithm to identify the children who are likely to default on subsequent immunization visits for any vaccine included in the routine immunization schedule.

METHODS

The algorithm was developed using 47,554 longitudinal immunization records, which were classified into the training and validation cohorts. Four machine learning models (random forest; recursive partitioning; support vector machines, SVMs; and C-forest) were used to generate the algorithm that predicts the likelihood of each child defaulting from the follow-up immunization visit. The following variables were used in the models as predictors of defaulting: gender of the child, language spoken at the child's house, place of residence of the child (town or city), enrollment vaccine, timeliness of vaccination, enrolling staff (vaccinator or others), date of birth (accurate or estimated), and age group of the child. The models were encapsulated in the predictive engine, which identified the most appropriate method to use in a given case. Each of the models was assessed in terms of accuracy, precision (positive predictive value), sensitivity, specificity and negative predictive value, and area under the curve (AUC).

RESULTS

Out of 11,889 cases in the validation dataset, the random forest model correctly predicted 8994 cases, yielding 94.9% sensitivity and 54.9% specificity. The C-forest model, SVMs, and recursive partitioning models improved prediction by achieving 352, 376, and 389 correctly predicted cases, respectively, above the predictions made by the random forest model. All models had a C-statistic of 0.750 or above, whereas the highest statistic (AUC 0.791, 95% CI 0.784-0.798) was observed in the recursive partitioning algorithm.

CONCLUSIONS

This feasibility study demonstrates that predictive analytics can accurately identify children who are at a higher risk for defaulting on follow-up immunization visits. Correct identification of potential defaulters opens a window for evidence-based targeted interventions in resource limited settings to achieve optimal immunization coverage and timeliness.

Assessing Statewide All-Cause Future One-Year Mortality: Prospective Study With Implications for Quality of Life, Resource Utilization, and Medical Futility

Background

For many elderly patients, a disproportionate amount of health care resources and expenditures is spent during the last year of life, despite the discomfort and reduced quality of life associated with many aggressive medical approaches. However, few prognostic tools have focused on predicting all-cause 1-year mortality among elderly patients at a statewide level, an issue that has implications for improving quality of life while distributing scarce resources fairly.

Objective

Using data from a statewide elderly population (aged ≥65 years), we sought to prospectively validate an algorithm to identify patients at risk for dying in the next year for the purpose of minimizing decision uncertainty, improving quality of life, and reducing futile treatment.

Methods

Analysis was performed using electronic medical records from the Health Information Exchange in the state of Maine, which covered records of nearly 95% of the statewide population. The model was developed from 125,896 patients aged at least 65 years who were discharged from any care facility in the Health Information Exchange network from September 5, 2013, to September 4, 2015. Validation was conducted using 153,199 patients with same inclusion and exclusion criteria from September 5, 2014, to September 4, 2016. Patients were stratified into risk groups. The association between all-cause 1-year mortality and risk factors was screened by chi-squared test and manually reviewed by 2 clinicians. We calculated risk scores for individual patients using a gradient tree-based boost algorithm, which measured the probability of mortality within the next year based on the preceding 1-year clinical profile.

Results

The development sample included 125,896 patients (72,572 women, 57.64%; mean 74.2 [SD 7.7] years). The final validation cohort included 153,199 patients (88,177 women, 57.56%; mean 74.3 [SD 7.8] years). The c-statistic for discrimination was 0.96 (95% CI 0.93-0.98) in the development group and 0.91 (95% CI 0.90-0.94) in the validation cohort. The mortality was 0.99% in the low-risk group, 16.75% in the intermediate-risk group, and 72.12% in the high-risk group. A total of 99 independent risk factors (n=99) for mortality were identified (reported as odds ratios; 95% CI). Age was on the top of list (1.41; 1.06-1.48); congestive heart failure (20.90; 15.41-28.08) and different tumor sites were also recognized as driving risk factors, such as cancer of the ovaries (14.42; 2.24-53.04), colon (14.07; 10.08-19.08), and stomach (13.64; 3.26-86.57). Disparities were also found in patients’ social determinants like respiratory hazard index (1.24; 0.92-1.40) and unemployment rate (1.18; 0.98-1.24). Among high-risk patients who expired in our dataset, cerebrovascular accident, amputation, and type 1 diabetes were the top 3 diseases in terms of average cost in the last year of life.

Conclusions

Our study prospectively validated an accurate 1-year risk prediction model and stratification for the elderly population (≥65 years) at risk of mortality with statewide electronic medical record datasets. It should be a valuable adjunct for helping patients to make better quality-of-life choices and alerting care givers to target high-risk elderly for appropriate care and discussions, thus cutting back on futile treatment.

Prediction of Incident Hypertension Within the Next Year: Prospective Study Using Statewide Electronic Health Records and Machine Learning

BACKGROUND

As a high-prevalence health condition, hypertension is clinically costly, difficult to manage, and often leads to severe and life-threatening diseases such as cardiovascular disease (CVD) and stroke.

OBJECTIVE

The aim of this study was to develop and validate prospectively a risk prediction model of incident essential hypertension within the following year.

METHODS

Data from individual patient electronic health records (EHRs) were extracted from the Maine Health Information Exchange network. Retrospective (N=823,627, calendar year 2013) and prospective (N=680,810, calendar year 2014) cohorts were formed. A machine learning algorithm, XGBoost, was adopted in the process of feature selection and model building. It generated an ensemble of classification trees and assigned a final predictive risk score to each individual.

RESULTS

The 1-year incident hypertension risk model attained areas under the curve (AUCs) of 0.917 and 0.870 in the retrospective and prospective cohorts, respectively. Risk scores were calculated and stratified into five risk categories, with 4526 out of 381,544 patients (1.19%) in the lowest risk category (score 0-0.05) and 21,050 out of 41,329 patients (50.93%) in the highest risk category (score 0.4-1) receiving a diagnosis of incident hypertension in the following 1 year. Type 2 diabetes, lipid disorders, CVDs, mental illness, clinical utilization indicators, and socioeconomic determinants were recognized as driving or associated features of incident essential hypertension. The very high risk population mainly comprised elderly (age>50 years) individuals with multiple chronic conditions, especially those receiving medications for mental disorders. Disparities were also found in social determinants, including some community-level factors associated with higher risk and others that were protective against hypertension.

CONCLUSIONS

With statewide EHR datasets, our study prospectively validated an accurate 1-year risk prediction model for incident essential hypertension. Our real-time predictive analytic model has been deployed in the state of Maine, providing implications in interventions for hypertension and related diseases and hopefully enhancing hypertension care.