Development and validation of risk models to predict the 7-year risk of type 2 diabetes: The Japan Epidemiology Collaboration on Occupational Health Study

Development and Validation of Prediction Models for the 5-year Risk of Type 2 Diabetes in a Japanese Population: Japan Public Health Center-based Prospective (JPHC) Diabetes Study

BACKGROUND

This study aimed to develop models to predict the 5-year incidence of type 2 diabetes mellitus (T2DM) in a Japanese population and validate them externally in an independent Japanese population.

METHODS

Data from 10,986 participants (aged 46-75 years) in the development cohort of the Japan Public Health Center-based Prospective Diabetes Study and 11,345 participants (aged 46-75 years) in the validation cohort of the Japan Epidemiology Collaboration on Occupational Health Study were used to develop and validate the risk scores in logistic regression models.

RESULTS

We considered non-invasive (sex, body mass index, family history of diabetes mellitus, and diastolic blood pressure) and invasive (glycated hemoglobin [HbA1c] and fasting plasma glucose [FPG]) predictors to predict the 5-year probability of incident diabetes. The area under the receiver operating characteristic curve was 0.643 for the non-invasive risk model, 0.786 for the invasive risk model with HbA1c but not FPG, and 0.845 for the invasive risk model with HbA1c and FPG. The optimism for the performance of all models was small by internal validation. In the internal-external cross-validation, these models tended to show similar discriminative ability across different areas. The discriminative ability of each model was confirmed using external validation datasets. The invasive risk model with only HbA1c was well-calibrated in the validation cohort.

CONCLUSION

Our invasive risk models are expected to discriminate between high- and low-risk individuals with T2DM in a Japanese population.

Application of deep neural survival networks to the development of risk prediction models for diabetes mellitus, hypertension, and dyslipidemia

OBJECTIVES

: Although numerous risk prediction models have been proposed, few such models have been developed using neural network-based survival analysis. We developed risk prediction models for three cardiovascular disease risk factors (diabetes mellitus, hypertension, and dyslipidemia) among a working-age population in Japan using DeepSurv, a deep feed-forward neural network.

METHODS

: Data were obtained from the Japan Epidemiology Collaboration on Occupational Health Study. A total of 51 258, 44 197, and 31 452 individuals were included in the development of risk models for diabetes mellitus, hypertension, and dyslipidemia, respectively; two-thirds of whom were used to develop prediction models, and the rest were used to validate the models. We compared the performances of DeepSurv-based models with those of prediction models based on the Cox proportional hazards model.

RESULTS

: The area under the receiver-operating characteristic curve was 0.878 [95% confidence interval (CI) = 0.864-0.892] for diabetes mellitus, 0.835 (95% CI = 0.826-0.845) for hypertension, and 0.826 (95% CI = 0.817-0.835) for dyslipidemia. Compared with the Cox proportional hazards-based models, the DeepSurv-based models had better reclassification performance [diabetes mellitus: net reclassification improvement (NRI) = 0.474, P  ≤ 0.001; hypertension: NRI = 0.194, P  ≤ 0.001; dyslipidemia: NRI = 0.397, P  ≤ 0.001] and discrimination performance [diabetes mellitus: integrated discrimination improvement (IDI) = 0.013, P  ≤ 0.001; hypertension: IDI = 0.007, P  ≤ 0.001; and dyslipidemia: IDI = 0.043, P  ≤ 0.001].

CONCLUSION

: This study suggests that DeepSurv has the potential to improve the performance of risk prediction models for cardiovascular disease risk factors.

Should insulin resistance (HOMA-IR), insulin secretion (HOMA-β), and visceral fat area be considered for improving the performance of diabetes risk prediction models

INTRODUCTION

Insulin resistance and defects in pancreatic beta cells are the two major pathophysiologic abnormalities that underlie type 2 diabetes. In addition, visceral fat area (VFA) is reported to be a stronger predictor for diabetes than body mass index (BMI). Here, we tested whether the performance of diabetes prediction models could be improved by adding HOMA-IR and HOMA-β and replacing BMI with VFA.

RESEARCH DESIGN AND METHODS

We developed five prediction models using data from a cohort study (5578 individuals, of whom 94.7% were male, and 943 had incident diabetes). We conducted a baseline model (model 1) including age, sex, BMI, smoking, dyslipidemia, hypertension, and HbA1c. Subsequently, we developed another four models: model 2, predictors in model 1 plus fasting plasma glucose (FPG); model 3, predictors in model 1 plus HOMA-IR and HOMA-β; model 4, predictors in model 1 plus FPG, HOMA-IR, and HOMA-β; model 5, replaced BMI with VFA in model 2. We assessed model discrimination and calibration for the first 10 years of follow-up.

RESULTS

The addition of FPG to model 1 obviously increased the value of the area under the receiver operating characteristic curve from 0.79 (95% CI 0.78, 0.81) to 0.84 (0.83, 0.85). Compared with model 1, model 2 also significantly improved the risk reclassification and discrimination, with a continuous net reclassification improvement index of 0.61 (0.56, 0.70) and an integrated discrimination improvement index of 0.09 (0.08, 0.10). Adding HOMA-IR and HOMA-β (models 3 and 4) or replacing BMI with VFA (model 5) did not further materially improve the performance.

CONCLUSIONS

This cohort study, primarily composed of male workers, suggests that a model with BMI, FPG, and HbA1c effectively identifies those at high diabetes risk. However, adding HOMA-IR, HOMA-β, or replacing BMI with VFA does not significantly improve the model. Further studies are needed to confirm our findings.

Risk prediction models for incident type 2 diabetes in Chinese people with intermediate hyperglycemia: a systematic literature review and external validation study

Background

People with intermediate hyperglycemia (IH), including impaired fasting glucose and/or impaired glucose tolerance, are at higher risk of developing type 2 diabetes (T2D) than those with normoglycemia. We aimed to evaluate the performance of published T2D risk prediction models in Chinese people with IH to inform them about the choice of primary diabetes prevention measures.

Methods

A systematic literature search was conducted to identify Asian-derived T2D risk prediction models, which were eligible if they were built on a prospective cohort of Asian adults without diabetes at baseline and utilized routinely-available variables to predict future risk of T2D. These Asian-derived and five prespecified non-Asian derived T2D risk prediction models were divided into BASIC (clinical variables only) and EXTENDED (plus laboratory variables) versions, with validation performed on them in three prospective Chinese IH cohorts: ACE (n = 3241), Luzhou (n = 1333), and TCLSIH (n = 1702). Model performance was assessed in terms of discrimination (C-statistic) and calibration (Hosmer–Lemeshow test).

Results

Forty-four Asian and five non-Asian studies comprising 21 BASIC and 46 EXTENDED T2D risk prediction models for validation were identified. The majority were at high (n = 43, 87.8%) or unclear (n = 3, 6.1%) risk of bias, while only three studies (6.1%) were scored at low risk of bias. BASIC models showed poor-to-moderate discrimination with C-statistics 0.52–0.60, 0.50–0.59, and 0.50–0.64 in the ACE, Luzhou, and TCLSIH cohorts respectively. EXTENDED models showed poor-to-acceptable discrimination with C-statistics 0.54–0.73, 0.52–0.67, and 0.59–0.78 respectively. Fifteen BASIC and 40 EXTENDED models showed poor calibration (P < 0.05), overpredicting or underestimating the observed diabetes risk. Most recalibrated models showed improved calibration but modestly-to-severely overestimated diabetes risk in the three cohorts. The NAVIGATOR model showed the best discrimination in the three cohorts but had poor calibration (P < 0.05).

Conclusions

In Chinese people with IH, previously published BASIC models to predict T2D did not exhibit good discrimination or calibration. Several EXTENDED models performed better, but a robust Chinese T2D risk prediction tool in people with IH remains a major unmet need.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12933-022-01622-5.

Prediction Models for Type 2 Diabetes Risk in the General Population: A Systematic Review of Observational Studies

OBJECTIVES

This study aimed to provide an overview of prediction models of undiagnosed type 2 diabetes mellitus (U-T2DM) or the incident T2DM (I-T2DM) using the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) checklist and the prediction model risk of the bias assessment tool (PROBAST).

DATA SOURCES

Both PUBMED and EMBASE databases were searched to guarantee adequate and efficient coverage.

STUDY SELECTION

Articles published between December 2011 and October 2019 were considered.

DATA EXTRACTION

For each article, information on model development requirements, discrimination measures, calibration, overall performance, clinical usefulness, overfitting, and risk of bias (ROB) was reported.

RESULTS

The median (interquartile range; IQR) number of the 46 study populations for model development was 5711 (1971 - 27426) and 2457 (2060 - 6995) individuals for I-T2DM and U-T2DM, respectively. The most common reported predictors were age and body mass index, and only the Qrisk-2017 study included social factors (e.g., Townsend score). Univariable analysis was reported in 46% of the studies, and the variable selection procedure was not clear in 17.4% of them. Moreover, internal and external validation was reported in 43% the studies, while over 63% of them reported calibration. The median (IQR) of AUC for I-T2DM models was 0.78 (0.74 - 0.82); the corresponding value for studies derived before October 2011 was 0.80 (0.77 - 0.83). The highest discrimination index was reported for Qrisk-2017 with C-statistics of 0.89 for women and 0.87 for men. Low ROB for I-T2DM and U-T2DM was assessed at 18% and 41%, respectively.

CONCLUSIONS

Among prediction models, an intermediate to poor quality was reassessed in several aspects of model development and validation. Generally, despite its new risk factors or new methodological aspects, the newly developed model did not increase our capability in screening/predicting T2DM, mainly in the analysis part. It was due to the lack of external validation of the prediction models.

Development and validation of a diabetes risk score among two populations

The purpose of this study was to assess the validity of a practical diabetes risk score amongst two heterogenous populations, a working population and a non-working population. Study population 1 (n = 2,089) participated in a large-scale screening program offered to retired workers to discover previously undetected/incipient chronic illness. Study population 2 (n = 3,293) was part of a Colorado worksite wellness program health risk assessment. We assessed the relationship between a continuous diabetes risk score at baseline and development of diabetes in the future using logistic regression. Receiver operating curves and sensitivity/specificity of the models were calculated. Across both study populations, we observed that participants with diabetes at follow-up had higher diabetes risk scores at baseline than participants who did not have diabetes at follow-up. On average, the odds ratio of developing diabetes in the future was 1.38 (95% CI: 1.26-1.50, p < 0.0001) for study population 1 and 1.68 (95% CI: 1.45-1.95, p-value < 0.0001) for study population 2. These findings indicate that the diabetes risk score may be generalizable to diverse individuals, and thus potentially a population level diabetes screening tool. Minimally-invasive diabetes risk scores can aid in the identification of sub-populations of individuals at risk for diabetes.

Smoking cessation after long-term sick leave due to cancer in comparison with cardiovascular disease: Japan Epidemiology Collaboration on Occupational Health Study

In occupational settings, smokers may take quitting smoking seriously if they experienced long-term sick leave due to cancer or cardiovascular disease (CVD). However, no study has elucidated the smoking cessation rate after long-term sick leave. We examined the smoking cessation rate after long-term sick leave due to cancer and CVD in Japan. We followed 23 survivors who experienced long-term sick leave due to cancer and 39 survivors who experienced long-term sick leave due to CVD who reported smoking at the last health exam before the leave. Their smoking habits before and after the leave were self-reported. Logistic regression was used to calculate adjusted smoking cessation rates. Smoking cessation rate after long-term sick leave due to cancer was approximately 70% and that due to CVD exceeded 80%. The adjusted smoking cessation rate was 67.6% (95% confidence interval [CI]: 47.0, 88.2) for cancer and 80.7% (95% CI: 67.7, 93.8) for CVD. Smoking cessation rate after a longer duration of sick leave (≥60 d) tended to increase for both CVD and cancer. Although any definite conclusion cannot be drawn, the data suggest that smoking cessation rate after long-term sick leave due to CVD is slightly higher than that for cancer.

Systematic review with meta-analysis of the epidemiological evidence relating smoking to type 2 diabetes

BACKGROUND

Evidence relating tobacco smoking to type 2 diabetes has accumulated rapidly in the last few years, rendering earlier reviews considerably incomplete.

AIM

To review and meta-analyse evidence from prospective studies of the relationship between smoking and the onset of type 2 diabetes.

METHODS

Prospective studies were selected if the population was free of type 2 diabetes at baseline and evidence was available relating smoking to onset of the disease. Papers were identified from previous reviews, searches on Medline and Embase and reference lists. Data were extracted on a range of study characteristics and relative risks (RRs) were extracted comparing current, ever or former smokers with never smokers, and current smokers with non-current smokers, as well as by amount currently smoked and duration of quitting. Fixed- and random-effects estimates summarized RRs for each index of smoking overall and by various subdivisions of the data: Sex; continent; publication year; method of diagnosis; nature of the baseline population (inclusion/exclusion of pre-diabetes); number of adjustment factors; cohort size; number of type 2 diabetes cases; age; length of follow-up; definition of smoking; and whether or not various factors were adjusted for. Tests of heterogeneity and publication bias were also conducted.

RESULTS

The literature searches identified 157 relevant publications providing results from 145 studies. Fifty-three studies were conducted in Asia and 53 in Europe, with 32 in North America, and seven elsewhere. Twenty-four were in males, 10 in females and the rest in both sexes. Fifteen diagnosed type 2 diabetes from self-report by the individuals, 79 on medical records, and 51 on both. Studies varied widely in size of the cohort, number of cases, length of follow-up, and age. Overall, random-effects estimates of the RR were 1.33 [95% confidence interval (CI): 1.28-1.38] for current vs never smoking, 1.28 (95%CI: 1.24-1.32) for current vs non-smoking, 1.13 (95%CI: 1.11-1.16) for former vs never smoking, and 1.25 (95%CI: 1.21-1.28) for ever vs never smoking based on, respectively, 99, 156, 100 and 100 individual risk estimates. Risk estimates were generally elevated in each subdivision of the data by the various factors considered (exceptions being where numbers of estimates in the subsets were very low), though there was significant (P < 0.05) evidence of variation by level for some factors. Dose-response analysis showed a clear trend of increasing risk with increasing amount smoked by current smokers and of decreasing risk with increasing time quit. There was limited evidence of publication bias.

CONCLUSION

The analyses confirmed earlier reports of a modest dose-related association of current smoking and a weaker dose-related association of former smoking with type 2 diabetes risk.

Risk Prediction of Dyslipidemia for Chinese Han Adults Using Random Forest Survival Model

OBJECTIVE

Dyslipidemia has been recognized as a major risk factor of several diseases, and early prevention and management of dyslipidemia is effective in the primary prevention of cardiovascular events. The present study aims to develop risk models for predicting dyslipidemia using Random Survival Forest (RSF), which take the complex relationship between the variables into account.

METHODS

We used data from 6328 participants aged between 19 and 90 years free of dyslipidemia at baseline with a maximum follow-up of 5 years. RSF was applied to develop gender-specific risk model for predicting dyslipidemia using variables from anthropometric and laboratory test in the cohort. Cox regression was also adopted in comparison with the RSF model, and Harrell's concordance statistic with 10-fold cross-validation was used to validate the models.

RESULTS

The incidence density of dyslipidemia was 101/1000 in total and subgroup incidence densities were 121/1000 for men and 69/1000 for women. Twenty-four predictors were identified in the prediction model of males and 23 in females. The C-statistics of the prediction models for males and females were 0.731 and 0.801, respectively. The RSF model shows better discriminative performance than CPH model (0.719 for males and 0.787 for females). Moreover, some predictors were observed to have a nonlinear effect on dyslipidemia.

CONCLUSION

The RSF model is a promising method in identifying high-risk individuals for the prevention of dyslipidemia and related diseases.

Patterns of changes in overtime working hours over 3 years and the risk for progression to type 2 diabetes in adults with pre-diabetes

No information exists regarding the effects of working hours on glucose metabolism in adults with pre-diabetes, a high-risk group for developing diabetes. Further, longitudinal patterns in working hours and their effects on glucose metabolism have not been described previously. We investigated the association between changes in overtime working hours over 3 years and the risk for progression to type 2 diabetes among adults with pre-diabetes. We analyzed patterns of overtime working hours from 2008 to 2011 among 18,172 workers in Japan (16,474 men, aged 30 to 64 years) with pre-diabetes in 2011 (baseline) using the sub-cohort data from the Japan Epidemiology Collaboration on Occupational Health Study. Participants were followed up to March 2016. Overtime working hours per month were self-reported annually in 2008-2011 and trajectory patterns were identified using group-based trajectory modeling. Type 2 diabetes was diagnosed by fasting or random plasma glucose test, hemoglobin A1c, and history of diabetes. Multivariable-adjusted hazard ratios of incident diabetes were calculated using Cox regression. We identified 3 distinct trajectories of overtime work: persistently short, long-to-short, and persistently long. During a mean follow-up of 3.5 years, 1613 participants (8.9%) developed diabetes. Compared with persistently short overtime working hours, no material increase in diabetes risk was observed for either long-to-short working hours or persistently long working hours. After adjustment for potential confounders, this association was materially unchanged. The results suggest that among individuals with pre-diabetes, persistently long working hours over 3 years were not associated with an increased risk of developing type 2 diabetes.