Dementia Population Risk Tool (DemPoRT): study protocol for a predictive algorithm assessing dementia risk in the community

Development and validation of a population-based risk algorithm for premature mortality in Canada: the Premature Mortality Population Risk Tool (PreMPoRT)

Introduction

To develop and validate the Premature Mortality Population Risk Tool (PreMPoRT), a population-based risk algorithm that predicts the 5-year incidence of premature mortality among the Canadian adult population.

Methods

Retrospective cohort analysis used six cycles of the Canadian Community Health Survey linked to the Canadian Vital Statistics Database (2000-2017). The cohort comprised 500 870 adults (18-74 years). Predictors included sociodemographic factors, self-perceived measures, health behaviours and chronic conditions. Three models (minimal, primary and full) were developed. PreMPoRT was internally validated using a split set approach and externally validated across three hold-out cycles. Performance was assessed based on predictive accuracy, discrimination and calibration.

Results

The cohort included 267 460 females and 233 410 males. Premature deaths occurred in 1.40% of females and 2.05% of males. Primary models had 12 predictors (females) and 13 predictors (males). Shared predictors included age, income quintile, education, self-perceived health, smoking, emphysema/chronic obstructive pulmonary disease, heart disease, diabetes, cancer and stroke. Male-specific predictors were marital status, Alzheimer's disease and arthritis while female-specific predictors were body mass index and physical activity. External validation cohort differed slightly in demographics. Female model performance: split set (c-statistic: 0.852), external (c-statistic: 0.856). Male model performance: split set and external (c-statistic: 0.846). Calibration showed slight overprediction for high-risk individuals and good calibration in key subgroups.

Conclusions

PreMPoRT achieved the strongest discrimination and calibration among existing prediction models for premature mortality. The model produces reliable estimates of future incidence of premature mortality and may be used to identify subgroups who may benefit from public health interventions.

A study protocol for a predictive model to assess population-based avoidable hospitalization risk: Avoidable Hospitalization Population Risk Prediction Tool (AvHPoRT)

INTRODUCTION

Avoidable hospitalizations are considered preventable given effective and timely primary care management and are an important indicator of health system performance. The ability to predict avoidable hospitalizations at the population level represents a significant advantage for health system decision-makers that could facilitate proactive intervention for ambulatory care-sensitive conditions (ACSCs). The aim of this study is to develop and validate the Avoidable Hospitalization Population Risk Tool (AvHPoRT) that will predict the 5-year risk of first avoidable hospitalization for seven ACSCs using self-reported, routinely collected population health survey data.

METHODS AND ANALYSIS

The derivation cohort will consist of respondents to the first 3 cycles (2000/01, 2003/04, 2005/06) of the Canadian Community Health Survey (CCHS) who are 18-74 years of age at survey administration and a hold-out data set will be used for external validation. Outcome information on avoidable hospitalizations for 5 years following the CCHS interview will be assessed through data linkage to the Discharge Abstract Database (1999/2000-2017/2018) for an estimated sample size of 394,600. Candidate predictor variables will include demographic characteristics, socioeconomic status, self-perceived health measures, health behaviors, chronic conditions, and area-based measures. Sex-specific algorithms will be developed using Weibull accelerated failure time survival models. The model will be validated both using split set cross-validation and external temporal validation split using cycles 2000-2006 compared to 2007-2012. We will assess measures of overall predictive performance (Nagelkerke R2), calibration (calibration plots), and discrimination (Harrell's concordance statistic). Development of the model will be informed by the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement.

ETHICS AND DISSEMINATION

This study was approved by the University of Toronto Research Ethics Board. The predictive algorithm and findings from this work will be disseminated at scientific meetings and in peer-reviewed publications.

Development and validation of the age-associated dementia policy (AgeD-Pol) computer simulation model in the USA and Europe

OBJECTIVE

To develop and validate a novel, microsimulation model that accounts for the prevalence and incidence of age-associated dementias (AAD), disease progression and associated mortality.

DESIGN, DATA SOURCES AND OUTCOME MEASURES

We developed the AAD policy (AgeD-Pol) model, a microsimulation model to simulate the natural history, morbidity and mortality associated with AAD. We populated the model with age-stratified and sex-stratified data on AAD prevalence, AAD incidence and mortality among people with AAD. We first performed internal validation using data from the Adult Changes in Thought (ACT) cohort study. We then performed external validation of the model using data from the Framingham Heart Study, the Rotterdam Study and Kaiser Permanente Northern California (KPNC). We compared model-projected AAD cumulative incidence and mortality with published cohort data using mean absolute percentage error (MAPE) and root-mean-square error (RMSE).

RESULTS

In internal validation, the AgeD-Pol model provided a good fit to the ACT cohort for cumulative AAD incidence, 10.4% (MAPE, 0.2%) and survival, 66.5% (MAPE, 8.8%), after 16 years of follow-up among those initially aged 65-69 years. In the external validations, the model-projected lifetime cumulative incidence of AAD was 30.5%-32.4% (females) and 16.7%-23.0% (males), using data from the Framingham and Rotterdam cohorts, and AAD cumulative incidence was 21.5% over 14 years using KPNC data. Model projections demonstrated a good fit to all three cohorts (MAPE, 0.9%-9.0%). Similarly, model-projected survival provided good fit to the Rotterdam (RMSE, 1.9-3.6 among those with and without AAD) and KPNC cohorts (RMSE, 7.6-18.0 among those with AAD).

CONCLUSIONS

The AgeD-Pol model performed well when validated to published data for AAD cumulative incidence and mortality and provides a useful tool to project the AAD disease burden for health systems planning in the USA.

Development and validation of a predictive algorithm for risk of dementia in the community setting

Background

Most dementia algorithms are unsuitable for population-level assessment and planning as they are designed for use in the clinical setting. A predictive risk algorithm to estimate 5-year dementia risk in the community setting was developed.

Methods

The Dementia Population Risk Tool (DemPoRT) was derived using Ontario respondents to the Canadian Community Health Survey (survey years 2001 to 2012). Five-year incidence of physician-diagnosed dementia was ascertained by individual linkage to administrative healthcare databases and using a validated case ascertainment definition with follow-up to March 2017. Sex-specific proportional hazards regression models considering competing risk of death were developed using self-reported risk factors including information on socio-demographic characteristics, general and chronic health conditions, health behaviours and physical function.

Results

Among 75 460 respondents included in the combined derivation and validation cohorts, there were 8448 cases of incident dementia in 348 677 person-years of follow-up (5-year cumulative incidence, men: 0.044, 95% CI: 0.042 to 0.047; women: 0.057, 95% CI: 0.055 to 0.060). The final full models each include 90 df (65 main effects and 25 interactions) and 28 predictors (8 continuous). The DemPoRT algorithm is discriminating (C-statistic in validation data: men 0.83 (95% CI: 0.81 to 0.85); women 0.83 (95% CI: 0.81 to 0.85)) and well-calibrated in a wide range of subgroups including behavioural risk exposure categories, socio-demographic groups and by diabetes and hypertension status.

Conclusions

This algorithm will support the development and evaluation of population-level dementia prevention strategies, support decision-making for population health and can be used by individuals or their clinicians for individual risk assessment.

Perspective: Big Data and Machine Learning Could Help Advance Nutritional Epidemiology

The field of nutritional epidemiology faces challenges posed by measurement error, diet as a complex exposure, and residual confounding. The objective of this perspective article is to highlight how developments in big data and machine learning can help address these challenges. New methods of collecting 24-h dietary recalls and recording diet could enable larger samples and more repeated measures to increase statistical power and measurement precision. In addition, use of machine learning to automatically classify pictures of food could become a useful complimentary method to help improve precision and validity of dietary measurements. Diet is complex due to thousands of different foods that are consumed in varying proportions, fluctuating quantities over time, and differing combinations. Current dietary pattern methods may not integrate sufficient dietary variation, and most traditional modeling approaches have limited incorporation of interactions and nonlinearity. Machine learning could help better model diet as a complex exposure with nonadditive and nonlinear associations. Last, novel big data sources could help avoid unmeasured confounding by offering more covariates, including both omics and features derived from unstructured data with machine learning methods. These opportunities notwithstanding, application of big data and machine learning must be approached cautiously to ensure quality of dietary measurements, avoid overfitting, and confirm accurate interpretations. Greater use of machine learning and big data would also require substantial investments in training, collaborations, and computing infrastructure. Overall, we propose that judicious application of big data and machine learning in nutrition science could offer new means of dietary measurement, more tools to model the complexity of diet and its relations with diseases, and additional potential ways of addressing confounding.

"AI's gonna have an impact on everything in society, so it has to have an impact on public health": a fundamental qualitative descriptive study of the implications of artificial intelligence for public health

Background

Our objective was to determine the impacts of artificial intelligence (AI) on public health practice.

Methods

We used a fundamental qualitative descriptive study design, enrolling 15 experts in public health and AI from June 2018 until July 2019 who worked in North America and Asia. We conducted in-depth semi-structured interviews, iteratively coded the resulting transcripts, and analyzed the results thematically.

Results

We developed 137 codes, from which nine themes emerged. The themes included opportunities such as leveraging big data and improving interventions; barriers to adoption such as confusion regarding AI’s applicability, limited capacity, and poor data quality; and risks such as propagation of bias, exacerbation of inequity, hype, and poor regulation.

Conclusions

Experts are cautiously optimistic about AI’s impacts on public health practice, particularly for improving disease surveillance. However, they perceived substantial barriers, such as a lack of available expertise, and risks, including inadequate regulation. Therefore, investment and research into AI for public health practice would likely be beneficial. However, increased access to high-quality data, research and education regarding the limitations of AI, and development of rigorous regulation are necessary to realize these benefits.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-020-10030-x.

A study protocol for a predictive algorithm to assess population-based premature mortality risk: Premature Mortality Population Risk Tool (PreMPoRT)

BACKGROUND

Premature mortality is an important population health indicator used to assess health system functioning and to identify areas in need of health system intervention. Predicting the future incidence of premature mortality in the population can facilitate initiatives that promote equitable health policies and effective delivery of public health services. This study protocol proposes the development and validation of the Premature Mortality Risk Prediction Tool (PreMPoRT) that will predict the incidence of premature mortality using large population-based community health surveys and multivariable modeling approaches.

METHODS

PreMPoRT will be developed and validated using various training, validation, and test data sets generated from the six cycles of the Canadian Community Health Survey (CCHS) linked to the Canadian Vital Statistics Database from 2000 to 2017. Population-level risk factor information on demographic characteristics, health behaviors, area level measures, and other health-related factors will be used to develop PreMPoRT and to predict the incidence of premature mortality, defined as death prior to age 75, over a 5-year period. Sex-specific Weibull accelerated failure time models will be developed using a Canadian provincial derivation cohort consisting of approximately 500,000 individuals, with approximately equal proportion of males and females, and about 12,000 events of premature mortality. External validation will be performed using separate linked files (CCHS cycles 2007-2008, 2009-2010, and 2011-2012) from the development cohort (CCHS cycles 2000-2001, 2003-2004, and 2005-2006) to check the robustness of the prediction model. Measures of overall predictive performance (e.g., Nagelkerke's R2), calibration (e.g., calibration plots), and discrimination (e.g., Harrell's concordance statistic) will be assessed, including calibration within defined subgroups of importance to knowledge users and policymakers.

DISCUSSION

Using routinely collected risk factor information, we anticipate that PreMPoRT will produce population-based estimates of premature mortality and will be used to inform population strategies for prevention.

Development and Validation of the Chronic Disease Population Risk Tool (CDPoRT) to Predict Incidence of Adult Chronic Disease

IMPORTANCE

Predicting chronic disease incidence for the population provides a comprehensive picture to health policy makers of their jurisdictions' overall future chronic disease burden. However, no population-based risk algorithm exists for estimating the risk of first major chronic disease.

OBJECTIVE

To develop and validate the Chronic Disease Population Risk Tool (CDPoRT), a population risk algorithm that predicts the 10-year incidence of the first major chronic disease in the adult population.

DESIGN, SETTING, AND PARTICIPANTS

In this cohort study, CDPoRT was developed and validated with 6 cycles of the Canadian Community Health Survey, linked to administrative data from January 2000 to December 2014. Development and internal validation (bootstrap and split sample) of CDPoRT occurred in Ontario, Canada, from June 2018 to April 2019 followed by external validation in Manitoba from May 2019 to July 2019. The study cohorts included 133 991 adults (≥20 years) representative of the Ontario and Manitoba populations who did not have a history of major chronic disease.

EXPOSURES

Predictors were routinely collected risk factors from the Canadian Community Health Survey, such as sociodemographic factors (eg, age), modifiable lifestyle risk factors (ie, alcohol consumption, cigarette smoking, unhealthy diet, and physical inactivity), and other health-related factors (eg, body mass index).

MAIN OUTCOMES AND MEASURES

Six major chronic diseases were considered, as follows: congestive heart failure, chronic obstructive pulmonary disease, diabetes, myocardial infarction, lung cancer, and stroke. Sex-specific CDPoRT algorithms were developed with a Weibull model. Model performance was evaluated with measures of overall predictive performance (eg, Brier score), discrimination (eg, Harrell C index), and calibration (eg, calibration curves).

RESULTS

The Ontario cohort (n = 118 747) was younger (mean [SD] age, 45.6 [16.1] vs 46.3 [16.4] years), had more immigrants (23 808 [20.0%] vs 1417 [10.7%]), and had a lower mean (SD) body mass index (26.9 [5.1] vs 27.7 [5.4]) than the Manitoba cohort (n = 13 244). During development, the full and parsimonious CDPoRT models had similar Brier scores (women, 0.087; men, 0.091), Harrell C index values (women, 0.779; men, 0.783), and calibration curves. A simple version consisting of cigarette smoking, age, and body mass index performed slightly worse than the other versions (eg, Brier score for women, 0.088; for men, 0.092). Internal validation showed consistent performance across models, and CDPoRT performed well during external validation. For example, the female parsimonious version had C index values for bootstrap, split sample, and external validation of 0.778, 0.776, and 0.752, respectively.

CONCLUSIONS AND RELEVANCE

In this study, CDPoRT provided accurate, population-based risk estimates for the first major chronic disease.

Chronic Disease Population Risk Tool (CDPoRT): a study protocol for a prediction model that assesses population-based chronic disease incidence

BACKGROUND

Population-based risk prediction tools exist for individual chronic diseases. From a population health perspective, studying chronic diseases together provides a comprehensive view of the burden of disease in the population. Thus, public health officials and health policymakers would benefit from a prediction tool that measures the incidence of chronic diseases compositely. This study protocol proposes the development and validation of the Chronic Disease Population Risk Tool (CDPoRT) that will predict the incidence of six chronic diseases in the population setting using multivariable modeling techniques.

METHODS

CDPoRT will be built using population-based responses to the first six cycles of the Canadian Community Health Survey linked to health administrative data in Ontario and Manitoba from 2000 to 2014. Predictors including modifiable lifestyle risk factors (i.e., alcohol consumption, cigarette smoking, diet, and physical activity) will be used to predict time-to-chronic disease incidence (i.e., congestive heart failure, chronic obstructive pulmonary disease, diabetes, lung cancer, myocardial infarction, and stroke including transient ischemic heart attack). Sex-specific Royston-Parmar models will be used for model development and validation with death free of chronic disease as a competing risk. CDPoRT will be developed using an Ontario derivation cohort consisting of 47,960 females and 38,267 males with 7035 and 6220 chronic disease events, respectively. The model will be validated using split-sample validation using an Ontario validation cohort consisting of 20,325 females and 16,627 males with 2972 and 2658 chronic disease events, respectively. The model will be externally validated in the Manitoba validation cohort (i.e., geographic validation) expected to consist of 11,800 females and 9700 males with 1650 and 1550 chronic disease events, respectively. Measures of overall predictive accuracy (e.g., Nagelkerke's R 2), discrimination (e.g., Harrell's concordance statistic), and calibration (e.g., calibration plots) will be used to assess predictive performance.

DISCUSSION

To the extent of our knowledge, CDPoRT will be the first population-based regression prediction model that will predict the incidence of multiple chronic diseases simultaneously at the population level.