How are missing data in covariates handled in observational time-to-event studies in oncology? A systematic review

A systematic review of reporting and handling of missing data in observational studies using the UNOS database

BACKGROUND

Missing data decreasing study power and introducing bias, thereby undermining a registry's ability to draw valid inferences. We evaluated how missing data are reported and addressed in heart transplantation (HT) studies using the United Network for Organ Sharing (UNOS) database.

METHODS

We conducted a systematic literature search of Medline from January 1, 2018 through August 22, 2023 and included studies that used the UNOS database to evaluate adult (≥18 years) de novo HT recipients. We collected details on the study population, timeframe, primary end-point, use of missing data, and whether and what methods were used to handle missing data. Approaches were classified as variable selection, complete case analysis (CCA), missing indicator method, single imputation, or multiple imputation.

RESULTS

Of the 229 included studies, 67 (29.3%) limited their cohorts to those without missing data for the outcome or key variables and 93 (40.6%) reported missing data in their final cohort. 78 (34.1%) studies reported how they handled missing data in their statistical modeling. Of these, CCA was most used (n = 41, 52.6%) followed by multiple imputation (n = 22, 28.2%), and other methods (n = 15, 19.2%). Thirty-one (13.5%) studies reported removing covariates from their analysis because of missingness.

CONCLUSIONS

Merely a third of the identified UNOS database studies reported how they handled missing data in their analysis, with strategies varying. Although no singular approach to handling missing data exists, methods are available that can improve upon the most used approaches. Future best practices should include explicit reporting of missingness, detailed methods, and sensitivity checks.

Addressing immortal time bias in precision medicine: Practical guidance and methods development

OBJECTIVE

To compare theoretical strengths and limitations of common immortal time adjustment methods, propose a new approach using multiple imputation (MI), and provide practical guidance for using MI in precision medicine evaluations centered on a real-world case study.

STUDY SETTING AND DESIGN

Methods comparison, guidance, and real-world case study based on previous literature. We compared landmark analysis, time-distribution matching, time-dependent analysis, and our proposed MI application. Guidance for MI spanned (1) selecting the imputation method; (2) specifying and applying the imputation model; and (3) conducting comparative analysis and pooling estimates. Our case study used a matched cohort design to evaluate overall survival benefits of whole-genome and transcriptome analysis, a precision medicine technology, compared to usual care for advanced cancers, and applied both time-distribution matching and MI. Bootstrap simulation characterized imputation sensitivity to varying data missingness and sample sizes.

DATA SOURCES AND ANALYTIC SAMPLE

Case study used population-based administrative data and single-arm precision medicine program data from British Columbia, Canada for the study period 2012 to 2015.

PRINCIPAL FINDINGS

While each method described can reduce immortal time bias, MI offers theoretical advantages. Compared to alternative approaches, MI minimizes information loss and better characterizes statistical uncertainty about the true length of the immortal time period, avoiding false precision. Additionally, MI explicitly considers the impacts of patient characteristics on immortal time distributions, with inclusion criteria and follow-up period definitions that do not inadvertently risk biasing evaluations. In the real-world case study, survival analysis results did not substantively differ across MI and time distribution matching, but standard errors based on MI were higher for all point estimates. Mean imputed immortal time was stable across simulations.

CONCLUSIONS

Precision medicine evaluations must employ immortal time adjustment methods for unbiased, decision-grade real-world evidence generation. MI is a promising solution to the challenge of immortal time bias.

Moving Beyond Medical Statistics: A Systematic Review on Missing Data Handling in Electronic Health Records

Background: Missing data in electronic health records (EHRs) presents significant challenges in medical studies. Many methods have been proposed, but uncertainty exists regarding the current state of missing data addressing methods applied for EHR and which strategy performs better within specific contexts. Methods: All studies referencing EHR and missing data methods published from their inception until 2024 March 30 were searched via the MEDLINE, EMBASE, and Digital Bibliography and Library Project databases. The characteristics of the included studies were extracted. We also compared the performance of various methods under different missingness scenarios. Results: After screening, 46 studies published between 2010 and 2024 were included. Three missingness mechanisms were simulated when evaluating the missing data methods: missing completely at random (29/46), missing at random (20/46), and missing not at random (21/46). Multiple imputation by chained equations (MICE) was the most popular statistical method, whereas generative adversarial network-based methods and the k nearest neighbor (KNN) classification were the common deep-learning-based or traditional machine-learning-based methods, respectively. Among the 26 articles comparing the performance among medical statistical and machine learning approaches, traditional machine learning or deep learning methods generally outperformed statistical methods. Med.KNN and context-aware time-series imputation performed better for longitudinal datasets, whereas probabilistic principal component analysis and MICE-based methods were optimal for cross-sectional datasets. Conclusions: Machine learning methods show significant promise for addressing missing data in EHRs. However, no single approach provides a universally generalizable solution. Standardized benchmarking analyses are essential to evaluate these methods across different missingness scenarios.

Excess weight by degree and duration and cancer risk (ABACus2 consortium): a cohort study and individual participant data meta-analysis

Background

Elevated body mass index (BMI) ≥25 kg/m2 is a major preventable cause of cancer. A single BMI measure does not capture the degree and duration of exposure to excess BMI. We investigate associations between adulthood overweight-years, incorporating exposure time to BMI ≥25 kg/m2, and cancer incidence, and compare this with single BMI.

Methods

In this cohort study and individual participant data meta-analysis, we obtained data from the ABACus 2 Consortium, consisting of four US cohorts: Atherosclerosis Risk in Communities (ARIC) study (1987-2015), Women's Health Initiative (WHI; 1991 to 2005 [main study], to 2010 [Extension 1], and to 2020 [Extension 2]), Prostate, Lung, Colorectal, Ovarian Cancer Screening (PLCO) Trial (1993-2009), NIH-AARP Diet and Health Study (1996-2011), and one European cohort, the European Prospective Investigation into Cancer and Nutrition (EPIC; participants enrolled in 1990 and administrative censoring was centre-specific). Participants with at least 3 BMI measurements and complete cancer follow-up data were included. We calculated overweight-years: degree of overweight (BMI ≥25 kg/m2) multiplied by the duration of overweight (years). Using random effects two-stage individual participant data meta-analyses, associations between cancer and overweight-years, single BMI, cumulative overweight degree and duration, measured at the same time and captured over a median of 41 years in men and 39 years in women, were evaluated with Cox proportional hazards models. Models were age-adjusted or multivariable (MV) adjusted for baseline age, ethnicity, alcohol, smoking and hormone replacement therapy (HRT). Harrell's C-statistic of metrics were compared. This study is registered at PROSPERO, CRD42021238270.

Findings

720,210 participants, including 312,132 men and 408,078 women, were followed up for cancer incidence over a median 9.85 years (interquartile range (IQR) 8.03, 11.67) in men and 10.80 years (IQR 6.05, 15.55) in women. 12,959 men (4.15%) and 36,509 women (8.95%) were diagnosed with obesity-related cancer. Hazard ratios for obesity-related cancers in men, per 1 standard deviation (SD) overweight-years were 1.15 (95% CI: 1.14, 1.16, I2: 0) age-adjusted and 1.15 (95% CI: 1.13, 1.17, I2: 0%) MV-adjusted and per 1SD increment in single BMI were 1.17 (95% CI: 1.16, 1.18, I2: 0) age-adjusted and 1.16 (95% CI: 1.15, 1.18, I2: 0%) MV-adjusted. The HR for overweight-years in women per 1 SD increment was 1.08 (95% CI: 1.04, 1.13, I2: 82%) age-adjusted and 1.08 (95% CI: 1.04, 1.13, I2: 83%) MV-adjusted and per 1SD increment in single BMI was 1.10 (95% CI: 1.07, 1.14, I2: 72%) age-adjusted and 1.11 (95% CI: 1.07, 1.15, I2: 79%) MV-adjusted. C-statistics for overweight-years and single BMI for obesity-related cancers were 0.612 (95% CI: 0.578, 0.646) and 0.611 (95% CI: 0.578, 0.644) respectively for men and 0.566 (95% CI: 0.534, 0.598) and 0.573 (95% CI: 0.546, 0.600) for women.

Interpretation

Adulthood degree and duration of excess BMI were associated with cancer risk. Both factors should be considered in cancer prevention strategies and policies. This study only focused on adulthood exposure to excess BMI, so the minimal differences in the predictive performance between adiposity metrics may be due to underestimation of cumulative excess BMI exposure.

Funding

Cancer Research UK, the Manchester NIHR Biomedical Research Centre, the National Cancer Institute, the National Heart, Lung, and Blood Institute, National Institutes of Health, Department of Health and Human Services, U.S. Department of Health and Human Services, the Intramural Research Program of the National Cancer Institute, the International Agency for Research on Cancer, Imperial College London, European Commission (DG-SANCO), the Danish Cancer Society, Ligue Contre le Cancer, Institut Gustave-Roussy, Mutuelle Générale de l'Education Nationale, Institut National de la Santé et de la Recherche Médicale, Deutsche Krebshilfe, Deutsches Krebsforschungszentrum, German Federal Ministry of Education and Research, the Hellenic Health Foundation, Associazione Italiana per la Ricerca sul Cancro-AIRC-Italy and National Research Council, Dutch Ministry of Public Health, Welfare, and Sports, Netherlands Cancer Registry, LK Research Funds, Dutch Prevention Funds, Dutch Zorg Onderzoek Nederland, World Cancer Research Fund, Statistics Netherlands, Health Research Fund, Instituto de Salud Carlos III, regional Spanish governments of Andalucía, Asturias, Basque Country, Murcia, and Navarra, the Catalan Institute of Oncology, Swedish Cancer Society, Swedish Scientific Council, and Region Skåne and Region Västerbotten, and the Medical Research Council.

A novel MissForest-based missing values imputation approach with recursive feature elimination in medical applications

BACKGROUND

Missing values in datasets present significant challenges for data analysis, particularly in the medical field where data accuracy is crucial for patient diagnosis and treatment. Although MissForest (MF) has demonstrated efficacy in imputation research and recursive feature elimination (RFE) has proven effective in feature selection, the potential for enhancing MF through RFE integration remains unexplored.

METHODS

This study introduces a novel imputation method, "recursive feature elimination-MissForest" (RFE-MF), designed to enhance imputation quality by reducing the impact of irrelevant features. A comparative analysis is conducted between RFE-MF and four classical imputation methods: mean/mode, k-nearest neighbors (kNN), multiple imputation by chained equations (MICE), and MF. The comparison is carried out across ten medical datasets containing both numerical and mixed data types. Different missing data rates, ranging from 10 to 50%, are evaluated under the missing completely at random (MCAR) mechanism. The performance of each method is assessed using two evaluation metrics: normalized root mean squared error (NRMSE) and predictive fidelity criterion (PFC). Additionally, paired samples t-tests are employed to analyze the statistical significance of differences among the outcomes.

RESULTS

The findings indicate that RFE-MF demonstrates superior performance across the majority of datasets when compared to four classical imputation methods (mean/mode, kNN, MICE, and MF). Notably, RFE-MF consistently outperforms the original MF, irrespective of variable type (numerical or categorical). Mean/mode imputation exhibits consistent performance across various scenarios. Conversely, the efficacy of kNN imputation fluctuates in relation to varying missing data rates.

CONCLUSION

This study demonstrates that RFE-MF holds promise as an effective imputation method for medical datasets, providing a novel approach to addressing missing data challenges in medical applications.

Gaps in the usage and reporting of multiple imputation for incomplete data: findings from a scoping review of observational studies addressing causal questions

BACKGROUND

Missing data are common in observational studies and often occur in several of the variables required when estimating a causal effect, i.e. the exposure, outcome and/or variables used to control for confounding. Analyses involving multiple incomplete variables are not as straightforward as analyses with a single incomplete variable. For example, in the context of multivariable missingness, the standard missing data assumptions ("missing completely at random", "missing at random" [MAR], "missing not at random") are difficult to interpret and assess. It is not clear how the complexities that arise due to multivariable missingness are being addressed in practice. The aim of this study was to review how missing data are managed and reported in observational studies that use multiple imputation (MI) for causal effect estimation, with a particular focus on missing data summaries, missing data assumptions, primary and sensitivity analyses, and MI implementation.

METHODS

We searched five top general epidemiology journals for observational studies that aimed to answer a causal research question and used MI, published between January 2019 and December 2021. Article screening and data extraction were performed systematically.

RESULTS

Of the 130 studies included in this review, 108 (83%) derived an analysis sample by excluding individuals with missing data in specific variables (e.g., outcome) and 114 (88%) had multivariable missingness within the analysis sample. Forty-four (34%) studies provided a statement about missing data assumptions, 35 of which stated the MAR assumption, but only 11/44 (25%) studies provided a justification for these assumptions. The number of imputations, MI method and MI software were generally well-reported (71%, 75% and 88% of studies, respectively), while aspects of the imputation model specification were not clear for more than half of the studies. A secondary analysis that used a different approach to handle the missing data was conducted in 69/130 (53%) studies. Of these 69 studies, 68 (99%) lacked a clear justification for the secondary analysis.

CONCLUSION

Effort is needed to clarify the rationale for and improve the reporting of MI for estimation of causal effects from observational data. We encourage greater transparency in making and reporting analytical decisions related to missing data.

Multi-metric comparison of machine learning imputation methods with application to breast cancer survival

Handling missing data in clinical prognostic studies is an essential yet challenging task. This study aimed to provide a comprehensive assessment of the effectiveness and reliability of different machine learning (ML) imputation methods across various analytical perspectives. Specifically, it focused on three distinct classes of performance metrics used to evaluate ML imputation methods: post-imputation bias of regression estimates, post-imputation predictive accuracy, and substantive model-free metrics. As an illustration, we applied data from a real-world breast cancer survival study. This comprehensive approach aimed to provide a thorough assessment of the effectiveness and reliability of ML imputation methods across various analytical perspectives. A simulated dataset with 30% Missing At Random (MAR) values was used. A number of single imputation (SI) methods - specifically KNN, missMDA, CART, missForest, missRanger, missCforest - and multiple imputation (MI) methods - specifically miceCART and miceRF - were evaluated. The performance metrics used were Gower's distance, estimation bias, empirical standard error, coverage rate, length of confidence interval, predictive accuracy, proportion of falsely classified (PFC), normalized root mean squared error (NRMSE), AUC, and C-index scores. The analysis revealed that in terms of Gower's distance, CART and missForest were the most accurate, while missMDA and CART excelled for binary covariates; missForest and miceCART were superior for continuous covariates. When assessing bias and accuracy in regression estimates, miceCART and miceRF exhibited the least bias. Overall, the various imputation methods demonstrated greater efficiency than complete-case analysis (CCA), with MICE methods providing optimal confidence interval coverage. In terms of predictive accuracy for Cox models, missMDA and missForest had superior AUC and C-index scores. Despite offering better predictive accuracy, the study found that SI methods introduced more bias into the regression coefficients compared to MI methods. This study underlines the importance of selecting appropriate imputation methods based on study goals and data types in time-to-event research. The varying effectiveness of methods across the different performance metrics studied highlights the value of using advanced machine learning algorithms within a multiple imputation framework to enhance research integrity and the robustness of findings.

A Principled Approach to Characterize and Analyze Partially Observed Confounder Data from Electronic Health Records

Objective

Partially observed confounder data pose challenges to the statistical analysis of electronic health records (EHR) and systematic assessments of potentially underlying missingness mechanisms are lacking. We aimed to provide a principled approach to empirically characterize missing data processes and investigate performance of analytic methods.

Methods

Three empirical sub-cohorts of diabetic SGLT2 or DPP4-inhibitor initiators with complete information on HbA1c, BMI and smoking as confounders of interest (COI) formed the basis of data simulation under a plasmode framework. A true null treatment effect, including the COI in the outcome generation model, and four missingness mechanisms for the COI were simulated: completely at random (MCAR), at random (MAR), and two not at random (MNAR) mechanisms, where missingness was dependent on an unmeasured confounder and on the value of the COI itself. We evaluated the ability of three groups of diagnostics to differentiate between mechanisms: 1)-differences in characteristics between patients with or without the observed COI (using averaged standardized mean differences [ASMD]), 2)-predictive ability of the missingness indicator based on observed covariates, and 3)-association of the missingness indicator with the outcome. We then compared analytic methods including "complete case", inverse probability weighting, single and multiple imputation in their ability to recover true treatment effects.

Results

The diagnostics successfully identified characteristic patterns of simulated missingness mechanisms. For MAR, but not MCAR, the patient characteristics showed substantial differences (median ASMD 0.20 vs 0.05) and consequently, discrimination of the prediction models for missingness was also higher (0.59 vs 0.50). For MNAR, but not MAR or MCAR, missingness was significantly associated with the outcome even in models adjusting for other observed covariates. Comparing analytic methods, multiple imputation using a random forest algorithm resulted in the lowest root-mean-squared-error.

Conclusion

Principled diagnostics provided reliable insights into missingness mechanisms. When assumptions allow, multiple imputation with nonparametric models could help reduce bias.

smdi: an R package to perform structural missing data investigations on partially observed confounders in real-world evidence studies

Objectives

Partially observed confounder data pose a major challenge in statistical analyses aimed to inform causal inference using electronic health records (EHRs). While analytic approaches such as imputation are available, assumptions on underlying missingness patterns and mechanisms must be verified. We aimed to develop a toolkit to streamline missing data diagnostics to guide choice of analytic approaches based on meeting necessary assumptions.

Materials and methods

We developed the smdi (structural missing data investigations) R package based on results of a previous simulation study which considered structural assumptions of common missing data mechanisms in EHR.

Results

smdi enables users to run principled missing data investigations on partially observed confounders and implement functions to visualize, describe, and infer potential missingness patterns and mechanisms based on observed data.

Conclusions

The smdi R package is freely available on CRAN and can provide valuable insights into underlying missingness patterns and mechanisms and thereby help improve the robustness of real-world evidence studies.

Comparison between inverse-probability weighting and multiple imputation in Cox model with missing failure subtype

Identifying and distinguishing risk factors for heterogeneous disease subtypes has been of great interest. However, missingness in disease subtypes is a common problem in those data analyses. Several methods have been proposed to deal with the missing data, including complete-case analysis, inverse-probability weighting, and multiple imputation. Although extant literature has compared these methods in missing problems, none has focused on the competing risk setting. In this paper, we discuss the assumptions required when complete-case analysis, inverse-probability weighting, and multiple imputation are used to deal with the missing failure subtype problem, focusing on how to implement these methods under various realistic scenarios in competing risk settings. Besides, we compare these three methods regarding their biases, efficiency, and robustness to model misspecifications using simulation studies. Our results show that complete-case analysis can be seriously biased when the missing completely at random assumption does not hold. Inverse-probability weighting and multiple imputation estimators are valid when we correctly specify the corresponding models for missingness and for imputation, and multiple imputation typically shows higher efficiency than inverse-probability weighting. However, in real-world studies, building imputation models for the missing subtypes can be more challenging than building missingness models. In that case, inverse-probability weighting could be preferred for its easy usage. We also propose two automated model selection procedures and demonstrate their usage in a study of the association between smoking and colorectal cancer subtypes in the Nurses' Health Study and Health Professional Follow-Up Study.

Predicting lung cancer survival prognosis based on the conditional survival bayesian network

Lung cancer is a leading cause of cancer deaths and imposes an enormous economic burden on patients. It is important to develop an accurate risk assessment model to determine the appropriate treatment for patients after an initial lung cancer diagnosis. The Cox proportional hazards model is mainly employed in survival analysis. However, real-world medical data are usually incomplete, posing a great challenge to the application of this model. Commonly used imputation methods cannot achieve sufficient accuracy when data are missing, so we investigated novel methods for the development of clinical prediction models. In this article, we present a novel model for survival prediction in missing scenarios. We collected data from 5,240 patients diagnosed with lung cancer at the Weihai Municipal Hospital, China. Then, we applied a joint model that combined a BN and a Cox model to predict mortality risk in individual patients with lung cancer. The established prognostic model achieved good predictive performance in discrimination and calibration. We showed that combining the BN with the Cox proportional hazards model is highly beneficial and provides a more efficient tool for risk prediction.

Handling missing covariates in observational studies: an illustration with the assessment of prognostic factors of survival outcomes in soft-tissue or visceral sarcomas in irradiated fields (SIF)

Background

Missing covariates are common in observational research and can lead to bias and loss of statistical power. Limited data regarding prognostic factors of survival outcomes of sarcomas in irradiated fields (SIF) are available. Because of the long lag time between irradiation of first cancer and scarcity of SIF, missing data are a critical issue when analyzing long-term outcomes. We assessed prognostic factors of overall (OS), progression-free (PFS), and metastatic-progression-free (MPFS) survivals in SIF using three methods to account for missing covariates.

Methods

We relied on the NETSARC French Sarcoma Group database, Cox (OS/PFS), and competitive hazards (MPFS) survival models. Covariates investigated were age, sex, histological subtype, tumor size, depth and grade, metastasis, surgery, surgical resection, surgeon's expertise, imaging, and neo-adjuvant treatment. We first applied multiple imputation (MI): observed data were used to estimate the missing covariate. With the missing-data modality approach, a category missing was created for qualitative variables. With the complete-case (CC) approach, analysis was restricted to patients without missing covariates.

Results

CC subjects (N = 167; 33%) presented more often with soft-tissue sarcoma (versus visceral sarcoma) and grade I-II tumors as compared to the 504 eligible cases. With MI (N = 504), factors associated with the worst outcome included metastasis (p = 0.04) and R1/R2 resection (p < 0.001) for OS; higher grade/non-gradable tumors (p = 0.002) and R1/R2 resection (p < 0.001) for PFS; and metastasis (p = 0.01) for M-PFS. The 'missing-data modality' approach (N = 504) led to different associations, including significance reached due to variables with the modality 'missing'. The CC analysis led to different results and reduced precision.

Conclusion

The CC population was not representative of the eligible population, introducing bias, in addition to worst precision. The 'missing-data modality method' results in biased estimates in non-randomized studies, as outcomes may be related to variables with missing values. Appropriate statistical methods for missing covariates, for example, MI, should therefore be considered.

Intake of polyphenols from cereal foods and colorectal cancer risk in the Melbourne Collaborative Cohort Study

BACKGROUND

Cereal-derived polyphenols have demonstrated protective mechanisms in colorectal cancer (CRC) models; however, confirmation in human studies is lacking. Therefore, this study examined the association between cereal polyphenol intakes and CRC risk in the Melbourne Collaborative Cohort Study (MCCS), a prospective cohort study in Melbourne, Australia that recruited participants between 1990 and 1994 to investigate diet-disease relationships.

METHODS

Using food frequency questionnaire diet data matched to polyphenol data, dietary intakes of alkylresorcinols, phenolic acids, lignans, and total polyphenols from cereals were estimated. Hazard ratios (HRs) and 95% confidence intervals for CRC risk were estimated for quintiles of intake with the lowest quintile as the comparison category, using multivariable adjusted Cox proportional hazards models with age as the time axis adjusted for sex, socio-economic status, alcohol consumption, fibre intake, country of birth, total energy intake, physical activity and smoking status.

RESULTS

From 35,245 eligible adults, mean (SD) age 54.7 (8.6) years, mostly female (61%) and Australian-born (69%), there were 1394 incident cases of CRC (946 colon cancers and 448 rectal cancers). Results for total cereal polyphenol intake showed reduced HRs in Q2 (HR: 0.80; 95% CI, 0.68-0.95) and Q4 (HR: 0.75; 95% CI, 0.62-0.90), and similar for phenolic acids. Alkylresorcinol intake showed reduced HR in Q3 (HR: 0.80; 95% CI, 0.67-0.95) and Q4 (HR: 0.79; 95% CI, 0.66-0.95).

CONCLUSIONS

Overall, the present study showed little evidence of association between intakes of cereal polyphenols and CRC risk. Future investigations may be useful to understand associations between cereal-derived polyphenols and additional cancers in different populations.

A systematic approach towards missing lab data in electronic health records: A case study in non-small cell lung cancer and multiple myeloma

Real-world data derived from electronic health records often exhibit high levels of missingness in variables, such as laboratory results, presenting a challenge for statistical analyses. We developed a systematic workflow for gathering evidence of different missingness mechanisms and performing subsequent statistical analyses. We quantify evidence for missing completely at random (MCAR) or missing at random (MAR), mechanisms using Hotelling's multivariate t-test, and random forest classifiers, respectively. We further illustrate how to apply sensitivity analyses using the not at random fully conditional specification procedure to examine changes in parameter estimates under missing not at random (MNAR) mechanisms. In simulation studies, we validated these diagnostics and compared analytic bias under different mechanisms. To demonstrate the application of this workflow, we applied it to two exemplary case studies with an advanced non-small cell lung cancer and a multiple myeloma cohort derived from a real-world oncology database. Here, we found strong evidence against MCAR, and some evidence of MAR, implying that imputation approaches that attempt to predict missing values by fitting a model to observed data may be suitable for use. Sensitivity analyses did not suggest meaningful departures of our analytic results under potential MNAR mechanisms; these results were also in line with results reported in clinical trials.

Handling missing covariate data in clinical studies in haematology

Missing data are frequently encountered across studies in clinical haematology. Failure to handle these missing values in an appropriate manner can complicate the interpretation of a study's findings, as estimates presented may be biased and/or imprecise. In the present work, we first provide an overview of current methods for handling missing covariate data, along with their advantages and disadvantages. Furthermore, a systematic review is presented, exploring both contemporary reporting of missing values in major haematological journals, and the methods used for handling them. A principal finding was that the method of handling missing data was explicitly specified in a minority of articles (in 76 out of 195 articles reporting missing values, 39%). Among these, complete case analysis and the missing indicator method were the most common approaches to dealing with missing values, with more complex methods such as multiple imputation being extremely rare (in 7 out of 195 articles). An example analysis (with associated code) is also provided using hematopoietic stem cell transplantation data, illustrating the different approaches to handling missing values. We conclude with various recommendations regarding the reporting and handling of missing values for future studies in clinical haematology.

Differences in county-level cardiovascular disease mortality rates due to damage caused by hurricane Matthew and the moderating effect of social capital: a natural experiment

BACKGROUND

As the climate continues to warm, hurricanes will continue to increase in both severity and frequency. Hurricane damage is associated with cardiovascular events, but social capital may moderate this relationship. Social capital is a multidimensional concept with a rich theoretical tradition. Simply put, social capital refers to the social relationships and structures that provide individuals with material, financial, and emotional resources throughout their lives. Previous research has found an association between high levels of social capital and lower rates of cardiovascular (CVD) mortality. In post-disaster settings, social capital may protect against CVD mortality by improving access to life-saving resources. We examined the association between county-level hurricane damage and CVD mortality rates after Hurricane Matthew, and the moderating effect of several aspects of social capital and hurricane damage on this relationship. We hypothesized that (1) higher (vs. lower) levels of hurricane damage would be associated with increased CVD mortality rates and (2) in highly damaged counties, higher (vs. lower) levels of social capital would be associated with lower CVD mortality.

METHODS

Analysis used yearly (2013-2018) county-level sociodemographic and epidemiological data (n = 183). Sociodemographic data were compiled from federal surveys before and after Hurricane Matthew to construct, per prior literature, a social capital index based on four dimensions of social capital (sub-indices): family unity, informal civil society, institutional confidence, and collective efficacy. Epidemiological data comprised monthly CVD mortality rates constructed from monthly county-level CVD death counts from the CDC WONDER database and the US Census population estimates. Changes in CVD mortality based on level of hurricane damage were assessed using regression adjustment. We used cluster robust Poisson population average models to determine the moderating effect of social capital on CVD mortality rates in both high and low-damage counties.

RESULTS

We found that mean levels of CVD mortality increased (before and after adjustment for sociodemographic controls) in both low-damage counties (unadjusted. Mean = 2.50, 95% CI [2.41, 2.59], adjusted mean = 2.50, 95% CI [2.40, 2.72]) and high-damage counties (mean = 2.44, CI [2.29, 2.46], adj. Mean = 2.51, 95% CI [2.49, 2.84]). Among the different social capital dimensions, institutional confidence was associated with reduced initial CVD mortality in low-damage counties (unadj. IRR 1.00, 95% CI [0.90, 1.11], adj. IRR 0.91 CI [0.87, 0.94]), but its association with CVD mortality trends was null. The overall effects of social capital and its sub-indices were largely nonsignificant.

CONCLUSION

Hurricane damage is associated with increased CVD mortality for 18 months after Hurricane Matthew. The role of social capital remains unclear. Future research should focus on improving measurement of social capital and quality of hurricane damage and CVD mortality data.

Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models

Risk prediction models need thorough validation to assess their performance. Validation of models for survival outcomes poses challenges due to the censoring of observations and the varying time horizon at which predictions can be made. This article describes measures to evaluate predictions and the potential improvement in decision making from survival models based on Cox proportional hazards regression.

As a motivating case study, the authors consider the prediction of the composite outcome of recurrence or death (the "event") in patients with breast cancer after surgery. They developed a simple Cox regression model with 3 predictors, as in the Nottingham Prognostic Index, in 2982 women (1275 events over 5 years of follow-up) and externally validated this model in 686 women (285 events over 5 years). Improvement in performance was assessed after the addition of progesterone receptor as a prognostic biomarker.

The model predictions can be evaluated across the full range of observed follow-up times or for the event occurring by the end of a fixed time horizon of interest. The authors first discuss recommended statistical measures that evaluate model performance in terms of discrimination, calibration, or overall performance. Further, they evaluate the potential clinical utility of the model to support clinical decision making according to a net benefit measure. They provide SAS and R code to illustrate internal and external validation.

The authors recommend the proposed set of performance measures for transparent reporting of the validity of predictions from survival models.

Strategies for data normalization and missing data imputation and consequences for potential diagnostic microRNA biomarkers in epithelial ovarian cancer

MicroRNAs (miRNAs) are small non-coding RNA molecules regulating gene expression with diagnostic potential in different diseases, including epithelial ovarian carcinomas (EOC). As only a few studies have been published on the identification of stable endogenous miRNA in EOC, there is no consensus which miRNAs should be used aiming standardization. Currently, U6-snRNA is widely adopted as a normalization control in RT-qPCR when investigating miRNAs in EOC; despite its variable expression across cancers being reported. Therefore, our goal was to compare different missing data and normalization approaches to investigate their impact on the choice of stable endogenous controls and subsequent survival analysis while performing expression analysis of miRNAs by RT-qPCR in most frequent subtype of EOC: high-grade serous carcinoma (HGSC). 40 miRNAs were included based on their potential as stable endogenous controls or as biomarkers in EOC. Following RNA extraction from formalin-fixed paraffin embedded tissues from 63 HGSC patients, RT-qPCR was performed with a custom panel covering 40 target miRNAs and 8 controls. The raw data was analyzed by applying various strategies regarding choosing stable endogenous controls (geNorm, BestKeeper, NormFinder, the comparative ΔCt method and RefFinder), missing data (single/multiple imputation), and normalization (endogenous miRNA controls, U6-snRNA or global mean). Based on our study, we propose hsa-miR-23a-3p and hsa-miR-193a-5p, but not U6-snRNA as endogenous controls in HGSC patients. Our findings are validated in two external cohorts retrieved from the NCBI Gene Expression Omnibus database. We present that the outcome of stability analysis depends on the histological composition of the cohort, and it might suggest unique pattern of miRNA stability profiles for each subtype of EOC. Moreover, our data demonstrates the challenge of miRNA data analysis by presenting various outcomes from normalization and missing data imputation strategies on survival analysis.

Robustness of Multiple Imputation Methods for Missing Risk Factor Data from Electronic Medical Records for Observational Studies

Evaluating appropriate methodologies for imputation of missing outcome data from electronic medical records (EMRs) is crucial but lacking for observational studies. Using US EMR in people with type 2 diabetes treated over 12 and 24 months with dipeptidyl peptidase 4 inhibitors (DPP-4i, n = 38,483) and glucagon-like peptide 1 receptor agonists (GLP-1RA, n = 8,977), predictors of missingness of disease biomarker (HbA1c) were explored. Robustness of multiple imputation (MI) by chained equations, two-fold MI (MI-2F) and MI with Monte Carlo Markov Chain were compared to complete case analyses for drawing inferences. Compared to younger people (age quartile Q1), those in age quartile Q3 and Q4 were less likely to have missing HbA1c by 25-32% (range of OR CI: 0.55-0.88) at 6-month follow-up and by 26-39% (range of OR CI: 0.50-0.80) at 12-month follow-up. People with HbA1c ≥ 7.5% at baseline were 12% (OR CI: 0.83, 0.93) and 14% (OR CI: 0.77, 0.97) less likely to have missing data at 6-month follow-up in the DPP-4i and GLP-1RA groups, respectively. All imputation methods provided similar HbA1c distributions during follow-up as observed with complete case analyses. The clinical inferences based on absolute change in HbA1c and by proportion of people reducing HbA1c to a clinically acceptable level (≤ 7%) were also similar between imputed data and complete case analyses. MI-2F method provided marginally smaller mean difference between observed and imputed data with relatively smaller standard error of difference, compared to other methods, while evaluating for consistency through artificial within-sample analyses. The established MI techniques can be reliably employed for missing outcome data imputations in large EMR-based relational databases, leading to efficiently designing and drawing robust clinical inferences in pharmaco-epidemiological studies.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-022-00119-w.

Predicting incident heart failure among patients with type 2 diabetes mellitus: The DM-CURE risk score

AIM

Early identification and prediction of incident heart failure (HF) is important because of severe morbidity and mortality. This study aimed to predict onset of HF among patients with diabetes.

METHODS

A time-varying Cox model was derived from ACCORD clinical trial to predict the risk of incident HF, defined by hospitalization for HF (HHF). External validation was performed on patient-level data from the Harmony Outcome trial and Chronic Renal Insufficiency Cohort (CRIC) study. The model was transformed into an integer-based scoring algorithm for 10-year risk evaluation. A stepwise algorithm identified and selected predictors from demographic characteristics, physical examination, laboratory results, medical history, medication and health care utilization, to develop a risk prediction model. The main outcome was incident HF, defined by HHF. The C statistic and Brier score were used to assess model performance.

RESULTS

In total, 9649 patients with diabetes free of HF were used, with median follow-up of 4 years and 299 incident hospitalization of HF events. The model identified several predictors for the 10-year HF incidence risk score 'DM-CURE': socio-Demographic [education, age at type 2 diabetes (T2DM) diagnosis], Metabolic (glycated haemoglobin, systolic blood pressure, body mass index, high-density lipoproteins), diabetes-related Complications (myocardial infarction, revascularization, cardiovascular medications, neuropathy, hypertension duration, albuminuria, urine albumin-to-creatinine ratio, End Stage Kidney Disease), and health care Utilization (all-cause hospitalization, emergency room visits) for Risk Evaluation. Among them, the strongest impact factors for future HF were age at T2DM diagnosis, health care utilization and cardiovascular disease-related variables. The model showed good discrimination (C statistic: 0.838, 95% CI: 0.821-0.855) and calibration (Brier score: 0.006, 95% CI: 0.006-0.007) in the ACCORD data and good performance in the validation data (Harmony: C statistic: 0.881, 95% CI: 0.863-0.899; CRIC: C statistic: 0.813, 95% CI: 0.794-0.833). The 10-year risk of incident HF increased in a graded fashion, from ≤1% in quintile 1 (score ≤14), 1%-5% in quintile 2 (score 15-23), 5%-10% in quintile 3 (score 24-27), 10%-20% in quintile 4 (score 28-33) and ≥20% in quintile 5 (score >33).

CONCLUSIONS

The DM-CURE model and score were useful for population risk stratification of incident HHF among patients with T2DM and can be easily applied in clinical practice.

Multiple imputation for cause-specific Cox models: Assessing methods for estimation and prediction

In studies analyzing competing time-to-event outcomes, interest often lies in both estimating the effects of baseline covariates on the cause-specific hazards and predicting cumulative incidence functions. When missing values occur in these baseline covariates, they may be discarded as part of a complete-case analysis or multiply imputed. In the latter case, the imputations may be performed either compatibly with a substantive model pre-specified as a cause-specific Cox model [substantive model compatible fully conditional specification (SMC-FCS)], or approximately so [multivariate imputation by chained equations (MICE)]. In a large simulation study, we assessed the performance of these three different methods in terms of estimating cause-specific regression coefficients and predicting cumulative incidence functions. Concerning regression coefficients, results provide further support for use of SMC-FCS over MICE, particularly when covariate effects are large and the baseline hazards of the competing events are substantially different. Complete-case analysis also shows adequate performance in settings where missingness is not outcome dependent. With regard to cumulative incidence prediction, SMC-FCS and MICE are performed more similarly, as also evidenced in the illustrative analysis of competing outcomes following a hematopoietic stem cell transplantation. The findings are discussed alongside recommendations for practising statisticians.

Precise Estimation for the Age of Initiation of Tobacco Use Among U.S. Youth: Finding from the Population Assessment of Tobacco and Health (PATH) Study, 2013-2017

Context

Youth tobacco use remains a prominent United States public health issue with a high economic and health burden.

Method

We pooled never and ever users at youth's first wave of PATH participation (waves 1-3) to estimate age of initiation for hookah, e-cigarettes, cigarettes, traditional cigars, cigarillos, and smokeless tobacco prospectively (waves 2-4). Age of initiation of each tobacco product was estimated using weighted interval-censored survival analyses. Weighted interval censoring Cox-proportional hazards regression models were used to assess the association of ever use of the TP at the first wave of PATH participation, sex, and race/ethnicity on the age of initiation of ever use of each tobacco product. Sensitivity analyses were performed to understand the impact of the recalled age of initiation for the left-censored participants by replacing the recalled age of initiation with a uniform "6" years lower bound.

Results

The proportion of those who ever used each tobacco product at the first wave of PATH participation ranged from 1.8% for traditional cigars to 10.4% for cigarettes. There was a significant increase in ever use of each tobacco product after the age of 14, with e-cigarettes and cigarettes showing the highest cumulative incidence of initiation by age 21, while smokeless and cigarillos recorded the lowest cumulative incidence by age 21. The adjusted Cox models showed boys initiated at earlier ages for all of these tobacco products except for hookah, which showed no difference. Similarly, apart from ever use of hookah, non-Hispanic White youth were more likely to initiate each tobacco product at earlier ages compared to Hispanic, non-Hispanic Black, and non-Hispanic Other youth.

Conclusion

The increased sample size and the inclusion of ever users yielded greater precision for age of initiation of each tobacco product than analyses limited to never users at the first wave of PATH participation. These analyses can help elucidate population selection criteria for estimating the age of initiation of tobacco products.

Social isolation, loneliness, and all-cause mortality: A cohort study of 35,254 Chinese older adults

BACKGROUND

Few studies of social isolation, loneliness and associations with all-cause mortality in older adults have been conducted in non-Western countries. The aim of this study was to conduct such an analysis in a nationally representative sample of Chinese older adults.

METHODS

This study used eight waves of data from the Chinese Longitudinal Healthy Longevity Survey from 1998 to 2018 and focused on participants aged ≥60 years. A total of 21,570 people died (61.2%) over a median follow-up of 4.8 years. Social isolation, loneliness, demographic, health and lifestyle factors were measured at baseline. The primary outcome was all-cause mortality. Cox proportional hazard regression models were used to examine the associations of isolation and loneliness with all-cause mortality.

RESULTS

This study included 35,254 participants with mean age of 86.63 ± 11.39 years. Social isolation was significantly associated with an increased mortality (adjusted HR 1.22; 95% CI 1.18-1.25; p < 0.01). The association of loneliness with mortality was nonsignificant after adjustment for health indicators and low psychological well-being (HR 1.01; 95% CI 0.98-1.04; p = 0.69). However, when stratified by age, there was a significant association of loneliness with mortality among participants aged <80 years (HR 1.15; 95% CI 1.05-1.26; p < 0.01).

CONCLUSIONS

Social isolation was associated with an increased all-cause mortality among the older Chinese adults. However, loneliness was associated with an increased mortality only among younger participants. Public health interventions aimed at increasing social connectedness may potentially reduce excess mortality among older adults.

The reporting and handling of missing data in longitudinal studies of older adults is suboptimal: a methodological survey of geriatric journals

Background

Missing data are common in longitudinal studies, and more so, in studies of older adults, who are susceptible to health and functional decline that limit completion of assessments. We assessed the extent, current reporting, and handling of missing data in longitudinal studies of older adults.

Methods

Medline and Embase databases were searched from 2015 to 2019 for publications on longitudinal observational studies conducted among persons ≥55 years old. The search was restricted to 10 general geriatric journals published in English. Reporting and handling of missing data were assessed using questions developed from the recommended standards. Data were summarised descriptively as frequencies and proportions.

Results

A total of 165 studies were included in the review from 7032 identified records. In approximately half of the studies 97 (62.5%), there was either no comment on missing data or unclear descriptions. The percentage of missing data varied from 0.1 to 55%, with a 14% average among the studies that reported having missing data. Complete case analysis was the most common method for handling missing data with nearly 75% of the studies (n = 52) excluding individual observations due to missing data, at the initial phase of study inclusion or at the analysis stage. Of the 10 studies where multiple imputation was used, only 1 (10.0%) study followed the guideline for reporting the procedure fully using online supplementary documents.

Conclusion

The current reporting and handling of missing data in longitudinal observational studies of older adults are inadequate. Journal endorsement and implementation of guidelines may potentially improve the quality of missing data reporting. Further, authors should be encouraged to use online supplementary files to provide additional details on how missing data were addressed, to allow for more transparency and comprehensive appraisal of studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01605-w.

Comparing methods for handling missing cost and quality of life data in the Early Endovenous Ablation in Venous Ulceration trial

Objectives

This study compares methods for handling missing data to conduct cost-effectiveness analysis in the context of a clinical study.

Methods

Patients in the Early Endovenous Ablation in Venous Ulceration (EVRA) trial had between 1 year and 5.5 years (median 3 years) of follow-up under early or deferred endovenous ablation. This study compares complete-case-analysis (CCA), multiple imputation using linear regression (MILR) and using predictive mean matching (MIPMM), Bayesian parametric approach using the R package missingHE (BPA), repeated measures fixed effect (RMFE) and repeated measures mixed model (RMM). The outcomes were total mean costs and total mean quality-adjusted life years (QALYs) at different time horizons (1 year, 3 years and 5 years).

Results

All methods found no statistically significant difference in cost at the 5% level in all time horizons, and all methods found statistically significantly greater mean QALY at year 1. By year 3, only BPA showed a statistically significant difference in QALY between treatments. Standard errors differed substantially between the methods employed.

Conclusion

CCA can be biased if data are MAR and is wasteful of the data. Hence the results for CCA are likely to be inaccurate. Other methods coincide in suggesting that early intervention is cost-effective at a threshold of £30,000 per QALY 1, 3 and 5 years. However, the variation in the results across the methods does generate some additional methodological uncertainty, underlining the importance of conducting sensitivity analyses using alternative approaches.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12962-022-00351-6.

Assessing proximate intermediates between ambient temperature, hospital admissions, and mortality in hemodialysis patients

BACKGROUND

Typical thermoregulatory responses to elevated temperatures among healthy individuals include reduced blood pressure and perspiration. Individuals with end-stage kidney disease (ESKD) are susceptible to systemic fluctuations caused by ambient temperature changes that may increase morbidity and mortality. We investigated whether pre-dialysis systolic blood pressure (preSBP) and interdialytic weight gain (IDWG) can independently mediate the association between ambient temperature, all-cause hospital admissions (ACHA), and all-cause mortality (ACM).

METHODS

The study population consisted of ESKD patients receiving hemodialysis treatments at Fresenius Medical Care facilities in Philadelphia County, PA, from 2011 to 2019 (n = 1981). Within a time-to-event framework, we estimated the association between daily maximum dry-bulb temperature (TMAX) and, as separate models, ACHA and ACM during warmer calendar months. Clinically measured preSBP and IDWG responses to temperature increases were estimated using linear mixed effect models. We employed the difference (c-c') method to decompose total effect models for ACHA and ACM using preSBP and IDWG as time-dependent mediators. Covariate adjustments for exposure-mediator and total and direct effect models include age, race, ethnicity, blood pressure medication use, treatment location, preSBP, and IDWG. We considered lags up to two days for exposure and 1-day lag for mediator variables (Lag 2-Lag 1) to assure temporality between exposure-outcome models. Sensitivity analyses for 2-day (Lag 2-only) and 1-day (Lag 1-only) lag structures were also conducted.

RESULTS

Based on Lag 2- Lag 1 temporal ordering, 1 °C increase in daily TMAX was associated with increased hazard of ACHA by 1.4% (adjusted hazard ratio (HR), 1.014; 95% confidence interval, 1.007-1.021) and ACM 7.5% (adjusted HR, 1.075, 1.050-1.100). Short-term lag exposures to 1 °C increase in temperature predicted mean reductions in IDWG and preSBP by 0.013-0.015% and 0.168-0.229 mmHg, respectively. Mediation analysis for ACHA identified significant indirect effects for all three studied pathways (preSBP, IDWG, and preSBP + IDWG) and significant indirect effects for IDWG and conjoined preSBP + IDWG pathways for ACM. Of note, only 1.03% of the association between temperature and ACM was mediated through preSBP. The mechanistic path for IDWG, independent of preSBP, demonstrated inconsistent mediation and, consequently, potential suppression effects in ACHA (-15.5%) and ACM (-6.3%) based on combined pathway models. Proportion mediated estimates from preSBP + IDWG pathways achieved 2.2% and 0.3% in combined pathway analysis for ACHA and ACM outcomes, respectively. Lag 2 discrete-time ACM mediation models exhibited consistent mediation for all three pathways suggesting that 2-day lag in IDWG and preSBP responses can explain 2.11% and 4.41% of total effect association between temperature and mortality, respectively.

CONCLUSION

We corroborated the previously reported association between ambient temperature, ACHA and ACM. Our results foster the understanding of potential physiological linkages that may explain or suppress temperature-driven hospital admissions and mortality risks. Of note, concomitant changes in preSBP and IDWG may have little intermediary effect when analyzed in combined pathway models. These findings advance our assessment of candidate interventions to reduce the impact of outdoor temperature change on ESKD patients.

Toward a framework for the design, implementation, and reporting of methodology scoping reviews

BACKGROUND AND OBJECTIVE

In view of the growth of published articles, there is an increasing need for studies that summarize scientific research. An increasingly common review is a "methodology scoping review," which provides a summary of existing analytical methods, techniques and software that have been proposed or applied in research articles to address an analytical problem or further an analytical approach. However, guidelines for their design, implementation, and reporting are limited.

METHODS

Drawing on the experiences of the authors, which were consolidated through a series of face-to-face workshops, we summarize the challenges inherent in conducting a methodology scoping review and offer suggestions of best practice to promote future guideline development.

RESULTS

We identified three challenges of conducting a methodology scoping review. First, identification of search terms; one cannot usually define the search terms a priori, and the language used for a particular method can vary across the literature. Second, the scope of the review requires careful consideration because new methodology is often not described (in full) within abstracts. Third, many new methods are motivated by a specific clinical question, where the methodology may only be documented in supplementary materials. We formulated several recommendations that build upon existing review guidelines. These recommendations ranged from an iterative approach to defining search terms through to screening and data extraction processes.

CONCLUSION

Although methodology scoping reviews are an important aspect of research, there is currently a lack of guidelines to standardize their design, implementation, and reporting. We recommend a wider discussion on this topic.