Comparison of techniques for handling missing covariate data within prognostic modelling studies: a simulation study

Early inflammatory markers as prognostic indicators following allogeneic stem cell transplantation

Allogeneic stem cell transplantation is used widely in the treatment of hematopoietic malignancy although graft versus host disease and relapse remain major complications. We measured the serum protein expression of 92 inflammation-related markers from 49 patients at Day 0 (D0) and 154 patients at Day 14 (D14) following transplantation and related values to subsequent clinical outcomes. Low levels of 7 proteins at D0 were linked to GvHD whilst high levels of 7 proteins were associated with relapse. The concentration of 38 proteins increased over 14 days and higher inflammatory response at D14 was strongly correlated with patient age. A marked increment in protein concentration during this period associated with GvHD but reduced risk of disease relapse, indicating a link with alloreactive immunity. In contrast, patients who demonstrated low dynamic elevation of inflammatory markers during the first 14 days were at increased risk of subsequent disease relapse. Multivariate time-to-event analysis revealed that high CCL23 at D14 was associative of AGvHD, CXCL10 with reduced rate of relapse, and high PD-L1 with reduced overall survival. This work identifies a dynamic pattern of inflammatory biomarkers in the very early post-transplantation period and reveals early protein markers that may help to guide patient management.

A Network Model of Health-Related Changes after a Lifestyle-Enhancing Treatment in Patients with Severe Mental Illness: the MULTI Study VI

Background/Objective

The effects of lifestyle interventions on physical and mental health in people with severe mental illness (SMI) are promising, but its underlying mechanisms remain unsolved. This study aims to examine changes in health-related outcomes after a lifestyle intervention, distinguishing between direct and indirect effects.

Method

We applied network intervention analysis on data from the 18-month cohort Multidisciplinary Lifestyle enhancing Treatment for Inpatients with SMI (MULTI) study in 106 subjects (62% male, mean age=54.7 (SD=10.8)) that evaluated changes in actigraphy-measured physical activity, metabolic health, psychopathology, psychosocial functioning, quality of life and medication use after MULTI (n=65) compared to treatment as usual (n=41).

Results

MULTI is directly connected to decreased negative symptoms and psychotropic medication dosage, and improved physical activity and psychosocial functioning, suggesting a unique and direct association between MULTI and the different outcome domains. Secondly, we identified associations between outcomes within the same domain (e.g., metabolic health) and between the domains (e.g., metabolic health and social functioning), suggesting potential indirect effects of MULTI.

Conclusions

This novel network approach shows that MULTI has direct and indirect associations with various health-related outcomes. These insights contribute to the development of effective treatment strategies in people with severe mental illness.

Comparison of Imputation Strategies for Incomplete Longitudinal Data in Life-Course Epidemiology

Incomplete longitudinal data are common in life-course epidemiology and may induce bias leading to incorrect inference. Multiple imputation (MI) is increasingly preferred for handling missing data, but few studies explore MI-method performance and feasibility in real-data settings. We compared 3 MI methods using real data under 9 missing-data scenarios, representing combinations of 10%, 20%, and 30% missingness and missing completely at random, at random, and not at random. Using data from Health and Retirement Study (HRS) participants, we introduced record-level missingness to a sample of participants with complete data on depressive symptoms (1998-2008), mortality (2008-2018), and relevant covariates. We then imputed missing data using 3 MI methods (normal linear regression, predictive mean matching, variable-tailored specification), and fitted Cox proportional hazards models to estimate effects of 4 operationalizations of longitudinal depressive symptoms on mortality. We compared bias in hazard ratios, root mean square error, and computation time for each method. Bias was similar across MI methods, and results were consistent across operationalizations of the longitudinal exposure variable. However, our results suggest that predictive mean matching may be an appealing strategy for imputing life-course exposure data, given consistently low root mean square error, competitive computation times, and few implementation challenges.

Body Mass Index, Adverse Pregnancy Outcomes, and Cardiovascular Disease Risk

BACKGROUND

Obesity is a well-established risk factor for both adverse pregnancy outcomes (APOs) and cardiovascular disease (CVD). However, it is not known whether APOs are mediators or markers of the obesity-CVD relationship. This study examined the association between body mass index, APOs, and postpartum CVD risk factors.

METHODS

The sample included adults from the nuMoM2b (Nulliparous Pregnancy Outcomes Study: Monitoring Mothers-To-Be) Heart Health Study who were enrolled in their first trimester (6 weeks-13 weeks 6 days gestation) from 8 United States sites. Participants had a follow-up visit at 3.7 years postpartum. APOs, which included hypertensive disorders of pregnancy, preterm birth, small-for-gestational-age birth, and gestational diabetes, were centrally adjudicated. Mediation analyses estimated the association between early pregnancy body mass index and postpartum CVD risk factors (hypertension, hyperlipidemia, and diabetes) and the proportion mediated by each APO adjusted for demographics and baseline health behaviors, psychosocial stressors, and CVD risk factor levels.

RESULTS

Among 4216 participants enrolled, mean±SD maternal age was 27±6 years. Early pregnancy prevalence of overweight was 25%, and obesity was 22%. Hypertensive disorders of pregnancy occurred in 15%, preterm birth in 8%, small-for-gestational-age birth in 11%, and gestational diabetes in 4%. Early pregnancy obesity, compared with normal body mass index, was associated with significantly higher incidence of postpartum hypertension (adjusted odds ratio, 1.14 [95% CI, 1.10-1.18]), hyperlipidemia (1.11 [95% CI, 1.08-1.14]), and diabetes (1.03 [95% CI, 1.01-1.04]) even after adjustment for baseline CVD risk factor levels. APOs were associated with higher incidence of postpartum hypertension (1.97 [95% CI, 1.61-2.40]) and hyperlipidemia (1.31 [95% CI, 1.03-1.67]). Hypertensive disorders of pregnancy mediated a small proportion of the association between obesity and incident hypertension (13% [11%-15%]) and did not mediate associations with incident hyperlipidemia or diabetes. There was no significant mediation by preterm birth or small-for-gestational-age birth.

CONCLUSIONS

There was heterogeneity across APO subtypes in their association with postpartum CVD risk factors and mediation of the association between early pregnancy obesity and postpartum CVD risk factors. However, only a small or nonsignificant proportion of the association between obesity and CVD risk factors was mediated by any of the APOs, suggesting APOs are a marker of prepregnancy CVD risk and not a predominant cause of postpartum CVD risk.

Risk-prediction models for intravenous immunoglobulin resistance in Kawasaki disease: Risk-of-Bias Assessment using PROBAST

BACKGROUND

The prediction model of intravenous immunoglobulin (IVIG) resistance in Kawasaki disease can calculate the probability of IVIG resistance and provide a basis for clinical decision-making. We aim to assess the quality of these models developed in the children with Kawasaki disease.

METHODS

Studies of prediction models for IVIG-resistant Kawasaki disease were identified through searches in the PubMed, Web of Science, and Embase databases. Two investigators independently performed literature screening, data extraction, quality evaluation, and discrepancies were settled by a statistician. The checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS) was used for data extraction, and the prediction models were evaluated using the Prediction Model Risk of Bias Assessment Tool (PROBAST).

RESULTS

Seventeen studies meeting the selection criteria were included in the qualitative analysis. The top three predictors were neutrophil measurements (peripheral neutrophil count and neutrophil %), serum albumin level, and C-reactive protein (CRP) level. The reported area under the curve (AUC) values for the developed models ranged from 0.672 (95% confidence interval [CI]: 0.631-0.712) to 0.891 (95% CI: 0.837-0.945); The studies showed a high risk of bias (ROB) for modeling techniques, yielding a high overall ROB.

CONCLUSION

IVIG resistance models for Kawasaki disease showed high ROB. An emphasis on improving their quality can provide high-quality evidence for clinical practice.

IMPACT STATEMENT

This study systematically evaluated the risk of bias (ROB) of existing prediction models for intravenous immunoglobulin (IVIG) resistance in Kawasaki disease to provide guidance for future model development meeting clinical expectations. This is the first study to systematically evaluate the ROB of IVIG resistance in Kawasaki disease by using PROBAST. ROB may reduce model performance in different populations. Future prediction models should account for this problem, and PROBAST can help improve the methodological quality and applicability of prediction model development.

Clinical tool for prognostication of discharge outcomes following craniotomy for meningioma

BACKGROUND

Patients' comorbidities might affect the immediate postoperative morbidity and discharge disposition after surgical resection of intracranial meningioma.

OBJECTIVE

To study the impact of comorbidities on outcomes and provide a web-based application to predict time to favorable discharge.

METHODS

A retrospective review of the prospectively collected national inpatient sample (NIS) database was conducted for the years 2009-2013. Time to favorable discharge was defined as hospital length of stay (LOS). A favorable discharge was defined as a discharge to home and a non-home discharge destination was defined as an unfavorable discharge. Cox proportional hazards model was built. Full model for time to discharge and separate reduced models were built.

RESULTS

Of 10,757 patients who underwent surgery for meningioma, 6554 (60%) had a favorable discharge. The median hospital LOS was 3 days (interquartile range [IQR] 2-5). In the full model, several clinical and socioeconomic factors were associated with a higher likelihood of unfavorable discharge. In the reduced model, 13 modifiable comorbidities were negatively associated with a favorable discharge except for drug abuse and obesity, which are not associated with discharge. Both models accurately predicted time to favorable discharge (c-index:0.68-0.71).

CONCLUSION

We developed a web application using robust prognostic model that accurately predicts time to favorable discharge after surgery for meningioma. Using this tool will allow physicians to calculate individual patient discharge probabilities based on their individual comorbidities and provide an opportunity to timely risk stratify and address some of the modifiable factors prior to surgery.

Promoter hypermethylation of SFRP1 as a prognostic and potentially predictive blood-based biomarker in patients with localized pancreatic ductal adenocarcinoma

Introduction

Current prognostic blood-based biomarkers for pancreatic adenocarcinoma (PDAC) are limited. Recently, promoter hypermethylation of SFRP1 (phSFRP1) has been linked to poor prognosis in patients with gemcitabine-treated stage IV PDAC. This study explores the effects of phSFRP1 in patients with lower stage PDAC.

Methods

Based on a bisulfite treatment process, the promoter region of the SFRP1 gene was analyzed with methylation-specific PCR. Kaplan-Meier curves, log-rank tests, and generalized linear regression analysis were used to assess restricted mean survival time survival at 12 and 24 months.

Results

The study included 211 patients with stage I-II PDAC. The median overall survival of patients with phSFRP1 was 13.1 months, compared to 19.6 months in patients with unmethylated SFRP1 (umSFRP1). In adjusted analysis, phSFRP1 was associated with a loss of 1.15 months (95%CI -2.11, -0.20) and 2.71 months (95%CI -2.71, -0.45) of life at 12 and 24 months, respectively. There was no significant effect of phSFRP1 on disease-free or progression-free survival. In stage I-II PDAC, patients with phSFRP1 have worse prognoses than patients with umSFRP1.

Discussion

Results could indicate that the poor prognosis may be caused by reduced benefit from adjuvant chemotherapy. SFRP1 may help guide the clinician and be a possible target for epigenetically modifying drugs.

Selection of statistical technique for imputation of single site-univariate and multisite-multivariate methods for particulate pollutants time series data with long gaps and high missing percentage

Monitoring air contaminants has become essential to exposure science, toxicology, and public health research. However, missing values are common while monitoring air contaminants, especially in resource-constrained settings such as power cuts, calibration, and sensor failure. In contaminants monitoring, evaluating existing imputation techniques for dealing with recurrent periods of missing and unobserved data are limited. The proposed study aims to perform a statistical evaluation of six univariate and four multivariate time series imputation methods. The univariate methods are based on inter-time correlation characteristics, and the multivariate approach considers muti-site to impute missing data. The present study retrieved data from 38 ground-based monitoring stations for particulate pollutants in Delhi for 4 years. For univariate methods, missing values were simulated under 0-20% (5%, 10%, 15%, and 20%), and high 40%, 60%, and 80% missing levels having long gaps. Before evaluating multivariate methods, input data underwent pre-processing steps: selecting the target station to be imputed, choosing covariates based on the spatial correlation between multiple sites, and framing a combination of target and neighbouring stations (covariates) under 20%, 40%, 60%, and 80%. Next, the particulate pollutants data of 1480 days is provided as input to four multivariate techniques. Finally, the performance of each algorithm was evaluated using error metrics. The results show that the long interval time series data and spatial correlation of multiple stations significantly improved outcomes for univariate and multivariate time series methods. The univariate Kalman_arima performs well for long-missing gaps and all missing levels (except for 60-80%), yielding low error and high R² and d values. In contrast, multivariate MIPCA performed better than Kalman-arima for all target stations with the highest missing percentage.

Predicting patient-reported outcomes following lumbar spine surgery: development and external validation of multivariable prediction models

BACKGROUND

This study aimed to develop and externally validate prediction models of spinal surgery outcomes based on a retrospective review of a prospective clinical database, uniquely comparing multivariate regression and random forest (machine learning) approaches, and identifying the most important predictors.

METHODS

Outcomes were change in back and leg pain intensity and Core Outcome Measures Index (COMI) from baseline to the last available postoperative follow-up (3-24 months), defined as minimal clinically important change (MCID) and continuous change score. Eligible patients underwent lumbar spine surgery for degenerative pathology between 2011 and 2021. Data were split by surgery date into development (N = 2691) and validation (N = 1616) sets for temporal external validation. Multivariate logistic and linear regression, and random forest classification and regression models, were fit to the development data and validated on the external data.

RESULTS

All models demonstrated good calibration in the validation data. Discrimination ability (area under the curve) for MCID ranged from 0.63 (COMI) to 0.72 (back pain) in regression, and from 0.62 (COMI) to 0.68 (back pain) in random forests. The explained variation in continuous change scores spanned 16%-28% in linear, and 15%-25% in random forests regression. The most important predictors included age, baseline scores on the respective outcome measures, type of degenerative pathology, previous spinal surgeries, smoking status, morbidity, and duration of hospital stay.

CONCLUSIONS

The developed models appear robust and generalisable across different outcomes and modelling approaches but produced only borderline acceptable discrimination ability, suggesting the need to assess further prognostic factors. External validation showed no advantage of the random forest approach.

Breast Cancer Risk Assessment Tools for Stratifying Women into Risk Groups: A Systematic Review

BACKGROUND

The benefits and harms of breast screening may be better balanced through a risk-stratified approach. We conducted a systematic review assessing the accuracy of questionnaire-based risk assessment tools for this purpose.

METHODS

Population: asymptomatic women aged ≥40 years; Intervention: questionnaire-based risk assessment tool (incorporating breast density and polygenic risk where available); Comparison: different tool applied to the same population; Primary outcome: breast cancer incidence; Scope: external validation studies identified from databases including Medline and Embase (period 1 January 2008-20 July 2021). We assessed calibration (goodness-of-fit) between expected and observed cancers and compared observed cancer rates by risk group. Risk of bias was assessed with PROBAST.

RESULTS

Of 5124 records, 13 were included examining 11 tools across 15 cohorts. The Gail tool was most represented (n = 11), followed by Tyrer-Cuzick (n = 5), BRCAPRO and iCARE-Lit (n = 3). No tool was consistently well-calibrated across multiple studies and breast density or polygenic risk scores did not improve calibration. Most tools identified a risk group with higher rates of observed cancers, but few tools identified lower-risk groups across different settings. All tools demonstrated a high risk of bias.

CONCLUSION

Some risk tools can identify groups of women at higher or lower breast cancer risk, but this is highly dependent on the setting and population.

Multidimensional individualized nutritional therapy for individuals with severe chronic obstructive pulmonary disease: study protocol for a registry-based randomized controlled trial

BACKGROUND

Individuals with severe chronic obstructive pulmonary disease (COPD) are often at risk of undernutrition with low health-related quality of life (HRQoL). Undernutrition can worsen COPD and other comorbidities, be an independent predictor of morbidity and functional decline resulting in increased healthcare consumption and increased risk of death. Especially exacerbations and acute infections result in unintentional weight loss. The aim is to investigate the effect of an individualized nutritional intervention among individuals with severe COPD.

METHODS

An open-label randomized controlled trial with two parallel groups. Participants are recruited from the pulmonary outpatient clinic at the Department of Pulmonary and Infectious Diseases, Copenhagen University Hospital, North Zealand, Denmark, and randomly allocated to either the intervention (intervention + standard of care) or control group (standard of care). The intervention has a duration of 3 months and combines individual nutritional care with adherence support and practical tools. It contains 4 elements including an individual nutritional plan, regular contacts, adherence support, and weight diary. The primary outcome is a difference in HRQoL (EQ-5D-5L) between the intervention and control group 3 months after baseline. Difference in functional capacity (grip strength, 30-s stand chair test, and physical activity), disease-specific quality of life (COPD Assessment Test), anxiety and depression (Hospital Anxiety and Depression Scale), nutritional parameters (energy and protein intake), anthropometry (weight, body mass index, waist, hip, and upper arm circumference), body composition (total fat-free and fat mass and indices), and prognosis (exacerbations, oxygen therapy, hospital contacts, and mortality) 3 months after baseline will be included as secondary outcomes. Data will be collected through home visits at baseline and 1 and 3 months after baseline.

DISCUSSION

Currently, nutritional care is a neglected area of outpatient care among individuals with severe COPD. If this patient-centered approach can demonstrate a positive impact on HRQoL, mortality, and hospital contacts, it should be recommended as part of end-of-life care for individuals with severe COPD.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04873856 . Registered on May 3, 2021.

Are Undernutrition and Obesity Associated with Post-Discharge Mortality and Re-Hospitalization after Hospitalization with Community-Acquired Pneumonia?

Undernutrition is associated with increased mortality after hospitalization with community-acquired pneumonia (CAP), whereas obesity is associated with decreased mortality in most studies. We aimed to determine whether undernutrition and obesity are associated with increased risk of re-hospitalization and post-discharge mortality after hospitalization. This study was nested within the Surviving Pneumonia cohort, which is a prospective cohort of adults hospitalized with CAP. Patients were categorized as undernourished, well-nourished, overweight, or obese. Undernutrition was based on diagnostic criteria by the European Society for Clinical Nutrition and Metabolism. Risk of mortality was investigated using multivariate logistic regression and re-hospitalization with competing risk Cox regression where death was the competing event. Compared to well-nourished patients, undernourished patients had a higher risk of 90-day (OR 3.0, 95% CI 1.0; 21.4) mortality, but a similar 30-day and 180-day mortality risk. Obese patients had a similar re-hospitalization and mortality risk as well-nourished patients. In conclusion, among patients with CAP, undernutrition was associated with increased risk of mortality. Undernourished patients are high-risk patients, and our results indicate that in-hospital screening of undernutrition should be implemented to identify patients at mortality risk. Studies are required to investigate whether nutritional therapy after hospitalization with CAP would improve survival.

Developing Clinical Prediction Models Using Primary Care Electronic Health Record Data: The Impact of Data Preparation Choices on Model Performance

Objective

To quantify prediction model performance in relation to data preparation choices when using electronic health records (EHR).

Study Design and Setting

Cox proportional hazards models were developed for predicting the first-ever main adverse cardiovascular events using Dutch primary care EHR data. The reference model was based on a 1-year run-in period, cardiovascular events were defined based on both EHR diagnosis and medication codes, and missing values were multiply imputed. We compared data preparation choices based on (i) length of the run-in period (2- or 3-year run-in); (ii) outcome definition (EHR diagnosis codes or medication codes only); and (iii) methods addressing missing values (mean imputation or complete case analysis) by making variations on the derivation set and testing their impact in a validation set.

Results

We included 89,491 patients in whom 6,736 first-ever main adverse cardiovascular events occurred during a median follow-up of 8 years. Outcome definition based only on diagnosis codes led to a systematic underestimation of risk (calibration curve intercept: 0.84; 95% CI: 0.83-0.84), while complete case analysis led to overestimation (calibration curve intercept: -0.52; 95% CI: -0.53 to -0.51). Differences in the length of the run-in period showed no relevant impact on calibration and discrimination.

Conclusion

Data preparation choices regarding outcome definition or methods to address missing values can have a substantial impact on the calibration of predictions, hampering reliable clinical decision support. This study further illustrates the urgency of transparent reporting of modeling choices in an EHR data setting.

Using independent cross-sectional survey data to predict post-migration health trajectories among refugees by estimating transition probabilities and their variances

Health research is often concerned with the transition of health conditions and their relation with given exposures, therefore requiring longitudinal data. However, such data is not always available and resource-intensive to collect. Our aim is to use a pseudo-panel of independent cross-sectional data (e.g., data of T 0 $T_0$ and T 1 $T_1$ ) to extrapolate and approximate longitudinal health trajectories ( T 0 $T_0$ - T 1 $T_1$ ). Methods will be illustrated by examples of studying contextual effects on health among refugees by calculating transition probabilities with associated variances. The data consist of two cross-sectional health surveys among randomly selected refugee samples in reception ( T 0 $T_0$ ) and accommodation centers ( T 1 $T_1$ ) located in Germany's third-largest federal state. Self-reported measures of physical and mental health, health-related quality of life, health care access, and unmet medical needs of 560 refugees were collected. Missing data were imputed by multiple imputation. For each imputed data set, transition probabilities were calculated based on (i) probabilistic discrete event systems with Moore-Penrose generalized inverse matrix method (PDES-MP) and (ii) propensity score matching (PSM). By application of sampling approaches, exploiting the fact that status membership is multinomially distributed, results of both methods were pooled by Rubin's Rule, accounting for within and between-imputation variance. Most of the analyzed estimates of the transition probabilities and their variances are comparable between both methods. However, it seems that they handle sparse cells differently: either assigning an average value for the transition probability for all states with high certainty (i) or assigning a more extreme value for the transition probability with large variance estimate (ii).

Harm as reported by affected others: A population-based cross-sectional Finnish Gambling 2019 study

This study investigates the prevalence of being an affected other (AO) of a person with problem gambling; and the associations between being an AO and socio-demographics, gambling behaviour, health-related correlates and the amount and type of gambling-related harm (GRH) for the AOs. Furthermore, perspectives of the affected family members (AFMs) and close friends (ACFs) were acknowledged. Cross-sectional, random sample Finnish Gambling population-based survey data (n = 3,994) were collected in 2019. AOs were identified using a question with seven options. Information on GRH was sought using structured questions. One-fifth (21.2 %) of all respondents were AOs, men being typically ACFs and women being more often AFMs. Being an AO was associated with younger age, gambling participation, having a gambling problem of their own and health barriers such as psychological distress. AFMs experienced GRH more often and the amount of different GRHs was greater among the AFMs. The most common harm category experienced by the AOs was emotional harm. Both health-related issues and the amount of GRHs was largest among the AFMs. A substantial amount of GRH was also experienced by ACFs. The study suggests that support could be tailored for AFMs and ACFs, based on their AO status and individual needs. A public health approach for effective harm prevention in primary, secondary and tertiary levels are discussed.

Tenets of Good Practice in Regression Analysis. A Brief Tutorial

BACKGROUND

Regression analysis quantifies the relationships between one or more independent variables and a dependent variable and is one of the most frequently used types of analysis in medical research. The aim of this article is to provide a brief theoretical and practical tutorial for neurosurgeons wishing to conduct or interpret regression analyses.

METHODS AND RESULTS

Data preparation, univariable and multivariable analysis, choice of model, model requirements and assumptions are discussed, as essential prerequisites to any regression analysis. Four main types of regression techniques are presented: linear, logistic, multinomial logistic, and proportional odds logistic. To illustrate the applications of regression to real-world data and exemplify the concepts introduced, we used a previously reported data set of patients with intracranial aneurysms treated by microsurgical clip reconstruction at the Department of Neurosurgery of Erasmus MC University Medical Center Rotterdam, between January 2000 and January 2019.

CONCLUSIONS

Regression analysis is a powerful and versatile instrument in data analysis. This material is intended as a starter for those wishing to critically interpret or perform regression analysis and we recommend multidisciplinary collaborations with trained methodologists, statisticians, or epidemiologists.

Comparing methods for handling missing cost and quality of life data in the Early Endovenous Ablation in Venous Ulceration trial

Objectives

This study compares methods for handling missing data to conduct cost-effectiveness analysis in the context of a clinical study.

Methods

Patients in the Early Endovenous Ablation in Venous Ulceration (EVRA) trial had between 1 year and 5.5 years (median 3 years) of follow-up under early or deferred endovenous ablation. This study compares complete-case-analysis (CCA), multiple imputation using linear regression (MILR) and using predictive mean matching (MIPMM), Bayesian parametric approach using the R package missingHE (BPA), repeated measures fixed effect (RMFE) and repeated measures mixed model (RMM). The outcomes were total mean costs and total mean quality-adjusted life years (QALYs) at different time horizons (1 year, 3 years and 5 years).

Results

All methods found no statistically significant difference in cost at the 5% level in all time horizons, and all methods found statistically significantly greater mean QALY at year 1. By year 3, only BPA showed a statistically significant difference in QALY between treatments. Standard errors differed substantially between the methods employed.

Conclusion

CCA can be biased if data are MAR and is wasteful of the data. Hence the results for CCA are likely to be inaccurate. Other methods coincide in suggesting that early intervention is cost-effective at a threshold of £30,000 per QALY 1, 3 and 5 years. However, the variation in the results across the methods does generate some additional methodological uncertainty, underlining the importance of conducting sensitivity analyses using alternative approaches.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12962-022-00351-6.

Multisystemic therapy for child abuse and neglect: Parental stress and parental mental health as predictors of change in child neglect

BACKGROUND

Despite high prevalence, child neglect has long been passed over in research. Serious long-term consequences call for effective intervention programs. However, as a result of the lack of research, there is a lack of effective interventions. In order to develop such intervention programs and to maximize the effectiveness of existing programs, it is necessary to examine what factors are related to the reduction of neglect and, subsequently, what change mechanisms their effectiveness is based on.

OBJECTIVE

In this intervention study we investigated whether changes in parental mental health and parental stress after Multisystemic Therapy for Child Abuse and Neglect (MST-CAN), an effective evidence-based intervention program for child neglect, are related to changes in child neglect.

PARTICIPANTS AND SETTING

Study participants were 144 parent-child dyads participating in the MST-CAN program.

METHODS

We analyzed changes from pre- to post-treatment in child neglect, parental mental health, and parental stress, and conducted a multiple regression analysis to examine whether changes in parental mental health and parental stress predict changes in child neglect.

RESULT

Our results showed that child neglect, as well as parental stress, significantly decreased and parental mental health significantly improved during the program. While improvements in parental mental health were not related to the reduction of child neglect, a decrease in parental stress significantly predicted the reduction of child neglect.

CONCLUSION

These findings suggest that parental stress might be a promising target for evidence-based intervention programs to reduce the occurrence of child neglect. Implications and suggestions for further research are discussed.

SuperMICE: An Ensemble Machine Learning Approach to Multiple Imputation by Chained Equations

Researchers often face the problem of how to address missing data. Multiple imputation is a popular approach, with multiple imputation by chained equations (MICE) being among the most common and flexible methods for execution. MICE iteratively fits a predictive model for each variable with missing values, conditional on other variables in the data. In theory, any imputation model can be used to predict the missing values. However, if the predictive models are incorrectly specified, they may produce biased estimates of the imputed data, yielding inconsistent parameter estimates and invalid inference. Given the set of modeling choices that must be made in conducting multiple imputation, in this paper we propose a data-adaptive approach to model selection. Specifically, we adapt MICE to incorporate an ensemble algorithm, Super Learner, to predict the conditional mean for each missing value, and we also incorporate a local kernel-based estimate of variance. We present a set of simulations indicating that this approach produces final parameter estimates with lower bias and better coverage than other commonly used imputation methods. These results suggest that using a flexible machine learning imputation approach can be useful in settings where data are missing at random, especially when the relationships among the variables are complex.

Potential drug-drug interactions when managing status epilepticus patients in intensive care: A cohort study

AIMS

The risk for drug-drug interactions (DDIs) associated with antiseizure drugs (ASDs) used to manage status epilepticus (SE) patients in the intensive care unit (ICU) has been poorly investigated. We aimed to quantify and describe those potential DDIs and determine SE patient risk profiles.

METHODS

We conducted an observational bi-centric cohort study including all SE patients admitted to the ICU in the period 2016-2020.

RESULTS

Overall, 431 SE patients were included and 5504 potential DDIs were identified including 1772 DDIs (33%) between ASDs, 2610 DDIs (47%) between ASDs and previous usual treatments (PUTs), and 1067 DDIs (20%) between ASDs and ICU treatments (ICUTs). DDIs were moderate (n = 4871), major (n = 562) or severe (n = 16). All patients exhibited potential DDIs, which were major-to-severe DDIs in 47% of the cases. DDIs were pharmacokinetic (n = 1972, 36%), mostly involving cytochrome P450 modulators, and pharmacodynamic (n = 3477, 64%), mainly leading to increased sedation. ASD/PUT DDIs were the most frequent and severe. Age, PUT and ASD drug numbers and length of ICU stay were significantly associated with increased DDI number. We identified four SE patient profiles with different DDI risks and outcomes including (1) epileptic or brain trauma patients, (2) withdrawal syndrome patients, (3) older patients with comorbidities and (4) self-poisoned patients with psychiatric disorders and/or past epilepsy.

CONCLUSION

SE patients are subject to potential DDIs between ASDs, ASD/PUT and ASD/ICUT. Major-to-severe DDIs mostly occur between ASDs and PUTs. Physicians should pay attention to SE patient characteristics and history to limit DDI numbers and prevent their consequences.

The relationship between wasting and stunting in Cambodian children: Secondary analysis of longitudinal data of children below 24 months of age followed up until the age of 59 months

The interrelationship between wasting and stunting has been poorly investigated. We assessed the association between two indicators of linear growth, height-for-age Z-score (HAZ) change and occurrence of accelerated linear growth, and selected indicators of wasting and wasting reversal in 5,172 Cambodian children aged less than 24 months at enrolment in the 'MyHealth' study. The specific objectives were to evaluate the relationship between temporal changes in wasting and 1) change in HAZ and 2) episodes of accelerated linear growth. At enrolment, the stunting and wasting prevalence were 22.2 (21.0;23.3) % and 9.1 (8.1;10.1) %, respectively, and reached 41.4 (39.3;43.6) %, and 12.4 (11.5;13.3) % respectively, two years later. Between 14-19% of stunted children were also wasted throughout the whole study period. For each centimetre increase in Mid-Upper Arm Circumference (MUAC) from the previous assessment, the HAZ increased by 0.162 (0.150; 0.174) Z-score. We also observed a delayed positive association between the weight for height Z score (WHZ) unit increase and HAZ change of +0.10 to +0.22 units consistent with a positive relationship between linear growth and an increase in WHZ occurring with a lag of approximately three months. A similar positive correlation was observed for the occurrence of an episode of accelerated linear growth. These results show that interventions to prevent and treat wasting can contribute to stunting reduction and call for integrated wasting and stunting programming.

The effect of hormone replacement therapy on the survival of UK women: a retrospective cohort study 1984-2017

OBJECTIVE

To estimate the effect of estrogen-only and combined hormone replacement therapy (HRT) on the hazards of overall and age-specific all-cause mortality in healthy women aged 46-65 at first prescription.

DESIGN

Matched cohort study.

SETTING

Electronic primary care records from The Health Improvement Network (THIN) database, UK (1984-2017).

POPULATION

105 199 HRT users (cases) and 224 643 non-users (controls) matched on age and general practice.

METHODS

Weibull-Double-Cox regression models adjusted for age at first treatment, birth cohort, type 2 diabetes, hypertension and hypertension treatment, coronary heart disease, oophorectomy, hysterectomy, body mass index, smoking and deprivation status.

MAIN OUTCOME MEASURES

All-cause mortality.

RESULTS

A total of 21 751 women died over an average of 13.5 years follow-up per participant, of whom 6329 were users and 15 422 non-users. The adjusted hazard ratio (HR) of overall all-cause mortality in combined HRT users was 0.91 (95% CI 0.88-0.94), and in estrogen-only users was 0.99 (0.93-1.07), compared with non-users. Age-specific adjusted HRs for participants aged 46-50, 51-55, 56-60 and 61-65 years at first treatment were 0.98 (0.92-1.04), 0.87 (0.82-0.92), 0.88 (0.82-0.93) and 0.92 (0.85-0.98) for combined HRT users compared with non-users, and 1.01 (0.84-1.21), 1.03 (0.89-1.18), 0.98 (0.86-1.12) and 0.93 (0.81-1.07) for estrogen-only users, respectively.

CONCLUSIONS

Combined HRT was associated with a 9% lower risk of all-cause mortality and estrogen-only formulation was not associated with any significant changes.

TWEETABLE ABSTRACT

Estrogen-only HRT is not associated with all-cause mortality and combined HRT reduces the risks.

Dealing with missing information on covariates for excess mortality hazard regression models - Making the imputation model compatible with the substantive model

Missing data is a common issue in epidemiological databases. Among the different ways of dealing with missing data, multiple imputation has become more available in common statistical software packages. However, the incompatibility between the imputation and substantive model, which can arise when the associations between variables in the substantive model are not taken into account in the imputation models or when the substantive model is itself nonlinear, can lead to invalid inference. Aiming at analysing population-based cancer survival data, we extended the multiple imputation substantive model compatible-fully conditional specification (SMC-FCS) approach, proposed by Bartlett et al. in 2015 to accommodate excess hazard regression models. The proposed approach was compared with the standard fully conditional specification multiple imputation procedure and with the complete-case analysis using a simulation study. The SMC-FCS approach produced unbiased estimates in both scenarios tested, while the fully conditional specification produced biased estimates and poor empirical coverages probabilities. The SMC-FCS algorithm was then used for handling missing data in the evaluation of socioeconomic inequalities in survival from colorectal cancer patients diagnosed in the North Region of Portugal. The analysis using SMC-FCS showed a clearer trend in higher excess hazards for patients coming from more deprived areas. The proposed algorithm was implemented in R software and is presented as Supplementary Material.

Evaluation of Multiple Imputation with Large Proportions of Missing Data: How Much Is Too Much?

Background:

Multiple Imputation (MI) is known as an effective method for handling missing data in public health research. However, it is not clear that the method will be effective when the data contain a high percentage of missing observations on a variable.

Methods:

Using data from “Predictive Study of Coronary Heart Disease” study, this study examined the effectiveness of multiple imputation in data with 20% missing to 80% missing observations using absolute bias (|bias|) and Root Mean Square Error (RMSE) of MI measured under Missing Completely at Random (MCAR), Missing at Random (MAR), and Not Missing at Random (NMAR) assumptions.

Results:

The |bias| and RMSE of MI was much smaller than of the results of CCA under all missing mechanisms, especially with a high percentage of missing. In addition, the |bias| and RMSE of MI were consistent regardless of increasing imputation numbers from M=10 to M=50. Moreover, when comparing imputation mechanisms, MCMC method had universally smaller |bias| and RMSE than those of Regression method and Predictive Mean Matching method under all missing mechanisms.

Conclusion:

As missing percentages become higher, using MI is recommended, because MI produced less biased estimates under all missing mechanisms. However, when large proportions of data are missing, other things need to be considered such as the number of imputations, imputation mechanisms, and missing data mechanisms for proper imputation.

Identification of preoperative predictors for acute postsurgical pain and for pain at three months after surgery: a prospective observational study

Identifying patients at risk is the start of adequate perioperative pain management. We aimed to identify preoperative predictors for acute postsurgical pain (APSP) and for pain at 3 months after surgery to develop prediction models. In a prospective observational study, we collected preoperative predictors and the movement-evoked numerical rating scale (NRS-MEP) of postoperative pain at day 1, 2, 3, 7, week 1, 6 and 3 months after surgery from patients with a range of surgical procedures. Regression analyses of data of 2258 surgical in- and outpatients showed that independent predictors for APSP using the mean NRS-MEP over the first three days after surgery were hospital admittance, female sex, higher preoperative pain, younger age, pain catastrophizing, anxiety, higher score on functional disability, highest categories of expected pain, medical specialty, unknown wound size, and wound size > 10 cm compared to wound size ≤ 10 cm (RMSE = 2.11). For pain at three months, the only predictors were preoperative pain and a higher score on functional disability (RMSE = 1.69). Adding pain trajectories improved the prediction of pain at three months (RMSE = 1.37). Our clinically applicable prediction models can be used preoperatively to identify patients at risk, as well as in the direct postoperative period.

Clinical predictive models of invasive Candida infection: a systematic literature review

Clinical predictive models (CPM) serve to identify and categorize patients into risk categories to assist in treatment and intervention recommendations. Predictive accuracy and practicality of models varies depending on methods used for their development, and should be evaluated.The aim of this study was to summarize currently available CPM for invasive candidiasis, analyze their performance, and assess their suitability for use in clinical decision making.We identified studies that described the construction of a CPM for invasive candidiasis from PubMed/MEDLINE, EMBASE, SCOPUS, Web of Science, Cochrane Library databases and Clinicaltrials.gov. Data extracted included: author, data source, study design, recruitment period, characteristics of study population, outcome types, predictor types, number of study participants and outcome events, modelling method and list of predictors used in the final model. Calibration and discrimination in the derivative datasets were used to assess the performance of each model.Ten articles were identified in our search and included for full text review. Five models were developed using data from ICUs, and five models included all hospitalized patients. The findings of this review highlight the limitations of currently available models to predict invasive candidiasis, including lack of generalizability, difficulty in everyday clinical use, and overly optimistic performance.There are significant concerns regarding predictive performance and usability in every day practice of existing CPM to predict invasive candidiasis.

LAY SUMMARY

Clinical predictive models may assist in early identification of patients at risk for invasive candidiasis to initiate appropriate treatment. The findings of this systematic review highlight the limitations of currently available models to predict invasive candidiasis.

Prognostic Models Predicting Mortality in Preterm Infants: Systematic Review and Meta-analysis

CONTEXT

Prediction models can be a valuable tool in performing risk assessment of mortality in preterm infants.

OBJECTIVE

Summarizing prognostic models for predicting mortality in very preterm infants and assessing their quality.

DATA SOURCES

Medline was searched for all articles (up to June 2020).

STUDY SELECTION

All developed or externally validated prognostic models for mortality prediction in liveborn infants born <32 weeks' gestation and/or <1500 g birth weight were included.

DATA EXTRACTION

Data were extracted by 2 independent authors. Risk of bias (ROB) and applicability assessment was performed by 2 independent authors using Prediction model Risk of Bias Assessment Tool.

RESULTS

One hundred forty-two models from 35 studies reporting on model development and 112 models from 33 studies reporting on external validation were included. ROB assessment revealed high ROB in the majority of the models, most often because of inadequate (reporting of) analysis. Internal and external validation was lacking in 41% and 96% of these models. Meta-analyses revealed an average C-statistic of 0.88 (95% confidence interval [CI]: 0.83-0.91) for the Clinical Risk Index for Babies score, 0.87 (95% CI: 0.81-0.92) for the Clinical Risk Index for Babies II score, and 0.86 (95% CI: 0.78-0.92) for the Score for Neonatal Acute Physiology Perinatal Extension II score.

LIMITATIONS

Occasionally, an external validation study was included, but not the development study, because studies developed in the presurfactant era or general NICU population were excluded.

CONCLUSIONS

Instead of developing additional mortality prediction models for preterm infants, the emphasis should be shifted toward external validation and consecutive adaption of the existing prediction models.

Generative adversarial networks for imputing missing data for big data clinical research

Background

Missing data is a pervasive problem in clinical research. Generative adversarial imputation nets (GAIN), a novel machine learning data imputation approach, has the potential to substitute missing data accurately and efficiently but has not yet been evaluated in empirical big clinical datasets.

Objectives

This study aimed to evaluate the accuracy of GAIN in imputing missing values in large real-world clinical datasets with mixed-type variables. The computation efficiency of GAIN was also evaluated. The performance of GAIN was compared with other commonly used methods, MICE and missForest.

Methods

Two real world clinical datasets were used. The first was that of a cohort study on the long-term outcomes of patients with diabetes (50,000 complete cases), and the second was of a cohort study on the effectiveness of a risk assessment and management programme for patients with hypertension (10,000 complete cases). Missing data (missing at random) to independent variables were simulated at different missingness rates (20, 50%). The normalized root mean square error (NRMSE) between imputed values and real values for continuous variables and the proportion of falsely classified (PFC) for categorical variables were used to measure imputation accuracy. Computation time per imputation for each method was recorded. The differences in accuracy of different imputation methods were compared using ANOVA or non-parametric test.

Results

Both missForest and GAIN were more accurate than MICE. GAIN showed similar accuracy as missForest when the simulated missingness rate was 20%, but was more accurate when the simulated missingness rate was 50%. GAIN was the most accurate for the imputation of skewed continuous and imbalanced categorical variables at both missingness rates. GAIN had a much higher computation speed (32 min on PC) comparing to that of missForest (1300 min) when the sample size is 50,000.

Conclusion

GAIN showed better accuracy as an imputation method for missing data in large real-world clinical datasets compared to MICE and missForest, and was more resistant to high missingness rate (50%). The high computation speed is an added advantage of GAIN in big clinical data research. It holds potential as an accurate and efficient method for missing data imputation in future big data clinical research.

Trial registration

ClinicalTrials.gov ID: NCT03299010; Unique Protocol ID: HKUCTR-2232

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01272-3.

Multiple Myeloma Patients Undergoing Carfilzomib: Development and Validation of a Risk Score for Cardiovascular Adverse Events Prediction

Simple Summary

Despite the relationship between Carfilzomib (CFZ) therapy in multiple myeloma (MM) and cardiovascular adverse events (CVAEs), no specific validated protocols on cardiovascular risk assessment are available. In this prospective study, we investigated major predictors of CVAEs prior to starting CFZ, applying the European Myeloma Network management protocol (EMN). Five predictors were identified: office systolic blood pressure, 24-h blood pressure variability, left ventricular hypertrophy, pulse wave velocity value and global longitudinal strain. The resulting ‘CVAEs risk score’ defined a low- and a high-risk group (negative predicting value for the high-risk group of 90%). 62 patients experienced one or more CVAEs: 17 major and 45 hypertension-related events. In conclusion, CVAEs are frequent and a specific management protocol is required. The EMN protocol and ‘CFZ risk score’ proved to be effective in estimating the baseline risk of CVAEs during CFZ therapy in MM patients, targeting the appropriate follow-up.

Abstract

Cardiovascular adverse events (CVAEs) are linked to Carfilzomib (CFZ) therapy in multiple myeloma (MM); however, no validated protocols on cardiovascular risk assessment are available. In this prospective study, the effectiveness of the European Myeloma Network protocol (EMN) in cardiovascular risk assessment was investigated, identifying major predictors of CVAEs. From January 2015 to March 2020, 116 MM patients who had indication for CFZ therapy underwent a baseline evaluation (including blood pressure measurements, echocardiography and arterial stiffness estimation) and were prospectively followed. The median age was 64.53 ± 8.42 years old, 56% male. Five baseline independent predictors of CVAEs were identified: office systolic blood pressure, 24-h blood pressure variability, left ventricular hypertrophy, pulse wave velocity value and global longitudinal strain. The resulting ‘CVAEs risk score’ distinguished a low- and a high-risk group, obtaining a negative predicting value for the high-risk group of 90%. 52 patients (44.9%) experienced one or more CVAEs: 17 (14.7%) had major and 45 (38.7%) had hypertension-related events. In conclusion, CVAEs are frequent and a specific management protocol is crucial. The EMN protocol and the risk score proved to be useful to estimate the baseline risk for CVAEs during CFZ therapy, allowing the identification of higher-risk patients.

Multiple imputation to balance unbalanced designs for two-way analysis of variance

A balanced ANOVA design provides an unambiguous interpretation of the F-tests, and has more power than an unbalanced design. In earlier literature, multiple imputation was proposed to create balance in unbalanced designs, as an alternative to Type-III sum of squares. In the current simulation study we studied four pooled statistics for multiple imputation, namely D₀, D₁, D₂, and D₃ in unbalanced data, and compared them with Type-III sum of squares. Statistics D₁ and D₂ generally performed best regarding Type-I error rates, and had power rates closest to that of Type-III sum of squares. Additionally, for the interaction, D₁ produced power rates higher than Type-III sum of squares. For multiply imputed datasets D₁ and D₂ may be the best methods for pooling the results in multiply imputed datasets, and for unbalanced data, D₁ might be a good alternative to Type-III sum of squares regarding the interaction.

Exploring the Relationship Between Movement Disorders and Physical Activity in Patients With Schizophrenia: An Actigraphy Study

Low physical activity (PA) and sedentary behavior (SB) are major contributors to mental health burden and increased somatic comorbidity and mortality in people with schizophrenia and related psychoses. Movement disorders are highly prevalent in schizophrenia populations and are related to impaired functioning and poor clinical outcome. However, the relationship between movement disorders and PA and SB has remained largely unexplored. Therefore, we aimed to examine the relationship between movement disorders (akathisia, dyskinesia, dystonia, and parkinsonism) and PA and SB in 216 patients with schizophrenia and related psychoses. Actigraphy, the St. Hans Rating Scale for extrapyramidal syndromes, and psychopathological ratings (PANSS-r) were applied. Data were analyzed using multiple linear regression, adjusting for sex, age, negative symptoms, and defined daily dose of prescribed antipsychotics. Parkinsonism was significantly associated with decreased PA (β = -0.21, P < .01) and increased SB (β = 0.26, P < .001). For dystonia, only the relationship with SB was significant (β = 0.15, P < .05). Akathisia was associated with more PA (β = 0.14, P < .05) and less SB (β = -0.15, P < .05). For dyskinesia, the relationships were non-significant. In a prediction model, akathisia, dystonia, parkinsonism and age significantly predicted PA (F(5,209) = 16.6, P < .001, R2Adjusted = 0.27) and SB (F(4,210) = 13.4, P < .001, R2Adjusted = 0.19). These findings suggest that movement disorders, in particular parkinsonism, are associated with reduced PA and increased SB in patients with psychotic disorders. Future studies should take movement disorders into account when examining PA and SB, to establish the clinical value of movement disorders in activating people with psychotic disorders to improve their mental and somatic health.

A tool to predict progression of non-alcoholic fatty liver disease in severely obese patients

BACKGROUND & AIMS

Severely obese patients are a growing population at risk of non-alcoholic fatty liver disease (NAFLD). Considering the increasing burden, a predictive tool of NAFLD progression would be of interest. Our objective was to provide a tool allowing general practitioners to identify and refer the patients most at risk, and specialists to estimate disease progression and adapt the therapeutic strategy.

METHODS

This predictive tool is based on a Markov model simulating steatosis, fibrosis and non-alcoholic steatohepatitis (NASH) evolution. This model was developped from data of 1801 severely obese, bariatric surgery candidates, with histological assessment, integrating duration of exposure to risk factors. It is then able to predict current disease severity in the absence of assessment, and future cirrhosis risk based on current stage.

RESULTS

The model quantifies the impact of sex, body-mass index at 20, diabetes, age of overweight onset, on progression. For example, for 40-year-old severely obese patients seen by the general practitioners: (a) non-diabetic woman overweight at 20, and (b) diabetic man overweight at 10, without disease assessment, the model predicts their current risk to have NASH or F3-F4: for (a) 5.7% and 0.6%, for (b) 16.1% and 10.0% respectively. If those patients have been diagnosed F2 by the specialist, the model predicts the 5-year cirrhosis risk: 1.8% in the absence of NASH and 6.0% in its presence for (a), 10.3% and 26.7% respectively, for (b).

CONCLUSIONS

This model provides a decision-making tool to predict the risk of liver disease that could help manage severely obese patients.

Hypertensive nephrosclerosis: wider kidney biopsy indications may be needed to improve diagnostics

BACKGROUND

Hypertensive nephrosclerosis is the presumed underlying cause in many end-stage kidney disease (ESKD) patients, but the diagnosis is disputed and based on clinical criteria with low diagnostic accuracy.

OBJECTIVE

To evaluate and improve the diagnostic process for nephrosclerosis patients.

METHODS

We included adults from the population-based HUNT study (n = 50 552), Norwegian CKD patients referred for kidney biopsy 1988-2012 (n = 7261), and unselected nephrology clinic patients (n = 193) used for matching. Decision tree analysis and ROC curve-based methods of optimal cut-offs were used to improve clinical nephrosclerosis criteria.

RESULTS

Nephrosclerosis prevalence was 2.7% in the general population, and eGFR decline and risk for kidney-related hospital admissions and ESKD were comparable to patients with diabetic kidney disease. In the biopsy cohort, current clinical criteria had very low sensitivity (0.13) but high specificity (0.94) for biopsy-verified arterionephrosclerosis. A new optimized diagnostic algorithm based on proteinuria (<0.75 g d^-1 ), systolic blood pressure (>155 mm Hg) and age (>75 years) only marginally improved diagnostic accuracy (sensitivity 0.19, specificity 0.96). Likewise, there were still false-positive cases with treatable diagnoses like glomerulonephritis, interstitial nephritis and others (40% of all test positive). Decision curve analysis showed that the new criteria can lead to higher clinical utility, especially for patients considering the potential harms to be close to the potential benefits, while the more risk-tolerant ones (harm:benefit ratio < 1:4) should consider kidney biopsy.

CONCLUSION

Further improvements of the current clinical criteria seem difficult, so risks and benefits of kidney biopsy could be more actively discussed with selected patients to reduce misclassification and direct treatment.

Much ado about nothing: Multiple imputation to balance unbalanced designs for two-way analysis of variance

In earlier literature, multiple imputation was proposed to create balance in unbalanced designs, as an alternative to Type III sum of squares in two-way ANOVA. In the current simulation study we studied four pooled statistics for multiple imputation, namely D₀, D₁, D₂, and D₃ in unbalanced data, and compared these statistics with Type III sum of squares. Statistics D₀ and D₂ generally performed best regarding Type-I error rates, and had power rates closest to that of Type III sum of squares. However, none of the statistics produced power rates higher than Type III sum of squares. The results lead to the conclusion that for multiply imputed datasets D₀ and D₂ may be the best methods for pooling the results of multiparameter estimates in multiply imputed datasets, and that for unbalanced data, Type III sum of square is to be preferred over using multiple imputation in obtaining ANOVA results.

Missing-Data Handling Methods for Lifelogs-Based Wellness Index Estimation: Comparative Analysis With Panel Data

Background

A lifelogs-based wellness index (LWI) is a function for calculating wellness scores based on health behavior lifelogs (eg, daily walking steps and sleep times collected via a smartwatch). A wellness score intuitively shows the users of smart wellness services the overall condition of their health behaviors. LWI development includes estimation (ie, estimating coefficients in LWI with data). A panel data set comprising health behavior lifelogs allows LWI estimation to control for unobserved variables, thereby resulting in less bias. However, these data sets typically have missing data due to events that occur in daily life (eg, smart devices stop collecting data when batteries are depleted), which can introduce biases into LWI coefficients. Thus, the appropriate choice of method to handle missing data is important for reducing biases in LWI estimations with panel data. However, there is a lack of research in this area.

Objective

This study aims to identify a suitable missing-data handling method for LWI estimation with panel data.

Methods

Listwise deletion, mean imputation, expectation maximization–based multiple imputation, predictive-mean matching–based multiple imputation, k-nearest neighbors–based imputation, and low-rank approximation–based imputation were comparatively evaluated by simulating an existing case of LWI development. A panel data set comprising health behavior lifelogs of 41 college students over 4 weeks was transformed into a reference data set without any missing data. Then, 200 simulated data sets were generated by randomly introducing missing data at proportions from 1% to 80%. The missing-data handling methods were each applied to transform the simulated data sets into complete data sets, and coefficients in a linear LWI were estimated for each complete data set. For each proportion for each method, a bias measure was calculated by comparing the estimated coefficient values with values estimated from the reference data set.

Results

Methods performed differently depending on the proportion of missing data. For 1% to 30% proportions, low-rank approximation–based imputation, predictive-mean matching–based multiple imputation, and expectation maximization–based multiple imputation were superior. For 31% to 60% proportions, low-rank approximation–based imputation and predictive-mean matching–based multiple imputation performed best. For over 60% proportions, only low-rank approximation–based imputation performed acceptably.

Conclusions

Low-rank approximation–based imputation was the best of the 6 data-handling methods regardless of the proportion of missing data. This superiority is generalizable to other panel data sets comprising health behavior lifelogs given their verified low-rank nature, for which low-rank approximation–based imputation is known to perform effectively. This result will guide missing-data handling in reducing coefficient biases in new development cases of linear LWIs with panel data.

Sentinel node biopsy in patients with melanoma improves the accuracy of staging when added to clinicopathological features of the primary tumor

BACKGROUND

It has been claimed, without supporting evidence, that knowledge of sentinel node (SN) status does not provide more accurate prognostic information than basic clinicopathological features of a primary cutaneous melanoma. We sought to investigate this claim and to quantify any additional value of SN status in predicting survival outcome.

PATIENTS AND METHODS

Data for a Dutch population-based cohort of melanoma patients (n = 9272) and for a validation cohort from a large Australian melanoma treatment center (n = 5644) were analyzed. Patients were adults diagnosed between 2004 and 2014 with histologically-proven, primary invasive cutaneous melanoma who underwent SN biopsy. Multivariable Cox proportional hazards analyses were carried out in the Dutch cohort to assess recurrence-free survival (RFS), melanoma-specific survival (MSS) and overall survival (OS). The findings were validated using the Australian cohort. Discrimination (Harrell's C-statistic), net benefit using decision curve analysis and net reclassification index (NRI) were calculated.

RESULTS

The Dutch cohort showed an improved C-statistic from 0.74 to 0.78 for OS and from 0.74 to 0.76 for RFS when SN status was included in the model with Breslow thickness, sex, age, site, mitoses, ulceration, regression and melanoma subtype. In the Australian cohort, the C-statistic increased from 0.70 to 0.73 for OS, 0.70 to 0.74 for RFS and 0.72 to 0.76 for MSS. Decision curve analyses showed that the 3-year and 5-year risk of death or recurrence were more accurately classified with a model that included SN status. At 3 years, sensitivity increased by 12% for both OS and RFS in the development cohort, and by 10% and 6% for OS and RFS, respectively, in the validation cohort.

CONCLUSIONS

Knowledge of SN status significantly improved the predictive accuracy for RFS, MSS and OS when added to a comprehensive suite of established clinicopathological prognostic factors. However, clinicians and patients must consider the magnitude of the improvement when weighing up the advantages and disadvantages of SN biopsy for melanoma.

Multiple imputation methods for handling missing values in longitudinal studies with sampling weights: Comparison of methods implemented in Stata

Many analyses of longitudinal cohorts require incorporating sampling weights to account for unequal sampling probabilities of participants, as well as the use of multiple imputation (MI) for dealing with missing data. However, there is no guidance on how MI and sampling weights should be implemented together. We simulated a target population based on the Australian Bureau of Statistics Estimated Resident Population and drew 1000 random samples dependent on three design variables to mimic the Longitudinal Study of Australian Children. The target analysis was the weighted prevalence of overweight/obesity over childhood. We evaluated the performance of several MI approaches available in Stata, based on multivariate normal imputation (MVNI), fully conditional specification (FCS) and twofold FCS: a weighted imputation model, imputing missing data separately for each quintile sampling weight grouping, including the design stratum indicator in the imputation model, and using sampling weights as a covariate in the imputation model. Approaches based on available cases and inverse probability weighting (IPW), with time-varying weights, were also compared. We observed severe issues of convergence with FCS and twofold FCS. All MVNI-based approaches performed similarly, producing minimal bias and nominal coverage, except for when imputation was conducted separately for each quintile sampling weight group. IPW performed equally as well as MVNI-based approaches in terms of bias, however, was less precise. In similar longitudinal studies, we recommend using MVNI with the design stratum as a covariate in the imputation model. If this is unknown, including the sampling weight as a covariate is an appropriate alternative.

Multiple imputation and selection of ordinal level 2 predictors in multilevel models: An analysis of the relationship between student ratings and teacher practices and attitudes

The article is motivated by the analysis of the relationship between university student ratings and teacher practices and attitudes, which are measured via a set of binary and ordinal items collected by an innovative survey. The analysis is conducted through a two-level random intercept model, where student ratings are nested within teachers. The analysis must face two issues about the items measuring teacher practices and attitudes, which are level 2 predictors: (a) the items are severely affected by missingness due to teacher non-response and (b) there is redundancy in both the number of items and the number of categories of their measurement scale. We tackle the missing data issue by considering a multiple imputation strategy exploiting information at both student and teacher levels. For the redundancy issue, we rely on regularization techniques for ordinal predictors, also accounting for the multilevel data structure. The proposed solution addresses the problem at hand in an original way, and it can be applied whenever it is required to select level 2 predictors affected by missing values. The results obtained with the final model indicate that ratings on teacher ability to motivate students are related to certain teacher practices and attitudes.

National trends in heart failure mortality in men and women, United Kingdom, 2000-2017

Aims

To understand gender differences in the prognosis of women and men with heart failure, we compared mortality, cause of death and survival trends over time.

Methods and results

We analysed UK primary care data for 26 725 women and 29 234 men over age 45 years with a new diagnosis of heart failure between 1 January 2000 and 31 December 2017 using the Clinical Practice Research Datalink, inpatient Hospital Episode Statistics and the Office for National Statistics death registry. Age‐specific overall survival and cause‐specific mortality rates were calculated by gender and year. During the study period 15 084 women and 15 822 men with heart failure died. Women were on average 5 years older at diagnosis (79.6 vs. 74.8 years). Median survival was lower in women compared to men (3.99 vs. 4.47 years), but women had a 14% age‐adjusted lower risk of all‐cause mortality [hazard ratio (HR) 0.86, 95% confidence interval (CI) 0.84–0.88]. Heart failure was equally likely to be cause of death in women and men (HR 1.03, 95% CI 0.96–1.12). There were modest improvements in survival for both genders, but these were greater in men. The reduction in mortality risk in women was greatest for those diagnosed in the community (HR 0.83, 95% CI 0.80–0.85).

Conclusions

Women are diagnosed with heart failure older than men but have a better age‐adjusted prognosis. Survival gains were less in women over the last two decades. Addressing gender differences in heart failure diagnostic and treatment pathways should be a clinical and research priority.

Assessment of the quality of end-of-life care: translation and validation of the German version of the "Care of the Dying Evaluation" (CODE-GER) - a questionnaire for bereaved relatives

Background

International studies indicate deficits in end-of-life care that can lead to distress for patients and their next-of-kin.

The aim of the study was to translate and validate the “Care of the Dying Evaluation” (CODE) into German (CODE-GER).

Methods

Translation according to EORTC (European Organisation for Research and Treatment of Cancer) guidelines was followed by data collection to evaluate psychometric properties of CODE-GER. Participants were next-of-kin of patients who had died an expected death in two hospitals. They were invited to participate at least eight, but not later than 16 weeks after the patient’s death. To calculate construct validity, the Palliative care Outcome Scale (POS) was assessed. Difficulty and perceived strain of answering the questionnaire were assessed by a numeric scale (0–10).

Results

Out of 1137 next-of-kin eligible, 317 completed the questionnaire (response rate: 27.9%). Data from 237 main sample participants, 38 interraters and 55 next-of-kin who participated for repeated measurement were analysed. Overall internal consistency, α = 0.86, interrater reliability, ICC (1) = 0.79, and retest-reliability, ICC (1, 2) = 0.85, were good. Convergent validity between POS and CODE-GER, r = −.46, was satisfactory. A principal component analysis with varimax rotation showed a 7-factor solution. Difficulty, M = 2.2; SD ± 2.4, and perceived strain, M = 4.1; SD ± 3.0, of completing the questionnaire were rather low.

Conclusion

The results from the present study confirm CODE-GER as a reliable and valid instrument to assess the quality of care of the dying person. More over our study adds value to the original questionnaire by proposing a deepened analysis of obtained data. The development of seven subscales increases its potential for further surveys and research.

Trial registration

This study was registered retrospectively on the 25th of January 2018 at the German Clinical Trials Register (DRKS00013916).

Accounting for missing data in statistical analyses: multiple imputation is not always the answer

Background

Missing data are unavoidable in epidemiological research, potentially leading to bias and loss of precision. Multiple imputation (MI) is widely advocated as an improvement over complete case analysis (CCA). However, contrary to widespread belief, CCA is preferable to MI in some situations.

Methods

We provide guidance on choice of analysis when data are incomplete. Using causal diagrams to depict missingness mechanisms, we describe when CCA will not be biased by missing data and compare MI and CCA, with respect to bias and efficiency, in a range of missing data situations. We illustrate selection of an appropriate method in practice.

Results

For most regression models, CCA gives unbiased results when the chance of being a complete case does not depend on the outcome after taking the covariates into consideration, which includes situations where data are missing not at random. Consequently, there are situations in which CCA analyses are unbiased while MI analyses, assuming missing at random (MAR), are biased. By contrast MI, unlike CCA, is valid for all MAR situations and has the potential to use information contained in the incomplete cases and auxiliary variables to reduce bias and/or improve precision. For this reason, MI was preferred over CCA in our real data example.

Conclusions

Choice of method for dealing with missing data is crucial for validity of conclusions, and should be based on careful consideration of the reasons for the missing data, missing data patterns and the availability of auxiliary information.

Standardized Regression Coefficients and Newly Proposed Estimators for [Formula: see text] in Multiply Imputed Data

Whenever statistical analyses are applied to multiply imputed datasets, specific formulas are needed to combine the results into one overall analysis, also called combination rules. In the context of regression analysis, combination rules for the unstandardized regression coefficients, the t-tests of the regression coefficients, and the F-tests for testing [Formula: see text] for significance have long been established. However, there is still no general agreement on how to combine the point estimators of [Formula: see text] in multiple regression applied to multiply imputed datasets. Additionally, no combination rules for standardized regression coefficients and their confidence intervals seem to have been developed at all. In the current article, two sets of combination rules for the standardized regression coefficients and their confidence intervals are proposed, and their statistical properties are discussed. Additionally, two improved point estimators of [Formula: see text] in multiply imputed data are proposed, which in their computation use the pooled standardized regression coefficients. Simulations show that the proposed pooled standardized coefficients produce only small bias and that their 95% confidence intervals produce coverage close to the theoretical 95%. Furthermore, the simulations show that the newly proposed pooled estimates for [Formula: see text] are less biased than two earlier proposed pooled estimates.

Missing tumor measurement (TM) data in the search for alternative TM-based endpoints in cancer clinical trials

PURPOSE

Missing data commonly occur in cancer clinical trials (CCT) and may hinder the search for alternative trial endpoints. We consider reasons for missing tumor measurement (TM) data in CCT and how missing TM data are typically handled. We explore the potential impact of missing TM data on predictive ability of a set of TM-based endpoints.

METHODS

Literature review identifies reasons for and approaches to handling missing TM data. Data from 3 actual clinical trials were used for illustration. A sensitivity analysis of the potential impact of missing TM data was performed by comparing overall survival (OS) predictive ability of alternative endpoints using observed and imputed data.

RESULTS

Reasons for missing TM data in CCT are presented, based on the literature review and the three trials. Although missing TM data impacted individual objective status (e.g. 12-week status changed for 53% of patients in one imputation set), it surprisingly only minimally impacted endpoint predictive ability (e.g. median c-indices of 500 imputed datasets ranged from 0.566 to 0.570 for N9741, 0.592-0.616 for N9841, and 0.542-0.624 for N0026).

CONCLUSION

By understanding the reasons for missingness, we can better anticipate them and minimize their occurrence. Our preliminary analysis suggests missing TM data may not impact endpoint predictive ability, but could impact objective response status classification; however these findings require further validation. With response status accepted as an important phase II endpoint in the development of new cancer therapies (including immunotherapy), we urge that in CCT complete TM data collection and adherence to protocol-defined disease evaluation as closely as possible be a priority.

Well-Child Care Adherence After Intrauterine Opioid Exposure

BACKGROUND AND OBJECTIVES

For children with intrauterine opioid exposure (IOE), well-child care (WCC) provides an important opportunity to address medical, developmental, and psychosocial needs. We evaluated WCC adherence for this population.

METHODS

In this retrospective cohort study, we used PEDSnet data from a pediatric primary care network spanning 3 states from 2011 to 2016. IOE was ascertained by using physician diagnosis codes. WCC adherence in the first year was defined as a postnatal or 1-month visit and completed 2-, 4-, 6-, 9-, and 12-month visits. WCC adherence in the second year was defined as completed 15- and 18-month visits. Gaps in WCC, defined as ≥2 missed consecutive WCC visits, were also evaluated. We used multivariable regression to test the independent effect of IOE status.

RESULTS

Among 11 334 children, 236 (2.1%) had a diagnosis of IOE. Children with IOE had a median of 6 WCC visits (interquartile range 5-7), vs 8 (interquartile range 6-8) among children who were not exposed (P < .001). IOE was associated with decreased WCC adherence over the first and second years of life (adjusted relative risk 0.54 [P < .001] and 0.74 [P < .001]). WCC gaps were more likely in this population (adjusted relative risk 1.43; P < .001). There were no significant adjusted differences in nonroutine primary care visits, immunizations by age 2, or lead screening.

CONCLUSIONS

Children <2 years of age with IOE are less likely to adhere to recommended WCC, despite receiving on-time immunizations and lead screening. Further research should be focused on the role of WCC visits to support the complex needs of this population.

Reporting and Methods in Developing Prognostic Prediction Models for Metabolic Syndrome: A Systematic Review and Critical Appraisal

PURPOSE

A prognostic prediction model for metabolic syndrome can calculate the probability of risk of experiencing metabolic syndrome within a specific period for individualized treatment decisions. We aimed to provide a systematic review and critical appraisal on prognostic models for metabolic syndrome.

MATERIALS AND METHODS

Studies were identified through searching in English databases (PubMed, EMBASE, CINAHL, and Web of Science) and Chinese databases (Sinomed, WANFANG, CNKI, and CQVIP). A checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS) and the prediction model risk of bias assessment tool (PROBAST) were used for the data extraction process and critical appraisal.

RESULTS

From the 29,668 retrieved articles, eleven studies meeting the selection criteria were included in this review. Forty-eight predictors were identified from prognostic prediction models. The c-statistic ranged from 0.67 to 0.95. Critical appraisal has shown that all modeling studies were subject to a high risk of bias in methodological quality mainly driven by outcome and statistical analysis, and six modeling studies were subject to a high risk of bias in applicability.

CONCLUSION

Future model development and validation studies should adhere to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement to improve methodological quality and applicability, thus increasing the transparency of the reporting of a prediction model study. It is not appropriate to adopt any of the identified models in this study for clinical practice since all models are prone to optimism and overfitting.

Lifestyle behaviours, ethnicity and menstruation have little added value in prediction models for low haemoglobin deferral in whole blood donors

OBJECTIVE

To investigate the added value of questionnaire-based predictors to existing prediction models for low haemoglobin (Hb) deferral in whole blood donors.

BACKGROUND

Prediction models for Hb deferral risk can be applied in the invitation process of donors for a blood donation. Existing prediction models are based on routinely collected data. The model performance might be improved by the addition of predictive factors.

METHODS

The added value of food consumption, smoking, physical activity, ethnicity and menstruation in the prediction of Hb deferral was assessed by comparing the existing models with extended models using the following measures: model X² , concordance (c)-statistic and net reclassification improvement (NRI).

RESULTS

Addition of one candidate predictor to the models did not substantially improve the model performance. Addition of multiple new candidate predictors significantly increased the model X² (from 137 to 159 for men, and from 157 to 199 for women) and resulted in a non-significant increase of the c-statistic (from 0.85 to 0.87 for men, and from 0.78 to 0.81 for women). The NRI for men was 11.4% and for women 1.5% after addition of multiple predictors.

CONCLUSION

Addition of lifestyle behaviours, ethnicity or menstruation to prediction models for low Hb deferral in whole blood donors improved the model performance, but not substantially. For easy use in practice, we do not recommend addition of the investigated predictors to the prediction models.