Investigating the missing data mechanism in quality of life outcomes: a comparison of approaches

Impact of missing values on the ability of the acute physiology and chronic health evaluation III and Japan risk of death models to predict mortality

PURPOSE

This study assessed model performance of the Acute Physiology and Chronic Health Evaluation (APACHE) III and Japan Risk of Death (JROD) when degraded by the number and category of missing variables. We also examined the impact of missing data on predicted mortality for facilities with missing physiological variables.

METHODS

We obtained data from the Japanese Intensive care PAtient Database (JIPAD). We calculated observed and predicted mortality rates using the APACHE III and JROD and the standardized mortality ratio (SMR) by the number and category of missing variables. Smoothed spline curves were calculated for the SMR to the missing proportion of the facility.

RESULTS

A total of 61,357 patients from 57 ICUs were included between April 2015 and March 2019. The APACHE III and JROD SMRs increased as the number of missing values increased. The SMR in the APACHE III model was elevated in facilities with a larger proportion of missing in each of the APS categories, arterial blood gas, albumin, glucose, and bilirubin. Facilities with a high proportion of missing albumin data preserved their SMRs in only the JROD model.

CONCLUSION

An increased number of missing physiological variables resulted in falsely low predicted mortality rates and high SMRs.

Circulating homocysteine and folate concentrations and risk of type 2 diabetes: A retrospective observational study in Chinese adults and a Mendelian randomization analysis

Background

The relation between circulating homocysteine (hcy) and folate concentrations and risk of type 2 diabetes mellitus (T2DM) has been evaluated in several observational studies with inconsistent results; and it is unclear about their causal relationships. Our aim was to assess the causality association between circulating hcy or folate concentrations and the development of T2DM using Mendelian randomization (MR) analysis, based on results of an observational study in Chinese adults.

Methods

We conducted an observational study of 370 patients with T2DM and 402 controls after routine physical examination who consulted at the Zhongnan Hospital of Wuhan University between March 2021 and December 2021. Correlations between hcy and folate and the incidence of T2DM were quantified using logistic regression models. Two-sample MR analysis was conducted using summary statistics of genetic variants gained from 2 genome-wide association studies (GWAS) on circulating hcy and folate concentrations in individuals of European ancestry and from an independent GWAS study based on DIAMANTE meta-analysis.

Results

In the observational study, after logistic regression with multiple adjustment, lower hcy and higher folate levels were identified to be associated with the risk of T2DM, with OR (95% CI) for hcy of 1.032 (1.003–1.060); while 0.909 (0.840–0.983) for folate. In the MR analysis, the OR for T2DM was 1.08 (95% CI: 0.95, 1.21; P = 0.249) for each SD unit increase in genetically predicted homocysteinemia and the OR for T2DM per SD increase in genetically predicted folate elevation was 0.80 (95% CI: 0.60, 1.00, P = 0.026).

Conclusions

We discovered that high circulating hcy and low folate concentrations were related with an increased risk of developing T2DM in Chinese adults. Moreover, MR analysis provided genetic evidence for a possible causal relationship between serum folate and the risk of T2DM.

To Explore What Isn't There-Glyph-Based Visualization for Analysis of Missing Values

This article contributes a novel visualization method, Missingness Glyph, for analysis and exploration of missing values in data. Missing values are a common challenge in most data generating domains and may cause a range of analysis issues. Missingness in data may indicate potential problems in data collection and pre-processing, or highlight important data characteristics. While the development and improvement of statistical methods for dealing with missing data is a research area in its own right, mainly focussing on replacing missing values with estimated values, considerably less focus has been put on visualization of missing values. Nonetheless, visualization and explorative analysis has great potential to support understanding of missingness in data, and to enable gaining of novel insights into patterns of missingness in a way that statistical methods are unable to. The Missingness Glyph supports identification of relevant missingness patterns in data, and is evaluated and compared to two other visualization methods in context of the missingness patterns. The results are promising and confirms that the Missingness Glyph in several cases perform better than the alternative visualization methods.

The reporting and handling of missing data in longitudinal studies of older adults is suboptimal: a methodological survey of geriatric journals

Background

Missing data are common in longitudinal studies, and more so, in studies of older adults, who are susceptible to health and functional decline that limit completion of assessments. We assessed the extent, current reporting, and handling of missing data in longitudinal studies of older adults.

Methods

Medline and Embase databases were searched from 2015 to 2019 for publications on longitudinal observational studies conducted among persons ≥55 years old. The search was restricted to 10 general geriatric journals published in English. Reporting and handling of missing data were assessed using questions developed from the recommended standards. Data were summarised descriptively as frequencies and proportions.

Results

A total of 165 studies were included in the review from 7032 identified records. In approximately half of the studies 97 (62.5%), there was either no comment on missing data or unclear descriptions. The percentage of missing data varied from 0.1 to 55%, with a 14% average among the studies that reported having missing data. Complete case analysis was the most common method for handling missing data with nearly 75% of the studies (n = 52) excluding individual observations due to missing data, at the initial phase of study inclusion or at the analysis stage. Of the 10 studies where multiple imputation was used, only 1 (10.0%) study followed the guideline for reporting the procedure fully using online supplementary documents.

Conclusion

The current reporting and handling of missing data in longitudinal observational studies of older adults are inadequate. Journal endorsement and implementation of guidelines may potentially improve the quality of missing data reporting. Further, authors should be encouraged to use online supplementary files to provide additional details on how missing data were addressed, to allow for more transparency and comprehensive appraisal of studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01605-w.

Early protein energy malnutrition impacts life-long developmental trajectories of the sources of EEG rhythmic activity

Protein Energy Malnutrition (PEM) has lifelong consequences on brain development and cognitive function. We studied the lifelong developmental trajectories of resting-state EEG source activity in 66 individuals with histories of Protein Energy Malnutrition (PEM) limited to the first year of life and in 83 matched classmate controls (CON) who are all participants of the 49 years longitudinal Barbados Nutrition Study (BNS). qEEGt source z-spectra measured deviation from normative values of EEG rhythmic activity sources at 5-11 years of age and 40 years later at 45-51 years of age. The PEM group showed qEEGt abnormalities in childhood, including a developmental delay in alpha rhythm maturation and an insufficient decrease in beta activity. These profiles may be correlated with accelerated cognitive decline.

Characteristics and patient-reported outcomes associated with dropout in severely affected oncological patients: an exploratory study

BACKGROUND

Patient-reported outcome measures (PROMs) are commonly-used surrogates for clinical outcomes in cancer research. When researching severe diseases such as cancer, it is difficult to avoid the problem of incomplete questionnaires from drop-outs or missing data from patients who pass away during the observation period. The aim of this exploratory study was to explore patient characteristics and the patient-reported outcomes associated with the time-to-dropout.

METHODS

In an Oncological Social Care Project (OSCAR) study, the condition of the participants was assessed four times within 12 months (t0: baseline, t1: 3 months, t2: 6 months, and t3: 12 months) by validated PROMs. We performed competing-risk regressions based on Fine and Gray's proportional sub-distribution hazards model for exploring factors associated with time-to-dropout. Death was considered a competing risk.

RESULTS

Three hundred sixty-two participants were analyzed in the study. 193 (53.3%) completed a follow-up after 12 months, 67 (18.5%) patients dropped out, and 102 patients (28.2%) died during the study period. Poor subjective social support was related to a higher risk of drop-out (SHR = 2.10; 95%CI: 1.01-4.35). Lower values in health-related quality of life were related to drop-out and death. The sub-scales global health status/QoL, role functioning, physical functioning, and fatigue symptom in the EORTC QLQ-C30 were key characteristics of early drop-out.

CONCLUSION

Severely affected cancer patients with poor social support and poor quality of life seem more likely to drop out of studies than patients with higher levels of social support and a better quality of life. This should be considered when planning studies to assess advanced cancer patients. Methods of close continued monitoring should be actively used when patient experiences a substantial deterioration in their health-related quality of life and symptoms during the study. Results for such studies have to be interpreted with caution in light of specific drop-out mechanisms.

TRIAL REGISTRATION

OSCAR study was registered to the German Clinical Trials Register (DRKS-ID: DRKS00013640 ). Registered 29 December 2017.

How to Develop Statistical Predictive Risk Models in Oncology Nursing to Enhance Psychosocial and Supportive Care

OBJECTIVES

Predictive risk models are advocated in psychosocial oncology practice to provide timely and appropriate support to those likely to experience the emotional and psychological consequences of cancer and its treatments. New digital technologies mean that large scale and routine data collection are becoming part of everyday clinical practice. Using these data to try to identify those at greatest risk for late psychosocial effects of cancer is an attractive proposition in a climate of unmet need and limited resource. In this paper, we present a framework to support the development of high-quality predictive risk models in psychosocial and supportive oncology. The aim is to provide awareness and increase accessibility of best practice literature to support researchers in psychosocial and supportive care to undertake a structured evidence-based approach.

DATA SOURCES

Statistical prediction risk model publications.

CONCLUSION

In statistical modeling and data science different approaches are needed if the goal is to predict rather than explain. The deployment of a poorly developed and tested predictive risk model has the potential to do great harm. Recommendations for best practice to develop predictive risk models have been developed but there appears to be little application within psychosocial and supportive oncology care.

IMPLICATIONS FOR NURSING PRACTICE

Use of best practice evidence will ensure the development and validation of predictive models that are robust as these are currently lacking. These models have the potential to enhance supportive oncology care through harnessing routine digital collection of patient-reported outcomes and the targeting of interventions according to risk characteristics.

Handling Missing Data in the Short Form-12 Health Survey (SF-12): Concordance of Real Patient Data and Data Estimated by Missing Data Imputation Procedures

If information on single items in the Short Form–12 health survey (SF-12) is missing, the analysis of only complete cases causes a loss of statistical power and, in case of nonrandom missing data (MD), systematic bias. This study aimed at evaluating the concordance of real patient data and data estimated by different MD imputation procedures in the items of the SF-12 assessment. For this ends, MD were examined in a sample of 1,137 orthopedic patients. Additionally, MD were simulated (a) in the subsample of orthopedic patients exhibiting no MD (n = 810; 71%) as well as (b) in a sample of 6,970 respondents representing the German general population (95.8% participants with complete data) using logistic regression modelling. Simulated MD were replaced by mean values as well as regression-, expectation-maximization- (EM-), and multiple imputation estimates. Higher age and lower education were associated with enhanced probabilities of MD. In terms of accuracy in both data sets, the EM-procedure (ICC_2,1 = .33-.72) outperformed alternative estimation approaches substantially (e.g., regression imputation: ICC_2,1 = .18-.48). The EM-algorithm can be recommended to estimate MD in the items of the SF-12, because it reproduces the actual patient data most accurately.

Circulating folate concentrations and risk of coronary artery disease: a prospective cohort study in Chinese adults and a Mendelian randomization analysis

BACKGROUND

The association between circulating folate concentrations and risk of coronary artery disease (CAD) has been evaluated in Western populations with inconsistent results; however, the observational and causal associations in Chinese populations with relatively low folate concentrations remain unclear.

OBJECTIVES

We aimed to examine the association of circulating folate concentrations with incident CAD in Chinese adults, and further evaluated the causal relation using Mendelian randomization (MR) analysis.

METHODS

We measured baseline serum folate in 1605 incident CAD cases and 1605 age- and sex-matched controls nested within the Dongfeng-Tongji (DFTJ) cohort, which recruited 27,009 individuals with a mean age of 63.6 y in 2008-2010 and followed up until the end of 2013 (mean: 4.4 y). We quantified the observational association between folate and incident CAD using conditional logistic regression models. A 2-sample MR analysis was performed using summary statistics obtained for genetic variants identified from a genome-wide association study (GWAS) of circulating folate concentrations in participants of European ancestry (n = 37,341) and from the CardiogramplusC4D 1000 genomes-based GWAS meta-analysis (n = 184,305). We also conducted 1-sample MR among 1545 incident CAD cases and 1444 controls with genotyping data in the DFTJ cohort.

RESULTS

In the DFTJ cohort, higher serum folate concentrations were associated with a lower risk of CAD: the OR (95% CI) across sex-specific quartiles of folate (from lowest to highest concentrations) was 1.00 (reference), 0.78 (0.63, 0.97), 0.77 (0.61, 0.97), and 0.75 (0.60, 0.95), respectively (P-trend = 0.01). In the MR analysis, the OR of CAD per SD increase in genetically predicted serum folate was 0.99 (0.82, 1.20) and 0.88 (0.59, 1.32) for European and Chinese populations, respectively.

CONCLUSIONS

We found an inverse association between circulating folate concentrations and incident CAD among Chinese populations. However, we confirmed that there was no genetic evidence to support the causal relation in both European and Chinese populations.

Impact on health-related quality of life deterioration-free survival of a first-line therapy combining nab-paclitaxel plus either gemcitabine or simplified leucovorin and fluorouracil for patients with metastatic pancreatic cancer: Results of the randomized phase II AFUGEM GERCOR clinical trial

BACKGROUND

The phase II AFUGEM GERCOR trial aimed to assess the efficacy of a first-line therapy combining nab-paclitaxel plus either gemcitabine (gemcitabine group) or simplified leucovorin and fluorouracil (sLV5FU2 group) in patients with previously untreated metastatic pancreatic cancer. Results of progression-free survival at 4 months (primary endpoint) were in favor of the sLV5FU2 group. This paper presents health-related quality of life (HRQoL) data as a secondary endpoint.

METHODS

HRQoL was assessed using the EORTC QLQ-C30 questionnaire at baseline and at each chemotherapy cycle until the end of treatment. The HRQoL deterioration-free survival (QFS) was used as a modality of longitudinal analysis. QFS was defined as the time between randomization and the first definitive HRQoL score deterioration as compared to the baseline score, or death. Sensitivity analysis was performed excluding death as an event. Univariate Cox models were used to estimate hazard ratios (HRs) and 90% confidence intervals (CIs) of the treatment effect.

RESULTS

Between 2013 and 2014, 114 patients were randomized in a 1:2 ratio (39 in the gemcitabine group and 75 in the sLV5FU2 group). Patients in the sLV5FU2 group seemed to present longer QFS than those of the gemcitabine group for 14 out of 15 dimensions, with HRs < 1. Results of the sensitivity analysis excluding death as an event were significantly in favor of the sLV5FU2 group for physical functioning (HR = 0.51 [90% CI 0.27-0.97]) and pain (HR = 0.26 [90% CI 0.09-0.74]).

CONCLUSION

The nab-paclitaxel plus simplified leucovorin and fluorouracil combination had no negative impact in exploratory HRQoL analyses.

The consequences of traumatic brain injury from the classroom to the courtroom: understanding pathways through structural equation modelling

Purpose: Paediatric traumatic brain injury (TBI) can have resultant ongoing significant impairments which can impact life outcomes. The primary aim of this research was to explore whether TBI contributes to the relationship between poor educational outcomes and offending trajectories.Materials and methods: Through analysis of a dataset consisting of self-reported health, educational, and offending histories of 70 incarcerated young males, structural equation modelling was used to explore the mediation of educational outcomes and patterns in offending behaviour by chronic symptoms following TBI.Results: Symptoms related to TBI significantly mediated the relationship between decreased educational attainment and more frequent convictions. It did not mediate any relationships involving age at first conviction.Conclusions: Traumatic brain injury appears to have more influence over frequency of offending patterns than age at first conviction. However, TBI remains a pervasive factor in both higher rates of offending and poorer educational attainment. In order to tackle this effect on adverse social outcomes, greater attention to the impact of TBI is required in education and criminal justice systems.IMPLICATIONS FOR REHABILITATIONHighlights traumatic brain injury as a contributory factor in some education to offending pathways, suggesting that greater focus on rehabilitation within the education and criminal justice systems is required.Reinforces that greater understanding of educational pathways post-injury is needed to better facilitate rehabilitation within the school system.

Methodological aspects of health-related quality of life measurement and analysis in patients with multiple myeloma

Multiple myeloma (MM) is an incurable but treatment-sensitive cancer. For most patients, this means treatment with multiple lines of anti-myeloma therapy and a life with disease- and treatment-related symptoms and complications. Health-related quality of life (HRQoL) issues play an important role in treatment decision-making. Methodological challenges in longitudinal HRQoL measurements and analyses have been identified, including non-responses (NR) to scheduled questionnaires. Publications were identified for inclusion in a systematic review of longitudinal HRQoL studies in MM, focussing on methodological aspects of HRQoL measurement and analysis. Diversity in timing of HRQoL data collection and applied statistical methods were noted. We observed a high rate of NR, but the impact of NR was investigated in only 8/23 studies. Thus, evidence-based knowledge of HRQoL in patients with MM is compromised. To improve quality of HRQoL results and their implementation in daily practice, future studies should follow established guidelines.

Predicting Lung Function Following Lobectomy: A New Method to Adjust for Inherent Selection Bias

BACKGROUND

Predictions that overestimate post-lobectomy lung function are more likely than underestimates to lead to lobectomy. Studies of post-lobectomy lung function have included only surgical patients, so overestimates are overrepresented. This selection bias has led to incorrect estimates of prediction bias, which has led to inaccurate threshold values for determining lobectomy eligibility.

OBJECTIVE

The objective of this study was to demonstrate and adjust for this selection bias in order to arrive at correct estimates of prediction bias, the 95% limits of agreement, and adjusted threshold values for determining when exercise testing is warranted.

METHODS

We conducted a retrospective study of patients evaluated for lobectomy. We used multiple imputations to determine postoperative results for patients who did not have surgery because their predicted postoperative values were low. We combined these results with surgical patients to adjust for selection bias. We used the Bland-Altman method and the bivariate normal distribution to determine threshold values for surgical eligibility.

RESULTS

Lobectomy evaluation was performed in 114 patients; 79 had lobectomy while 35 were ineligible based on predicted values. Prediction bias using the Bland-Altman method changed significantly after controlling for selection bias. To achieve a postoperative FEV1 > 30% and DLCO ≥30%, a predicted FEV1 > 46% and DLCO ≥53% were required. Compared to current guidelines, using these thresholds would change management in 17% of cases.

CONCLUSION

The impact of selection bias on estimates of prediction accuracy was significant but can be corrected. Threshold values for determining surgical eligibility should be reassessed.

Simple descriptive missing data indicators in longitudinal studies with attrition, intermittent missing data and a high number of follow-ups

Objective

Missing data in longitudinal studies may constitute a source of bias. We suggest three simple missing data indicators for the initial phase of getting an overview of the missingness pattern in a dataset with a high number of follow-ups. Possible use of the indicators is exemplified in two datasets allowing wave nonresponse; a Norwegian dataset of 420 subjects examined at 21 occasions during 6.5 years and a Dutch dataset of 350 subjects with ten repeated measurements over a period of 35 years.

Results

The indicators Last response (the timing of last response), Retention (the number of responded follow-ups), and Dispersion (the evenness of the distribution of responses) are introduced. The proposed indicators reveal different aspects of the missing data pattern, and may give the researcher a better insight into the pattern of missingness in a study with several follow-ups, as a starting point for analyzing possible bias. Although the indicators are positively correlated to each other, potential predictors of missingness can have a different relationship with different indicators leading to a better understanding of the missing data mechanism in longitudinal studies. These indictors may be useful descriptive tools when starting to look into a longitudinal dataset with many follow-ups.

Electronic supplementary material

The online version of this article (10.1186/s13104-018-3228-6) contains supplementary material, which is available to authorized users.

Value of quality of life analysis in liver cancer: A clinician’s perspective

Health related quality of life (HRQOL) is increasingly recognized as an important clinical parameter and research endpoint in patients with hepatocellular carcinoma (HCC). HRQOL in HCC patients is multifaceted and affected by medical factor which encompasses HCC and its complications, oncological and palliative treatment for HCC, underlying liver disease, as well as the psychological, social or spiritual reaction to the disease. Many patients presented late with advanced disease and limited survival, plagued with multiple symptoms, rendering QOL a very important aspect in their general well being. Various instruments have been developed and validated to measure and report HRQOL in HCC patients, these included general HRQOL instruments, e.g., Short form (SF)-36, SF-12, EuroQoL-5D, World Health Organization Quality of Life Assessment 100 (WHOQOL-100), World Health Organization Quality of Life Assessment abbreviated version; general cancer HRQOL instruments, e.g., the European Organisation for Research and Treatment of Cancer (EORTC) QLQ-C30, Functional Assessment of Cancer Therapy (FACT)-General, Spitzer Quality of Life Index; and liver-cancer specific HRQOL instruments, e.g., EORTC QLQ-HCC18, FACT-Hepatobiliary (FACT-Hep), FACT-Hep Symptom Index, Trial Outcome Index. Important utilization of HRQOL in HCC patients included description of symptomatology and HRQOL of patients, treatment endpoint in clinical trial, prognostication of survival, benchmarking of palliative care service and health care valuation. In this review, difficulties regarding the use of HRQOL data in research and clinical practice, including choosing a suitable instrument, problems of missing data, data interpretation, analysis and presentation are examined. Potential solutions are also discussed.

Modernizing quality of life assessment: development of a multidimensional computerized adaptive questionnaire for patients with schizophrenia

OBJECTIVE

Quality of life (QoL) is still assessed using paper-based and fixed-length questionnaires, which is one reason why QoL measurements have not been routinely implemented in clinical practice. Providing new QoL measures that combine computer technology with modern measurement theory may enhance their clinical use. The aim of this study was to develop a QoL multidimensional computerized adaptive test (MCAT), the SQoL-MCAT, from the fixed-length SQoL questionnaire for patients with schizophrenia.

METHODS

In this multicentre cross-sectional study, we collected sociodemographic information, clinical characteristics (i.e., duration of illness, the PANSS, and the Calgary Depression Scale), and quality of life (i.e., SQoL). The development of the SQoL-CAT was divided into three stages: (1) multidimensional item response theory (MIRT) analysis, (2) multidimensional computerized adaptive test (MCAT) simulations with analyses of accuracy and precision, and (3) external validity.

RESULTS

Five hundred and seventeen patients participated in this study. The MIRT analysis found that all items displayed good fit with the multidimensional graded response model, with satisfactory reliability for each dimension. The SQoL-MCAT was 39% shorter than the fixed-length SQoL questionnaire and had satisfactory accuracy (levels of correlation >0.9) and precision (standard error of measurement <0.55 and root mean square error <0.3). External validity was confirmed via correlations between the SQoL-MCAT dimension scores and symptomatology scores.

CONCLUSION

The SQoL-MCAT is the first computerized adaptive QoL questionnaire for patients with schizophrenia. Tailored for patient characteristics and significantly shorter than the paper-based version, the SQoL-MCAT may improve the feasibility of assessing QoL in clinical practice.

Reporting and dealing with missing quality of life data in RCTs: has the picture changed in the last decade?

PURPOSE

Missing data are a major problem in the analysis of data from randomised trials affecting power and potentially producing biased treatment effects. Specifically focussing on quality of life outcomes, we aimed to report the amount of missing data, whether imputation was used and what methods and was the missing mechanism discussed from four leading medical journals and compare the picture to our previous review nearly a decade ago.

METHODS

A random selection (50 %) of all RCTS published during 2013-2014 in BMJ, JAMA, Lancet and NEJM was obtained. RCTs reported in research letters, cluster RCTs, non-randomised designs, review articles and meta-analysis were excluded.

RESULTS

We included 87 RCTs in the review of which 35 % the amount of missing primary QoL data was unclear, 31 (36 %) used imputation. Only 23 % discussed the missing data mechanism. Nearly half used complete case analysis. Reporting was more unclear for secondary QoL outcomes. Compared to the previous review, multiple imputation was used more prominently but mainly in sensitivity analysis.

CONCLUSIONS

Inadequate reporting and handling of missing QoL data in RCTs are still an issue. There is a large gap between statistical methods research relating to missing data and the use of the methods in applications. A sensitivity analysis should be undertaken to explore the sensitivity of the main results to different missing data assumptions. Medical journals can help to improve the situation by requiring higher standards of reporting and analytical methods to deal with missing data, and by issuing guidance to authors on expected standard.

Ovarian cancer study dropouts had worse health-related quality of life and psychosocial symptoms at baseline and over time

AIMS

Participant drop out is a major barrier to high-quality patient-reported outcome (PRO) data analysis in cancer research as patients with worsening health are more likely to dropout. To test the hypothesis that ovarian cancer patients with worse PROs would drop out earlier, we examined how patients differed by time of dropout on health-related quality of life (HRQOL), anxiety, depression, optimism and insomnia.

METHODS

This analysis included 619 participants, stratified by time of dropout, from the Australian Ovarian Cancer Study - Quality of Life substudy, in which participants completed PRO questionnaires at three-monthly intervals for 21 months. Trends in PROs over time were examined. Pearson correlations examined the relationship between time of dropout and baseline PROs. Multiple linear regression models including age, disease stage and time since diagnosis examined relationships between baseline and final PRO scores, and final PRO scores and dropout group.

RESULTS

Participants who dropped out earlier had significantly worse baseline HRQOL (p < 0.0001) and higher depression (p < 0.0001). For all five PROs, final scores were significantly associated with baseline scores (p < 0.0001). Time of dropout was significantly associated with final HRQOL (p = 0.003), anxiety (p = 0.05), depression (p = 0.02) and optimism (p = 0.02) scores. Depression, HRQOL and anxiety worsened at a faster rate overtime in dropouts than study completers.

CONCLUSIONS

Poorer HRQOL and higher depression at baseline, and final HRQOL, anxiety, depression and optimism scores were predictive of time of dropout. These results highlight the importance of collecting auxiliary data to inform careful and considered handling of missing PRO data during analysis, interpretation and reporting.

Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: a systematic review

OBJECTIVES

Patient-reported outcomes (PROs) provide important information about the impact of treatment from the patients' perspective. However, missing PRO data may compromise the interpretability and value of the findings. We aimed to report: (1) a non-technical summary of problems caused by missing PRO data; and (2) a systematic review by collating strategies to: (A) minimise rates of missing PRO data, and (B) facilitate transparent interpretation and reporting of missing PRO data in clinical research. Our systematic review does not address statistical handling of missing PRO data.

DATA SOURCES

MEDLINE and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases (inception to 31 March 2015), and citing articles and reference lists from relevant sources.

ELIGIBILITY CRITERIA

English articles providing recommendations for reducing missing PRO data rates, or strategies to facilitate transparent interpretation and reporting of missing PRO data were included.

METHODS

2 reviewers independently screened articles against eligibility criteria. Discrepancies were resolved with the research team. Recommendations were extracted and coded according to framework synthesis.

RESULTS

117 sources (55% discussion papers, 26% original research) met the eligibility criteria. Design and methodological strategies for reducing rates of missing PRO data included: incorporating PRO-specific information into the protocol; carefully designing PRO assessment schedules and defining termination rules; minimising patient burden; appointing a PRO coordinator; PRO-specific training for staff; ensuring PRO studies are adequately resourced; and continuous quality assurance. Strategies for transparent interpretation and reporting of missing PRO data include utilising auxiliary data to inform analysis; transparently reporting baseline PRO scores, rates and reasons for missing data; and methods for handling missing PRO data.

CONCLUSIONS

The instance of missing PRO data and its potential to bias clinical research can be minimised by implementing thoughtful design, rigorous methodology and transparent reporting strategies. All members of the research team have a responsibility in implementing such strategies.

Identifying reprioritization response shift in a stroke caregiver population: a comparison of missing data methods

PURPOSE

Response shift (RS) is an important phenomenon that influences the assessment of longitudinal changes in health-related quality of life (HRQOL) studies. Given that RS effects are often small, missing data due to attrition or item non-response can contribute to failure to detect RS effects. Since missing data are often encountered in longitudinal HRQOL data, effective strategies to deal with missing data are important to consider. This study aims to compare different imputation methods on the detection of reprioritization RS in the HRQOL of caregivers of stroke survivors.

METHODS

Data were from a Canadian multi-center longitudinal study of caregivers of stroke survivors over a one-year period. The Stroke Impact Scale physical function score at baseline, with a cutoff of 75, was used to measure patient stroke severity for the reprioritization RS analysis. Mean imputation, likelihood-based expectation-maximization imputation, and multiple imputation methods were compared in test procedures based on changes in relative importance weights to detect RS in SF-36 domains over a 6-month period. Monte Carlo simulation methods were used to compare the statistical powers of relative importance test procedures for detecting RS in incomplete longitudinal data under different missing data mechanisms and imputation methods.

RESULTS

Of the 409 caregivers, 15.9 and 31.3 % of them had missing data at baseline and 6 months, respectively. There were no statistically significant changes in relative importance weights on any of the domains when complete-case analysis was adopted. But statistical significant changes were detected on physical functioning and/or vitality domains when mean imputation or EM imputation was adopted. There were also statistically significant changes in relative importance weights for physical functioning, mental health, and vitality domains when multiple imputation method was adopted. Our simulations revealed that relative importance test procedures were least powerful under complete-case analysis method and most powerful when a mean imputation or multiple imputation method was adopted for missing data, regardless of the missing data mechanism and proportion of missing data.

CONCLUSIONS

Test procedures based on relative importance measures are sensitive to the type and amount of missing data and imputation method. Relative importance test procedures based on mean imputation and multiple imputation are recommended for detecting RS in incomplete data.

A systematic review finds variable use of the intention-to-treat principle in musculoskeletal randomized controlled trials with missing data

OBJECTIVES

In randomized trials, the primary analysis should be consistent with the intention-to-treat (ITT) principle and should address missing data appropriately to draw valid inferences. This review focuses on current practices relating to the ITT principle and methods to handle missing data in the major musculoskeletal journals.

STUDY DESIGN AND SETTING

A systematic review of randomized trials published in 2010 and 2011 in five musculoskeletal journals was performed.

RESULTS

We reviewed 91 trials: 38% performed a full ITT analysis (analyzing outcome data for all randomized participants) and 31% performed a partial ITT analysis (excluding participants with no follow-up data). The overall median dropout was 12%; 60% of trials had more than 10% dropouts, and 32% of trials had more than 20% dropouts. Among those that performed an ITT analysis, the majority adopted a form of single imputation; last observation carried forward was the designated approach in most cases. Mixed models for repeated measures and/or multiple imputations were limited to eight trials.

CONCLUSION

It appears that many trials reporting missing data are inappropriately analyzed and may therefore be prone to biased estimates and invalid inferences.

Characteristics of missing physical activity data in children and youth

PURPOSE

The purpose of this study was to investigate the characteristics of missing physical activity (PA) data of children and youth.

METHOD

PA data from the Chinese City Children and Youth Physical Activity Study (N = 2,758; 1,438 boys and 1,320 girls; aged 9-17 years old) were used for the study. After the data were sorted by the weekday (WD) and recording day (RD), the missing ratio (MR) was calculated by gender, age, and body mass index (BMI). Chi-square tests were used to determine the effect of WD and RD on missing data. The joint impact (WD x RD) on the MR, as well as their interactions with age, gender, and BMI, were also analyzed.

RESULTS

Out of a total of 19,306 records, 5,400 (28.0%) were missing. The total MR significantly differed by WD and RD. There were more missing data during weekend days than during WDs, with the highest being on Sunday (36.2%). Older youth (aged 15-17 years old) had more missing data than did the 9- to 11-year-old group and 12- to 14-year-old group. In terms of RD, the 7th day had the most total missing data (36.0%), and again, older youth (15-17 years old) had more missing data than did the other 2 age groups. Gender and BMI had no impact on missing data by WD or RD. When the joint effect of WD and RD was examined, it was found that providing the measurement devices on Thursday and collecting data from Friday onward resulted in the lowest amount of missing data (18.5%).

CONCLUSION

This study examined missing data characteristics with different sorting orders, which may help design more effective measures to prevent missing data in future PA measurement using accelerometers.

Tensor-based methods for handling missing data in quality-of-life questionnaires

A common problem with self-report quality-of-life questionnaires is missing data. Despite enormous care and effort to prevent it, some level of missing data is common and unavoidable. Missing data can have a detrimental impact on the data analysis. In this paper, a novel approach to imputing missing data in quality-of-life questionnaires is proposed, based on matrix and tensor decompositions. In order to illustrate and assess those methods, two datasets are considered: The first dataset contains the responses of 100 patients to a systemic lupus erythematosus-specific quality-of-life questionnaire; the other contains the responses of 43 patients to a rhino-conjunctivitis quality-of-life questionnaire. The two datasets contain almost no missing data, and for testing purposes, data entries are removed at random to have missing completely at random data. Several proportions of missing values are considered, and for each, the imputation error is assessed through k-fold cross validation. We also evaluate different imputation methods for missing at random and missing not at randomdata. The numerical results demonstrate that the proposed tensor factorization-based methods outperform standard methods in terms of root mean square error with at least 4% improvement, while the bias and variance are similar.

IQuaD dental trial; improving the quality of dentistry: a multicentre randomised controlled trial comparing oral hygiene advice and periodontal instrumentation for the prevention and management of periodontal disease in dentate adults attending dental primary care

BACKGROUND

Periodontal disease is the most common oral disease affecting adults, and although it is largely preventable it remains the major cause of poor oral health worldwide. Accumulation of microbial dental plaque is the primary aetiological factor for both periodontal disease and caries. Effective self-care (tooth brushing and interdental aids) for plaque control and removal of risk factors such as calculus, which can only be removed by periodontal instrumentation (PI), are considered necessary to prevent and treat periodontal disease thereby maintaining periodontal health. Despite evidence of an association between sustained, good oral hygiene and a low incidence of periodontal disease and caries in adults there is a lack of strong and reliable evidence to inform clinicians of the relative effectiveness (if any) of different types of Oral Hygiene Advice (OHA). The evidence to inform clinicians of the effectiveness and optimal frequency of PI is also mixed. There is therefore an urgent need to assess the relative effectiveness of OHA and PI in a robust, sufficiently powered randomised controlled trial (RCT) in primary dental care.

METHODS/DESIGN

This is a 5 year multi-centre, randomised, open trial with blinded outcome evaluation based in dental primary care in Scotland and the North East of England. Practitioners will recruit 1860 adult patients, with periodontal health, gingivitis or moderate periodontitis (Basic Periodontal Examination Score 0-3). Dental practices will be cluster randomised to provide routine OHA or Personalised OHA. To test the effects of PI each individual patient participant will be randomised to one of three groups: no PI, 6 monthly PI (current practice), or 12 monthly PI.Baseline measures and outcome data (during a three year follow-up) will be assessed through clinical examination, patient questionnaires and NHS databases.The primary outcome measures at 3 year follow up are gingival inflammation/bleeding on probing at the gingival margin; oral hygiene self-efficacy and net benefits.

DISCUSSION

IQuaD will provide evidence for the most clinically-effective and cost-effective approach to managing periodontal disease in dentate adults in Primary Care. This will support general dental practitioners and patients in treatment decision making.

TRIAL REGISTRATION

Protocol ID: ISRCTN56465715.

Can electronic web-based technology improve quality of life data collection? Analysis of Radiation Therapy Oncology Group 0828

PURPOSE

Missing data are a significant problem in clinical trials, particularly for quality of life (QOL), which cannot be obtained retrospectively. The purpose of this study was to evaluate the feasibility of an electronic web-based strategy for QOL data collection in a cooperative group radiation oncology trial setting.

METHODS AND MATERIALS

Radiation Therapy Oncology Group (RTOG) 0828 was a prospective National Cancer Institute cooperative group companion study of RTOG-0415, a randomized study of conventional versus hypofractionated radiation. Forty-nine English-speaking patients with favorable risk prostate cancer who enrolled on RTOG-0415 consented to using web-based technology for completing QOL. In RTOG-0415, using paper forms, the 6-month QOL compliance rate was 52%. The purpose of RTOG-0828 was to test the feasibility of a web-based strategy with the goal of increasing the 6-month QOL completion rate by 25% (from 52% to 77%) for a relative improvement of ~50%. The web-based tool used in this study was VisionTree Optimal Care (VTOC; VisionTree Software, Inc, San Diego, CA), a Health-Insurance-Portability-Accountability-Act secure, online technology that allows real-time tracking and e-mail reminders. The primary endpoint was the 6-month compliance rate for the validated QOL instrument, Expanded Prostate Index Composite.

RESULTS

The QOL completion rate at baseline was 98%. Compared with the prior 52% QOL completion rate at 6 months using paper forms, the QOL web-based completion rate at 6 months was 90% (2-sided P value < .001). At 12 months, the EPIC completion rate was 82% (compared with 36% using paper forms).

CONCLUSIONS

This RTOG study suggests that a web-based strategy to collect QOL appears to be feasible in the cooperative group radiation oncology trial setting and is associated with an increase in the 6-month QOL compliance rate compared with the prior method of using paper forms. The RTOG plans to further test this strategy in a head-and-neck cancer trial across all participating RTOG sites.

Health-related quality of life in women with newly diagnosed polycystic ovary syndrome randomized between clomifene citrate plus metformin or clomifene citrate plus placebo

STUDY QUESTION

What is the health-related quality of life (HRQoL) in women with polycystic ovary syndrome (PCOS) undergoing ovulation induction with clomifene citrate (CC) combined with metformin compared with those using CC combined with placebo?

SUMMARY ANSWER

Overall quality of life in women with PCOS treated with CC plus metformin was significantly lower than in women treated with CC plus placebo.

WHAT IS KNOWN ALREADY

There are no data on HRQoL in adult women who receive ovulation induction with the purpose of conceiving. Women with PCOS have higher scores on depression and anxiety scales and lower QoL scores than women without PCOS.

STUDY DESIGN, SIZE AND DURATION

This study was a secondary analysis of a multi-centre RCT completed between June 2001 and May 2004. The randomization was stratified per centre, and the centres received blinded, numbered containers with medication. There were172 women available for the HRQoL assessment: 85 were allocated to metformin and 87 were allocated to placebo.

PARTICIPANTS, SETTING AND METHODS

The Rotterdam Symptom Checklist (RSCL), a standard self-administered questionnaire, was used to assess physical symptoms, psychological distress, activity levels and overall HRQoL.

MAIN RESULTS AND THE ROLE OF CHANCE

In the intention to treat analysis, we found differences between the treatment groups with respect to physical symptoms and overall HRQoL. Physical well-being was significantly impaired in women allocated to metformin but not in women allocated to placebo. The increase in physical symptoms in the metformin group was caused by side-effects typical of metformin, and was most pronounced at Week 1 (mean difference 12 [95% confidence interval (CI): 8-16] and still apparent at Week 16 [mean difference 7 (95% CI 2-12]. Overall well-being was significantly impaired in the metformin group compared with the placebo group [mean difference 13 (95% CI 6-20)].

LIMITATIONS AND REASONS FOR CAUTION

RSCL measurements were available only for three quarters of the participants. Although the number of missing questionnaires and the baseline measurements, were comparable between the treatment groups, some form of selection bias cannot be ruled out.

WIDER IMPLICATIONS OF THE FINDINGS

Our finding that metformin was more burdensome than placebo, strengthens the recommendation that CC only and not CC plus metformin should be the drug of choice in this patient population.

STUDY FUNDING/COMPETING INTEREST(S)

None of the authors declared a conflict of interest. There was no study funding.

TRIAL REGISTRATION NUMBER

ISRCTN55906981.

Do the methods used to analyse missing data really matter? An examination of data from an observational study of Intermediate Care patients

Background

Missing data is a common statistical problem in healthcare datasets from populations of older people. Some argue that arbitrarily assuming the mechanism responsible for the missingness and therefore the method for dealing with this missingness is not the best option—but is this always true? This paper explores what happens when extra information that suggests that a particular mechanism is responsible for missing data is disregarded and methods for dealing with the missing data are chosen arbitrarily.

Regression models based on 2,533 intermediate care (IC) patients from the largest evaluation of IC done and published in the UK to date were used to explain variation in costs, EQ-5D and Barthel index. Three methods for dealing with missingness were utilised, each assuming a different mechanism as being responsible for the missing data: complete case analysis (assuming missing completely at random—MCAR), multiple imputation (assuming missing at random—MAR) and Heckman selection model (assuming missing not at random—MNAR). Differences in results were gauged by examining the signs of coefficients as well as the sizes of both coefficients and associated standard errors.

Results

Extra information strongly suggested that missing cost data were MCAR. The results show that MCAR and MAR-based methods yielded similar results with sizes of most coefficients and standard errors differing by less than 3.4% while those based on MNAR-methods were statistically different (up to 730% bigger). Significant variables in all regression models also had the same direction of influence on costs. All three mechanisms of missingness were shown to be potential causes of the missing EQ-5D and Barthel data. The method chosen to deal with missing data did not seem to have any significant effect on the results for these data as they led to broadly similar conclusions with sizes of coefficients and standard errors differing by less than 54% and 322%, respectively.

Conclusions

Arbitrary selection of methods to deal with missing data should be avoided. Using extra information gathered during the data collection exercise about the cause of missingness to guide this selection would be more appropriate.

Methods in public health services and systems research: a systematic review

CONTEXT

Public Health Services and Systems Research (PHSSR) is concerned with evaluating the organization, financing, and delivery of public health services and their impact on public health. The strength of the current PHSSR evidence is somewhat dependent on the methods used to examine the field. Methods used in PHSSR articles, reports, and other documents were reviewed to assess their methodologic strengths and challenges in light of PHSSR goals.

EVIDENCE ACQUISITION

A total of 364 documents from the PHSSR library met the inclusion criteria as empirical and based in the U.S. After additional exclusions, 327 of these were analyzed.

EVIDENCE SYNTHESIS

A detailed codebook was used to classify articles in terms of (1) study design; (2) sampling; (3) instrumentation; (4) data collection; (5) data analysis; and (6) study validity. Inter-coder reliability was assessed for the codebook; once it was found reliable, the available empirical documents were coded.

CONCLUSIONS

Although there has been a dramatic increase in the amount of published PHSSR recently, methods used remain primarily cross-sectional and descriptive. Moreover, although appropriate for exploratory and foundational work in a new field, these approaches are limiting progress toward some PHSSR goals. Recommendations are given to advance and strengthen the methods used in PHSSR to better meet the goals and challenges facing the field.

Compensation effects on clinical trial data collection in opioid-dependent young adults

BACKGROUND

Attrition in studies of substance use disorder treatment is problematic, potentially introducing bias into data analysis.

OBJECTIVES

This study aimed to determine the effect of participant compensation amounts on rates of missing data and observed rates of drug use.

METHODS

We performed a secondary analysis of a clinical trial of buprenorphine/naloxone among 152 treatment-seeking opioid-dependent subjects aged 15-21 during participation in a randomized trial. Subjects were randomized to a 2-week detoxification with buprenorphine/naloxone (DETOX; N = 78) or 12 weeks buprenorphine/naloxone (BUP; N = 74). Participants were compensated $5 for weekly urine drug screens and self-reported drug use information and $75 for more extensive assessments at weeks 4, 8, and 12.

RESULTS

Though BUP assignment decreased the likelihood of missing data, there were significantly less missing data at 4, 8, and 12 weeks than other weeks, and the effect of compensation on the probability of urine screens being positive was more pronounced in DETOX subjects.

CONCLUSION

These findings suggest that variations in the amount of compensation for completing assessments can differentially affect outcome measurements, depending on treatment group assignment.

SCIENTIFIC SIGNIFICANCE

Adequate financial compensation may minimize bias when treatment condition is associated with differential dropout and may be a cost-effective way to reduce attrition. Moreover, active users may be more likely than non-active users to drop out if compensation is inadequate, especially in control groups or in groups who are not receiving active treatment.

Inverse probability weighting to control for censoring in a post hoc analysis of quality-adjusted survival data from a clinical trial of temsirolimus for renal cell carcinoma

OBJECTIVE

This post hoc analysis evaluated treatment-associated quality-adjusted survival (QAS) in patients randomly assigned to temsirolimus or interferon alfa (IFN-alfa), corrected for censoring using inverse probability weighting (IPW), in the Advanced Renal Cell Carcinoma (ARCC) trial.

METHODS

Follow-up was divided into 11 time intervals; Kaplan-Meier estimates for not being censored were estimated for each interval. The QAS for each interval was weighted by the inverse probability of not being censored in that interval. Overall treatment-associated QAS was calculated as the sum of the weighted QAS across all follow-up intervals. Differences in mean QAS between temsirolimus and IFN-alfa were evaluated with t-statistics at a two-sided α = 0.05.

RESULTS

In total, 416 patients were randomly assigned to temsirolimus (n = 209) or IFN-alfa (n = 207); 400 patients were included in this analysis. Overall weighted mean (standard deviation) QAS during progression-free survival was 111.9 (5.3) days with temsirolimus (n = 204) and 75.7 (6.3) days with IFN-alfa (n = 196). The mean weighted QAS difference of 36.2 days in favor of temsirolimus was significant (p < 0.05).

LIMITATIONS

One potential limitation is that the weights developed by the Kaplan-Meier estimates did not allow for covariates to be adjusted among treatment arms. Another possible limitation is that the ARCC trial included patients with advanced renal cell carcinoma, and thus it cannot be conclusively determined how our findings would apply to patients with less advanced disease.

CONCLUSIONS

Patients with poor-prognosis advanced renal cell carcinoma treated with temsirolimus had an incremental gain of 48% (36.2 days) in QAS compared with patients treated with IFN-alfa.

Missing data methods for dealing with missing items in quality of life questionnaires. A comparison by simulation of personal mean score, full information maximum likelihood, multiple imputation, and hot deck techniques applied to the SF-36 in the French 2003 decennial health survey

PURPOSE

Missing items are common in quality of life (QoL) questionnaires and present a challenge for research in this field. It remains unclear which of the various methods proposed to deal with missing data performs best in this context. We compared personal mean score, full information maximum likelihood, multiple imputation, and hot deck techniques using various realistic simulation scenarios of item missingness in QoL questionnaires constructed within the framework of classical test theory.

METHODS

Samples of 300 and 1,000 subjects were randomly drawn from the 2003 INSEE Decennial Health Survey (of 23,018 subjects representative of the French population and having completed the SF-36) and various patterns of missing data were generated according to three different item non-response rates (3, 6, and 9%) and three types of missing data (Little and Rubin's "missing completely at random," "missing at random," and "missing not at random"). The missing data methods were evaluated in terms of accuracy and precision for the analysis of one descriptive and one association parameter for three different scales of the SF-36.

RESULTS

For all item non-response rates and types of missing data, multiple imputation and full information maximum likelihood appeared superior to the personal mean score and especially to hot deck in terms of accuracy and precision; however, the use of personal mean score was associated with insignificant bias (relative bias <2%) in all studied situations.

CONCLUSIONS

Whereas multiple imputation and full information maximum likelihood are confirmed as reference methods, the personal mean score appears nonetheless appropriate for dealing with items missing from completed SF-36 questionnaires in most situations of routine use. These results can reasonably be extended to other questionnaires constructed according to classical test theory.

Predicting missing quality of life data that were later recovered: an empirical comparison of approaches

Background and Purpose The aim was to compare simple imputation, multiple imputation, and modeling approaches to deal with ‘missing’ quality of life data. Data were obtained from five clinical trials, which employed a reminder system for follow-up questionnaires. Previous studies have compared imputation strategies by artificially removing data according to prespecified mechanisms. Our approach differs from previous study as actual collected data are utilized.

Methods Data obtained by reminder were initially treated as missing. These missing values were imputed using a variety of simple and multiple imputation strategies. The trials were analyzed using the imputed datasets, and the resulting treatment effects compared to analyses using the full dataset including responses following reminders. A repeated measures model was also carried out on the available data and the pattern mixture models were employed. The accuracy of the different strategies was assessed by calculating the bias seen in the calculated treatment difference compared to the actual observed treatment difference.

Results Baseline carried forward or last value carried forward were shown to be the best simple imputation methods in this setting. Multiple imputation using a regression model or predictive mean match model tended to provide treatment difference estimates with the least bias when compared to the actual observed data. Pattern mixture models did not perform well. Overall, the multiple imputation procedures were generally the least biased approaches.

Limitations A number of imputation and modeling procedures have been investigated but this list is not exhaustive. All the example datasets come from the same data source and perhaps studies from additional disease areas would have been useful. However, we feel the results are generalizable to other quality of life outcomes and clinical areas.

Conclusions Multiple imputation is recommended for missing quality of life data as it makes the assumption of missing at random which in the quality of life setting is more plausible than the assumption of missing completely at random for which most simple imputation methods are based. Pattern mixture models can be complex and did not perform well in this setting. Clinical Trials 2010; 7: 333—342. http://ctj.sagepub.com