On the prevention and analysis of missing data in randomized clinical trials: the state of the art

Health Equity Implications of Missing Data Among Youths With Childhood-Onset Systemic Lupus Erythematosus: A Proof-of-Concept Study in the Childhood Arthritis and Rheumatology Research Alliance Registry

OBJECTIVE

Health disparities in childhood-onset systemic lupus erythematosus (SLE) disproportionately impact marginalized populations. Socioeconomically patterned missing data can magnify existing health inequities by supporting inferences that may misrepresent populations of interest. Our objective was to assess missing data and subsequent health equity implications among participants with childhood-onset SLE enrolled in a large pediatric rheumatology registry.

METHODS

We evaluated co-missingness of 12 variables representing demographics, socioeconomic position, and clinical factors (e.g., disease-related indices) using Childhood Arthritis and Rheumatology Research Alliance Registry childhood-onset SLE enrollment data (2015-2022; n = 766). We performed logistic regression to calculate odds ratios (ORs) and 95% confidence intervals (95% CIs) for missing disease-related indices at enrollment (Systemic Lupus Erythematosus Disease Activity Index 2000 [SLEDAI-2K] and/or Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index [SDI]) associated with data missingness. We used linear regression to assess the association between socioeconomic factors and SLEDAI-2K at enrollment using 3 analytic methods for missing data: complete case analysis, multiple imputation, and nonprobabilistic bias analyses, with missing values imputed to represent extreme low or high disadvantage.

RESULTS

On average, participants were missing 6.2% of data, with over 50% of participants missing at least 1 variable. Missing data correlated most closely with variables within data categories (i.e., demographic). Government-assisted health insurance was associated with missing SLEDAI-2K and/or SDI scores compared to private health insurance (OR 2.04 [95% CI 1.22, 3.41]). The different analytic approaches resulted in varying analytic sample sizes and fundamentally conflicting estimated associations.

CONCLUSION

Our results support intentional evaluation of missing data to inform effect estimate interpretation and critical assessment of causal statements that might otherwise misrepresent health inequities.

Factors Associated with Intention of Serbian Public Health Workers to Leave the Job: A Cross-Sectional, Population-Based Study

Recruitment and retention of public health workers (PHWs) is crucial for the optimal functioning of the public health system at a time of budget cuts and the threat of a pandemic. Individual and job-related variables were examined by univariate and multivariate logistic regression to identify predictors of the intention to leave a job during the COVID-19 outbreak among Serbian PHWs in 25 institutes of public health (n = 1663 respondents, of which 73.1% were female). A total of 20.3% of PHWs intended to leave their current job within the next five years. Males and persons aged younger than 55 years who had additional practice were more likely to report an intention to leave their job than females, those older than 54 years and those without additional work. While uncertainty and fear of infection during the COVID-19 pandemic were almost perceived as job attractiveness, other job-related characteristics were identified as significant barriers to maintaining the sufficient capacity of qualified PHWs in the future. Authorities need to address these factors, including the following: the feeling of tension, stress or pressure, and unavailability of information during the COVID-19 pandemic, as well as dissatisfaction with respect, valuation, and the job in general.

To attend, or not to attend: Examining caregiver intentions and study compliance in a pediatric, randomized controlled trial

BACKGROUND/AIMS

The Intent to Attend is a brief questionnaire recommended by the National Research Council to address dropout concerns and improve prediction of missing data in clinical trials, although implementation has been very limited. As a formative study in pediatric research, the relationship between caregiver intentions and study compliance was investigated in a 180-day trial of dietary supplementation of preterm toddlers. Treatment effect estimation in the context of missing data was also explored.

METHODS

Study compliance (i.e. study completion, supplement adherence, and diary completion) was tracked over three study visits. Baseline questionnaires asked caregivers about intentions concerning study completion via the Intent to Attend, screened for mental health symptoms (depression, trait anxiety), and captured family demographics. Simple and multiple logistic regression models were built to examine associations between caregiver intent and compliance outcomes. The Intent to Attend was also employed as an auxiliary variable to account for missing data within mixed models estimating the treatment effect on the primary outcomes.

RESULTS

Of the 316 caregiver-child dyads included, 95% of caregivers with low intentions had a child complete the study, but only 87% of caregivers with high intentions had a child complete the study. Low intentions to complete the study were associated with a more than 60% lower odds of study non-completion, but the confidence interval included the null (odds ratio: 0.36; 95% confidence interval: 0.11, 1.20). No effect measure modification by caregiver mental health, child sex, or annual income was detected. Income was the only significant predictor of study non-completion; the lowest income group was almost four times more likely to be study non-completers compared with the highest income group, even after adjustment for child sex and caregiver mental health (adjusted odds ratio = 3.59, 95% confidence interval: 1.38, 9.31). When using Intent to Attend as an auxiliary variable, similar results were obtained when compared with the original treatment effect estimates on the primary outcomes.

CONCLUSION

Contrary to prior adult studies, there is no clear relationship between caregiver intentions and study compliance. Findings elucidate the complexities of caregiver-child interactions during pediatric trial participation.

A prospective, international, randomized, noninferiority study comparing an implantable titanium vertebral augmentation device versus balloon kyphoplasty in the reduction of vertebral compression fractures (SAKOS study)

BACKGROUND CONTEXT

Balloon kyphoplasty (BKP) is a commonly performed vertebral augmentation procedure for painful osteoporotic vertebral compression fractures (OVCFs).

OBJECTIVE

This study aimed to support a non-inferiority finding for the use of a titanium implantable vertebral augmentation device (TIVAD) compared to BKP.

STUDY DESIGN

Prospective, parallel group, controlled comparative randomized study.

PATIENT SAMPLE

Patients who presented with one or two painful OVCFs located between T7 and L4 aged <3 months, failed conservative treatment, and had an Oswestry Disability Index (ODI) score ≥30/100 were eligible for the study.

OUTCOME MEASURES

The primary composite endpoint was defined as: reduction in VCF fracture-related pain at 12 months from baseline and maintenance or functional improvement (ODI) at 12 months from baseline, and absence of device-related adverse event or surgical reintervention. If the primary composite endpoint was successful, a fourth component (absence of adjacent level fracture) was added for analysis. If the analysis of this additional composite endpoint was successful, then midline target height restoration at 6 and 12 months was assessed. Secondary clinical outcomes included back pain intensity, ODI score, EQ-5D index score (range 0=death to 1=full health) and EQ-VAS score (range 0-100).

METHODS

Patients were recruited in 13 hospitals across 5 countries and were randomly assigned (1:1) to either TIVAD or BKP with electronic randomization as described in the protocol. A total of 152 patients with OVCFs were initially randomized. Eleven patients were excluded (six met exclusion criteria, one with evidence of tumor, and four patients had T score out of requested range). Anterior vertebral body height ratio, midline vertebral body height ratio, and Cobb angle were measured preoperatively and postoperatively by an independent imaging core lab. Adjacent and subsequent fractures and safety parameters were recorded throughout the study. Cement extravasation was evaluated on X-rays. All patients were followed at screening at 5 days, 1 month, 6 months, and 12 months postoperatively. This study was supported by Vexim SA. Seven authors received study-specific support less than $10,000 per year and seven authors received no study-specific support.

RESULTS

Among the 141 patients (78.7% female, mean age 73.3±9.5 years) who underwent surgery (TIVAD=68; BKP=73), 126 patients (89.4%) completed the 12-month follow-up period (TIVAD=61; BKP=65). The analysis of primary endpoint on the ITT population demonstrated non-inferiority of the TIVAD to BKP. The analysis of the additional composite endpoint demonstrated the superiority of TIVAD over BKP (p<0.0001) at 6 months (88.1% vs. 60.9%) and at 12 months (79.7% vs. 59.3%). Midline VB height restoration was more improved for TIVAD than for BKP at 6 months (1.14±2.61 mm vs. 0.31±2.22 mm); p=0.0246) and 12 months after surgery (1.31±2.58 mm vs. 0.10±2.34 mm; p=0.0035). No statistically significant differences were shown between procedures for improvement in functional capacity and quality of life. Pain relief was significantly more marked in the TIVAD group compared to the BKP group at 1 month (p=0.029) and at 6 months (p=0.021) after surgery. No patient required surgical reintervention or retreatment at the treated level. No symptomatic cement leakage was reported. Adverse events were similar for both groups (41.2% in the TIVAD group and 45.2% in the BKP group). The incidence of adjacent fractures was significantly lower after the TIVAD procedure than after BKP (12.9% vs. 27.3%; p=0.043).

CONCLUSIONS

Study results demonstrated non-inferiority of the TIVAD to the predicate BKP with an excellent risk/benefit profile for results up to 12 months.

Statistical issues and methods in designing and analyzing survival studies

BACKGROUND

Cancer studies that are designed for early detection and screening, or used for identifying prognostic factors, or assessing treatment efficacy and health outcome are frequently assessed with survival or time-to-event outcomes. These studies typically require specific methods of data analysis. Appropriate statistical methods in the context of study design and objectives are required for obtaining reliable results and valid inference. Unfortunately, variable methods for the same study objectives and dubious reporting have been noticed in the survival analysis of oncology research. Applied researchers often face difficulties in selecting appropriate statistical methods due to the complex nature of cancer studies.

RECENT FINDINGS

In this report, we describe briefly major statistical issues along with related challenges in planning, designing, and analyzing of survival studies. For applied researchers, we provided flow charts for selecting appropriate statistical methods. Various available statistical procedures in common statistical packages for applying survival analysis were classified according to different objectives of the study. In addition, an illustration of the statistical analysis of some common types of time-to-event outcomes was shown with STATA codes.

CONCLUSIONS

We anticipate that this review article assists oncology researchers in understanding important statistical concepts involved in survival analysis and appropriately select the statistical approaches for survival analysis studies. Overall, the review may help in improving designing, conducting, analyzing, and reporting of data in survival studies.

Missing Data in the Field of Otorhinolaryngology and Head & Neck Surgery: Need for Improvement

OBJECTIVE

Clinical studies are often facing missing data. Data can be missing for various reasons, for example, patients moved, certain measurements are only administered in high-risk groups, and patients are unable to attend clinic because of their health status. There are various ways to handle these missing data (e.g., complete cases analyses, mean substitution). Each of these techniques potentially influences both the analyses and the results of a study. The first aim of this structured review was to analyze how often researchers in the field of otorhinolaryngology/head & neck surgery report missing data. The second aim was to systematically describe how researchers handle missing data in their analyses. The third aim was to provide a solution on how to deal with missing data by means of the multiple imputation technique. With this review, we aim to contribute to a higher quality of reporting in otorhinolaryngology research.

DESIGN

Clinical studies among the 398 most recently published research articles in three major journals in the field of otorhinolaryngology/head & neck surgery were analyzed based on how researchers reported and handled missing data.

RESULTS

Of the 316 clinical studies, 85 studies reported some form of missing data. Of those 85, only a small number (12 studies, 3.8%) actively handled the missingness in their data. The majority of researchers exclude incomplete cases, which results in biased outcomes and a drop in statistical power.

CONCLUSIONS

Within otorhinolaryngology research, missing data are largely ignored and underreported, and consequently, handled inadequately. This has major impact on the results and conclusions drawn from this research. Based on the outcomes of this review, we provide solutions on how to deal with missing data. To illustrate, we clarify the use of multiple imputation techniques, which recently became widely available in standard statistical programs.

A systematic survey on reporting and methods for handling missing participant data for continuous outcomes in randomized controlled trials

OBJECTIVE

To assess analytic approaches randomized controlled trial (RCT) authors use to address missing participant data (MPD) for patient-important continuous outcomes.

STUDY DESIGN AND SETTING

We conducted a systematic survey of RCTs published in 2014 in the core clinical journals that reported at least one patient-important outcome analyzed as a continuous variable.

RESULTS

Among 200 studies, 187 (93.5%) trials explicitly reported whether MPD occurred. In the 163 (81.5%) trials that reported the occurrence of MPD, the median and interquartile ranges of the percentage of participants with MPD were 11.4% (2.5%-22.6%).Among the 147 trials in which authors made clear their analytical approach to MPD, the approaches chosen included available data only (109, 67%); mixed-effect models (10, 6.1%); multiple imputation (9, 4.5%); and last observation carried forward (9, 4.5). Of the 163 studies reporting MPD, 16 (9.8%) conducted sensitivity analyses examining the impact of the MPD and (18, 11.1%) discussed the risk of bias associated with MPD.

CONCLUSION

RCTs reporting continuous outcomes typically have over 10% of participant data missing. Most RCTs failed to use optimal analytic methods, and very few conducted sensitivity analyses addressing the possible impact of MPD or commented on how MPD might influence risk of bias.

Empty polyetheretherketone (PEEK) cages in anterior cervical diskectomy and fusion (ACDF) show slow radiographic fusion that reduces clinical improvement: results from the prospective multicenter "PIERCE-PEEK" study

Background

Anterior cervical diskectomy and fusion (ACDF) is a well-established surgical treatment for radiculopathy and myelopathy. Previous studies showed that empty PEEK cages have lower radiographic fusion rates, but the clinical relevance remains unclear. This paper’s aim is to provide high-quality evidence on the outcomes of ACDF with empty PEEK cages and on the relevance of radiographic fusion for clinical outcomes.

Methods

This large prospective multicenter clinical trial performed single-level ACDF with empty PEEK cages on patients with cervical radiculopathy or myelopathy. The main clinical outcomes were VAS (0–10) for pain and NDI (0–100) for functioning. Radiographic fusion was evaluated by two investigators for three different aspects.

Results

The median (range) improvement of the VAS pain score was: 3 (1–6) at 6 months, 3 (2–8) at 12 months, and 4 (2–8) at 18 months. The median (range) improvement of the NDI score was: 12 (2–34) at 6 months, 18 (4–46) at 12 months, and 22 (2–44) at 18 months. Complete radiographic fusion was reached by 126 patients (43%) at 6 months, 214 patients (73%) at 12 months, and 241 patients (83%) at 18 months. Radiographic fusion was a highly significant (p < 0.001) predictor of the improvement of VAS and NDI scores.

Conclusion

This study provides strong evidence that ACDF is effective treatment, but the overall rate of radiographic fusion with empty PEEK cages is slow and insufficient. Lack of complete radiographic fusion leads to less improvement of pain and disability. We recommend against using empty uncoated pure PEEK cages in ACDF.

Trial registration

ISRCTN42774128. Retrospectively registered 14 April 2009.

The (in)stability of 21st century orthopedic patient contact information and its implications on clinical research: A cross-sectional study

BACKGROUND

In clinical research, minimizing patients lost to follow-up is essential for data validity. Researchers can employ better methodology to prevent patient loss. We examined how orthopedic surgery patients' contact information changes over time to optimize data collection for long-term outcomes research.

METHODS

Patients presenting to orthopedic outpatient clinics completed questionnaires regarding methods of contact: home phone, cell phone, mailing address, and e-mail address. They reported currently available methods of contact, if they changed in the past 5 and 10 years, and when they changed. Differences in the rates of change among methods were assessed via Fisher's exact tests. Whether participants changed any of their contact information in the past 5 and 10 years was determined via multivariate modeling, controlling for demographic variables.

RESULTS

Among 152 patients, 51% changed at least one form of contact information within 5 years, and 66% changed at least one form within 10 years. The rate of change for each contact method was similar over 5 (15%-28%) and 10 years (26%-41%). One patient changed all four methods of contact within the past 5 years and seven within the past 10 years. Females and younger patients were more likely to change some type of contact information.

CONCLUSION

The type of contact information least likely to change over 5-10 years is influenced by demographic factors such as sex and age, with females and younger participants more likely to change some aspect of their contact information. Collecting all contact methods appears necessary to minimize patients lost to follow-up, especially as technological norms evolve.

The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey

BACKGROUND CONTEXT

Randomized controlled trials (RCTs) are the most trustworthy source for evaluating treatment effects, but RCTs of spine surgery interventions often produce discordant results. The Fragility Index is a novel metric to inform about the robustness of statistically significant results.

PURPOSE

The aim was to determine the robustness of statistically significant results from RCTs of spine surgery interventions.

STUDY DESIGN/SETTING

This was a systematic survey.

PATIENT SAMPLE

The sample included RCTs of spine surgery interventions.

OUTCOME MEASURES

The Fragility Index is the minimum number of patients in a trial whose status would have to change from a nonevent to an event to change a statistically significant result to a nonsignificant result. Events refer to the occurrence of any dichotomous outcome, such as successful fusion, incident fracture, adjacent segment degeneration, or achievement of a certain functional score. A small Fragility Index indicates that the statistical significance of a result hinges on only a few events, and a large Fragility Index increases one's confidence in the observed treatment effects.

METHODS

We systematically reviewed a database for evidence-based orthopedics and identified all the RCTs that reported at least one positive outcome (ie, p<.05). Two reviewers independently assessed eligibility and extracted data. We used the Fisher exact test to compute Fragility Index values and multivariable linear regression to evaluate potential associated factors.

RESULTS

We identified 40 eligible RCTs with a median sample size of 132 patients (interquartile range [IQR] 79-208) and a median total number of outcome events for the chosen outcome of 31 (IQR 13-63). The median Fragility Index was two (IQR 1-3), which means that adding two events to one of the trial's treatment arms eliminated its statistical significance. The Fragility Index was less than or equal to three events in 75% of the trials, and was less than or equal to the number of patients lost to follow-up in 65% of the trials. Fragility Index values correlated positively with total sample size (r=0.35; p<.05). When adjusted for losses to follow-up and risk of bias, increasing Fragility Index values were associated only with increasingly significant reported p values (p<.01).

CONCLUSIONS

Statistically significant results in spine surgery RCTs are frequently fragile. The addition of only a small number of outcome events can completely eliminate significance. Surgeons, researchers, and other evidence users should exercise caution when interpreting the findings from RCTs with low Fragility Index values and applying these results to patient care.

Methylprednisolone for the Treatment of Patients with Acute Spinal Cord Injuries: A Propensity Score-Matched Cohort Study from a Canadian Multi-Center Spinal Cord Injury Registry

In prior analyses of the effectiveness of methylprednisolone for the treatment of patients with acute traumatic spinal cord injuries (TSCIs), the prognostic importance of patients' neurological levels of injury and their baseline severity of impairment has not been considered. Our objective was to determine whether methylprednisolone improved motor recovery among participants in the Rick Hansen Spinal Cord Injury Registry (RHSCIR). We identified RHSCIR participants who received methylprednisolone according to the Second National Spinal Cord Injury Study (NASCIS-II) protocol and used propensity score matching to account for age, sex, time of neurological exam, varying neurological level of injury, and baseline severity of neurological impairment. We compared changes in total, upper extremity, and lower extremity motor scores using the Wilcoxon signed-rank test and performed sensitivity analyses using negative binomial regression. Forty-six patients received methylprednisolone and 1555 received no steroid treatment. There were no significant differences between matched participants for each of total (13.7 vs. 14.1, respectively; p=0.43), upper extremity (7.3 vs. 6.4; p=0.38), and lower extremity (6.5 vs. 7.7; p=0.40) motor recovery. This result was confirmed using a multivariate model and, as predicted, only cervical (C1-T1) rather than thoracolumbar (T2-L3) injury levels (p<0.01) and reduced baseline injury severity (American Spinal Injury Association [ASIA] Impairment Scale grades; p<0.01) were associated with greater motor score recovery. There was no in-hospital mortality in either group; however, the NASCIS-II methylprednisolone group had a significantly higher rate of total complications (61% vs. 36%; p=0.02) NASCIS-II methylprednisolone did not improve motor score recovery in RHSCIR patients with acute TSCIs in either the cervical or thoracic spine when the influence of anatomical level and severity of injury were included in the analysis. There was a significantly higher rate of total complications in the NASCIS-II methylprednisolone group. These findings support guideline recommendations against routine administration of methylprednisolone in acute TSCI.

OARSI Clinical Trials Recommendations: Key analytic considerations in design, analysis, and reporting of randomized controlled trials in osteoarthritis

To highlight methodologic challenges pertinent to design, analysis, and reporting of results of randomized clinical trials in OA and offer practical suggestions to overcome these challenges. The topics covered in this paper include subject selection, randomization, approaches to handling missing data, subgroup analysis, sample size, and issues related to changing design mid-way through the study. Special attention is given to standardizing the reporting of results and economic analyses. Key findings include the importance of blinding and concealment, the distinction between superiority and non-inferiority trials, the need to minimize missing data, and appropriate analysis and interpretation of subgroup effects. Investigators may use the findings and recommendations advanced in this paper to guide design and conduct of randomized controlled trials of interventions for osteoarthritis.

Challenges in outcome measurement: clinical research perspective

Comparative effectiveness research evaluates treatments as actually delivered in routine clinical practice, shifting research focus from efficacy and internal validity to effectiveness and external validity ("generalizability"). Such research requires accurate assessments of the numbers of patients treated and the completeness of their followup, their clinical outcomes, and the setting in which their care was delivered. Choosing measures and methods for clinical outcome research to produce meaningful information that may be used to improve patient care presents a number of challenges. WHERE ARE WE NOW?: Orthopaedic surgery research has many stakeholders, including patients, providers, payers, and policy makers. A major challenge in orthopaedic surgery outcome measurement and clinical research is providing all of these users with valid information for their respective decision making. At present, no plan exists for capturing data on such a broad scale and scope. WHERE DO WE NEED TO GO?: Practical challenges include identifying and obtaining resources for widespread data collection and merging multiple data sources. Challenges of study design include sampling to obtain representative data, timing of data collection in the episode of care, and minimizing missing data and study dropout. HOW DO WE GET THERE?: Resource limitations may be addressed by repurposing existing clinical resources and capitalizing on technologic advances to increase efficiencies. Increasing use of rigorous, well-designed observational research designs can provide information that may be unattainable in clinical trials. Such study designs should incorporate methods to minimize missing data, to sample multiple providers, facilities, and patients, and to include evaluation of potential confounding variables to minimize bias and allow generalization to broad populations.

Strategies for dealing with missing data in clinical trials: from design to analysis

Randomized clinical trials are the gold standard for evaluating interventions as randomized assignment equalizes known and unknown characteristics between intervention groups. However, when participants miss visits, the ability to conduct an intent-to-treat analysis and draw conclusions about a causal link is compromised. As guidance to those performing clinical trials, this review is a non-technical overview of the consequences of missing data and a prescription for its treatment beyond the typical analytic approaches to the entire research process. Examples of bias from incorrect analysis with missing data and discussion of the advantages/disadvantages of analytic methods are given. As no single analysis is definitive when missing data occurs, strategies for its prevention throughout the course of a trial are presented. We aim to convey an appreciation for how missing data influences results and an understanding of the need for careful consideration of missing data during the design, planning, conduct, and analytic stages.

The prevention and handling of the missing data

Even in a well-designed and controlled study, missing data occurs in almost all research. Missing data can reduce the statistical power of a study and can produce biased estimates, leading to invalid conclusions. This manuscript reviews the problems and types of missing data, along with the techniques for handling missing data. The mechanisms by which missing data occurs are illustrated, and the methods for handling the missing data are discussed. The paper concludes with recommendations for the handling of missing data.