Pooling of studies in meta-analysis of observational research leads to precise but spurious results

The effect of confounding data features on a deep learning algorithm to predict complete coronary occlusion in a retrospective observational setting

Aims

Deep learning (DL) has emerged in recent years as an effective technique in automated ECG analysis.

Methods and results

A retrospective, observational study was designed to assess the feasibility of detecting induced coronary artery occlusion in human subjects earlier than experienced cardiologists using a DL algorithm. A deep convolutional neural network was trained using data from the STAFF III database. The task was to classify ECG samples as showing acute coronary artery occlusion, or no occlusion. Occluded samples were recorded after 60 s of balloon occlusion of a single coronary artery. For the first iteration of the experiment, non-occluded samples were taken from ECGs recorded in a restroom prior to entering theatres. For the second iteration of the experiment, non-occluded samples were taken in the theatre prior to balloon inflation. Results were obtained using a cross-validation approach. In the first iteration of the experiment, the DL model achieved an F1 score of 0.814, which was higher than any of three reviewing cardiologists or STEMI criteria. In the second iteration of the experiment, the DL model achieved an F1 score of 0.533, which is akin to the performance of a random chance classifier.

Conclusion

The dataset was too small for the second model to achieve meaningful performance, despite the use of transfer learning. However, 'data leakage' during the first iteration of the experiment led to falsely high results. This study highlights the risk of DL models leveraging data leaks to produce spurious results.

Withholding Feeds and Transfusion-Associated Necrotizing Enterocolitis in Preterm Infants: A Systematic Review

Limited evidence exists to support the withholding of feeds during packed red blood cell (PRBC) transfusion to reduce the incidence of transfusion-associated necrotizing enterocolitis (TANEC) in preterm infants. The aim of the manuscript was to systematically review studies reporting the effect of implementing a policy of withholding feeds on the incidence of TANEC in preterm infants. The following databases were searched for relevant studies published between the databases' inception and December 2016: PubMed, Embase, the Cochrane Central Register of Controlled Trials, the Cumulative Index of Nursing and Allied Health Literature, and Pediatric Academic Societies Abstract Archive. Other relevant sources were also searched. There were no restrictions on study design. Studies reporting on the incidence of TANEC (stage ≥2 necrotizing enterocolitis within 48-72 h) after implementation of a policy of withholding feeds in the peritransfusion period in preterm infants were included. This meta-analysis used a random-effects model with assessment of quality of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. There were no randomized controlled trials (RCTs). Pooled results from 7 non-RCTs (n = 7492) showed that withholding feeds during PRBC transfusion significantly reduced the incidence of TANEC (RR: 0.47; 95% CI: 0.28, 0.80; P = 0.005; I2 = 11%). The overall quality of evidence was moderate on GRADE analysis. These findings suggest that withholding feeds during the peritransfusion period may reduce the risk of TANEC in preterm infants. Adequately powered RCTs are needed to confirm these findings.

Expanded newborn screening in New South Wales: missed cases

There have been few reports of cases missed by expanded newborn screening. Tandem mass spectrometry was introduced in New South Wales, Australia in 1998 to screen for selected disorders of amino acid, organic acid and fatty acid metabolism. Of 1,500,000 babies screened by 2012, 1:2700 were diagnosed with a target disorder. Fifteen affected babies were missed by testing, and presented clinically or in family studies. In three cases (cobalamin C defect, very-long-chain acyl-CoA dehydrogenase deficiency and glutaric aciduria type 1), this led to modification of analyte cut-off values or protocols during the first 3 years. Two patients with intermittent MSUD, two with β-ketothiolase deficiency, two with citrin deficiency, two siblings with arginosuccinic aciduria, two siblings with homocystinuria, and one with cobalamin C defect had analyte values and ratios below the action limits which could not have been detected without unacceptable false-positive rates. A laboratory interpretation error led to missing one case of cobalamin C defect. Reference ranges, regularly reviewed, were not altered. For citrin deficiency, while relevant metabolites are detectable by tandem mass spectrometry, our cut-off values do not specifically screen for that disorder. Most of the missed cases are doing well and with no acute presentations although eight of 15 are likely to have been somewhat adversely affected by a late diagnosis. Analyte ratio and cut-off value optimisations are important, but for some disorders occasional missed cases may have to be tolerated to maintain an acceptable specificity, and avoid harm from screening.

Newborn screening by tandem mass spectrometry for glutaric aciduria type 1: a cost-effectiveness analysis

BACKGROUND

Glutaric aciduria type I (GA-I) is a rare metabolic disorder caused by inherited deficiency of glutaryl-CoA dehydrogenase. Despite high prognostic relevance of early diagnosis and start of metabolic treatment as well as an additional cost saving potential later in life, only a limited number of countries recommend newborn screening for GA-I. So far only limited data is available enabling health care decision makers to evaluate whether investing into GA-I screening represents value for money. The aim of our study was therefore to assess the cost-effectiveness of newborn screening for GA-I by tandem mass spectrometry (MS/MS) compared to a scenario where GA-I is not included in the MS/MS screening panel.

METHODS

We assessed the cost-effectiveness of newborn screening for GA-I against the alternative of not including GA-I in MS/MS screening. A Markov model was developed simulating the clinical course of screened and unscreened newborns within different time horizons of 20 and 70 years. Monte Carlo simulation based probabilistic sensitivity analysis was used to determine the probability of GA-I screening representing a cost-effective therapeutic strategy.

RESULTS

Within a 20 year time horizon, GA-I screening averts approximately 3.7 DALYs (95% CI 2.9 - 4.5) and about one life year is gained (95% CI 0.7 - 1.4) per 100,000 neonates screened initially . Moreover, the screening programme saves a total of around 30,682 Euro (95% CI 14,343 to 49,176 Euro) per 100,000 screened neonates over a 20 year time horizon.

CONCLUSION

Within the limitations of the present study, extending pre-existing MS/MS newborn screening programmes by GA-I represents a highly cost-effective diagnostic strategy when assessed under conditions comparable to the German health care system.

[Treatment of loculated parapneumonic empyema. Video assisted thoracoscopy or fibrinolytics?]

OBJECTIVE

There is no consensus regarding the ideal treatment of loculated parapneumonic empyema (PPE). The aim of this study is to compare the effectiveness of drainage plus urokinase (DF) with video-assisted thoracoscopy (VATS).

MATERIAL AND METHODS

Retrospective review. Patients admitted with a PPE between January 2001 and July 2008. Loculated empyema was diagnosed by chest ultrasound and patients were treated with DF or VATS depending on the attending surgeon. Compared variables were: post-operative stay, total hospital stay, days of tube thoracostomy, post-operative fever and treatment failure.

RESULTS

One hundred and twenty one patients were admitted with the diagnosis of PPE. Seventeen patients were excluded from analysis because of simple parapneumonic effusions. Of the 104 patients included in the study, 47 were treated with urokinase and 57 with videothoracoscopy. No statistically significant differences (P>.05) were found between the median values in the DF and VATS groups for hospital stay (median 12 vs 12 days) or post-operative stay (median 9 vs 9 days). There were differences in duration of tube thoracostomy (median DF group 5 days, VATS, 4 days, P<.05) and in the post-operative fever (median 3 vs 2 days, p<0,05).Twenty two per cent of children needed a second procedure (14 patients of DF and 9 of VATS group, P=.09).

CONCLUSIONS

According to our experience, the results of DF and VATS for the treatment of loculated parapneumonic empyema are similar. Although there are no statistical differences, there seems to be a higher rate of failure, with the need of more procedures in the DF group. This difference does not affect the average total hospital stay.

Do nonphysical punishments reduce antisocial behavior more than spanking? a comparison using the strongest previous causal evidence against spanking

BACKGROUND

The strongest causal evidence that customary spanking increases antisocial behavior is based on prospective studies that control statistically for initial antisocial differences. None of those studies have investigated alternative disciplinary tactics that parents could use instead of spanking, however. Further, the small effects in those studies could be artifactual due to residual confounding, reflecting child effects on the frequency of all disciplinary tactics. This study re-analyzes the strongest causal evidence against customary spanking and uses these same methods to determine whether alternative disciplinary tactics are more effective in reducing antisocial behavior.

METHODS

This study re-analyzed a study by Straus et al.1 on spanking and antisocial behavior using a sample of 785 children who were 6 to 9 years old in the 1988 cohort of the American National Longitudinal Survey of Youth. The comprehensiveness and reliability of the covariate measure of initial antisocial behavior were varied to test for residual confounding. All analyses were repeated for grounding, privilege removal, and sending children to their room, and for psychotherapy. To account for covarying use of disciplinary tactics, the analyses were redone first for the 73% who had reported using at least one discipline tactic and second by controlling for usage of other disciplinary tactics and psychotherapy.

RESULTS

The apparently adverse effect of spanking on antisocial behavior was replicated using the original trichotomous covariate for initial antisocial behavior. A similar pattern of adverse effects was shown for grounding and psychotherapy and partially for the other two disciplinary tactics. All of these effects became non-significant after controlling for latent comprehensive measures of externalizing behavior problems.

CONCLUSIONS

These results are consistent with residual confounding, a statistical artifact that makes all corrective actions by parents and psychologists appear to increase children's antisocial behavior due to child effects on parents. Improved research methods are needed to discriminate between effective vs. counterproductive implementations of disciplinary tactics. How and when disciplinary tactics are used may be more important than which type of tactic is used.

Cost-effectiveness of competing strategies for the treatment of pediatric empyema

BACKGROUND

The optimal management of pediatric empyema is controversial. The purpose of this decision analysis was to assess the relative merits in terms of costs and clinical outcomes associated with competing treatment strategies.

METHODS

A cost-effectiveness analysis was conducted using a Bayesian tree approach. Probability and outcome estimates were derived from the published literature, with preference given to data derived from randomized trials. Costing was based on published estimates from Great Ormond Street Hospital (London, United Kingdom), supplemented by American and Canadian data. Five strategies were evaluated: (1) nonoperative; (2) chest tube insertion; (3) repeated thoracentesis; (4) chest tube insertion with instillation of fibrinolytics; or (5) video-assisted thorascopic surgery. The model was used to project overall costs, survival in life-years, and incremental cost-effectiveness ratios for competing strategies.

RESULTS

In the base-case analysis, chest tube with instillation of fibrinolytics was the least expensive therapy, at $7787 per episode. This strategy was projected to cost less but provide equivalent health benefit when compared with all of the competing strategies except repeated thoracentesis, which had an incremental cost-effectiveness ratio of approximately $6,422,699 per life-year gained relative to chest tube with instillation of fibrinolytics. In univariable and multivariable sensitivity analyses, thorascopic surgery was preferred only when the length of stay associated with chest tube with instillation of fibrinolytics exceeded 10.3 days or when the probability of dying as a result of this strategy exceeded 0.2%, assuming a threshold willingness to pay of $75,000 per life-year gained. Chest tube with instillation of fibrinolytics was preferred in >58% of Monte Carlo simulations.

CONCLUSIONS

On the basis of the best available data, chest tube with instillation of fibrinolytics is the most cost-effective strategy for treating pediatric empyema. Video-assisted thorascopic surgery would be preferred to chest tube with instillation of fibrinolytics if the differential in length of stay between these 2 strategies were proven to be greater than that suggested by currently available data.

Systemic fungal infection is associated with the development of retinopathy of prematurity in very low birth weight infants: a meta-review

OBJECTIVE

To study the association between systemic fungal infection (SFI) and the development of retinopathy of prematurity (ROP) and severe ROP in very low birth weight (VLBW) infants by systematic review and meta-analysis.

STUDY DESIGN

A meta-review was performed using a fixed effects model. The exposure and outcomes studied were SFI and all ROP/severe ROP, respectively in VLBW infants. Results and effect sizes analyzed with Review Manager 4.2 software are expressed as relative risk (RR), odds ratio (OR), risk difference (RD) and number needed to harm (NNH) with 95% confidence intervals.

RESULT

Data for severe ROP were available from eight studies and on all ROP from seven of those eight studies. Estimated gestational age ranged from 24.7+/-1.6 to 28.6+/-4 weeks and birth weight from 673 (median) (range 426 to 995) to 1108+/-266 g (mean+/-s.d.). A total of 261 of 303 babies with SFI had all ROP vs 1081 of 1648 babies without SFI (OR 3.4(*), 2.34-4.95) and 118 of 330 babies with SFI had severe ROP vs 235 of 1951 babies without SFI (OR 4.06(*), 3.05-5.42). The NNH was 5.56(*) (4.54-7.14) for all ROP and 4.54(*) (3.70 to 5.88) for severe ROP ((*) P<0.00001).

CONCLUSION

SFIs are associated with the development of all degrees of ROP and severe ROP in VLBW infants.