Scientific reporting is suboptimal for aspects that characterize genetic risk prediction studies: a review of published articles based on the Genetic RIsk Prediction Studies statement

Exploration, Inference, and Prediction in Neuroscience and Biomedicine

Recent decades have seen dramatic progress in brain research. These advances were often buttressed by probing single variables to make circumscribed discoveries, typically through null hypothesis significance testing. New ways for generating massive data fueled tension between the traditional methodology that is used to infer statistically relevant effects in carefully chosen variables, and pattern-learning algorithms that are used to identify predictive signatures by searching through abundant information. In this article we detail the antagonistic philosophies behind two quantitative approaches: certifying robust effects in understandable variables, and evaluating how accurately a built model can forecast future outcomes. We discourage choosing analytical tools via categories such as 'statistics' or 'machine learning'. Instead, to establish reproducible knowledge about the brain, we advocate prioritizing tools in view of the core motivation of each quantitative analysis: aiming towards mechanistic insight or optimizing predictive accuracy.

A Survey of the Prevalence and Impact of Reporting Guideline Endorsement in Pathology Journals

OBJECTIVES

To determine the prevalence of reporting guideline endorsement in pathology journals and to estimate the impact of guideline endorsement.

METHODS

We compared the quality of reporting in two sets of studies: (1) studies published in journals that explicitly mentioned a guideline vs studies published in journals that did not and (2) studies that cited a guideline vs studies that did not. The quality of reporting in prognostic biomarker studies was assessed using the REporting recommendations for tumor MARKer prognostic studies (REMARK) guideline.

RESULTS

We found that six (10%) of the 59 leading pathology journals explicitly mention reporting guidelines in the instructions to authors. Only one journal required authors to submit a checklist. There was significant variation in the rate at which various REMARK items were reported (P < .001). Journal endorsement was associated with more complete reporting (P = .04). Studies that cited REMARK had greater adherence to the REMARK reporting guidelines than studies that did not (P = .02).

CONCLUSIONS

The prevalence of guideline endorsement is relatively low in pathology journals, but guideline endorsement may improve the quality of reporting.

Optimal threshold estimator of a prognostic marker by maximizing a time-dependent expected utility function for a patient-centered stratified medicine

Defining thresholds of prognostic markers is essential for stratified medicine. Such thresholds are mostly estimated from purely statistical measures regardless of patient preferences potentially leading to unacceptable medical decisions. Quality-Adjusted Life-Years are a widely used preferences-based measure of health outcomes. We develop a time-dependent Quality-Adjusted Life-Years-based expected utility function for censored data that should be maximized to estimate an optimal threshold. We performed a simulation study to compare estimated thresholds when using the proposed expected utility approach and purely statistical estimators. Two applications illustrate the usefulness of the proposed methodology which was implemented in the R package ROCt ( www.divat.fr ). First, by reanalysing data of a randomized clinical trial comparing the efficacy of prednisone vs. placebo in patients with chronic liver cirrhosis, we demonstrate the utility of treating patients with a prothrombin level higher than 89%. Second, we reanalyze the data of an observational cohort of kidney transplant recipients: we conclude to the uselessness of the Kidney Transplant Failure Score to adapt the frequency of clinical visits. Applying such a patient-centered methodology may improve future transfer of novel prognostic scoring systems or markers in clinical practice.

Non-HLA genomics: does it have a role in predicting haematopoietic stem cell transplantation outcome?

Haematopoietic stem cell transplantation (HSCT) remains the only cure for many haematological neoplasms; however, the mortality rate remains high, at around 30-80%. Complications after HSCT include relapse, graft-versus-host disease, graft rejection and infection. High-resolution HLA matching has improved survival in HSCT over recent years; however, GVHD still remains a serious complication. Single nucleotide polymorphisms (SNPS) within genes that are involved with an individual's capability to mount an immune response to infectious pathogens, residual leukaemia, alloantigens or genes involved in drug metabolism have been studied for their association with HSCT outcome. Indeed, over the last 15 years, several groups, including ourselves, have demonstrated that non-HLA gene polymorphisms can be predictive of HSCT outcome. Can genetic characteristics of the patient and donor be used in the future to tailor HSCT protocols and determine GVHD prophylaxis? This review summarizes some of the recent SNP association studies in HSCT and highlights some of the disparities therein, discussing the integral problems of performing genetic association studies on diseases with complex outcomes using heterogeneous cohorts. The review will comment on recent genomewide association studies (GWAS) and discuss their relevance in this field, and it will also comment on recent meta-analysis combining GWAS studies with other studies such as gene expression micro array data in the field of autoimmune disease and solid organ transplantation. It will mention possible novel candidate gene polymorphisms, for example SNPS in microRNAs. In addition, it will discuss some of the inherent problems associated with gene association studies including the GRIPs (genetic risk prediction studies) recommendations. In summary, this review will assess the usefulness of non-HLA genomic studies in HSCT with regard to predicting outcome and modifying therapy.

P < 5 × 10(-8) has emerged as a standard of statistical significance for genome-wide association studies

OBJECTIVES

In genome-wide association studies (GWASs), the practice regarding the choice of thresholds of significance and of thresholds used to include single nucleotide polymorphisms (SNPs) in a further validation stage is not well known. Here, we performed a systematic analysis of all GWASs published in two recent but nonconsecutive periods to assess whether any consensus was emerging regarding the choice of these thresholds.

STUDY DESIGN AND SETTING

We identified 167 GWASs published during the first semester of 2011 and 105 published in the third trimester of 2012 and retrieved the genome-wide threshold of significance and the thresholds to include SNPs in a replication stage if applicable.

RESULTS

The proportion of studies using 5 × 10(-8) as a genome-wide significance threshold increased between 2011 and 2012 (40% vs. 64%, P < 0.001), whereas the proportion of articles that gave a justification for this threshold decreased. The distribution of thresholds used to include SNPs in the replication stage remained stable over time (median 10(-5)), and the variance remained large [interquartile range (10(-6), 10(-4))].

CONCLUSION

Although 5 × 10(-8) has become the de facto standard genome-wide threshold, practice regarding the choice of inclusion threshold for the replication step remains heterogeneous and did not ensure the best trade-off between power and type 1 error.