Making the cut: improved ranking and selection for large-scale inference

Using Billing Codes to Create a Pediatric Functional Status e-Score for Children Receiving Inpatient Rehabilitation

OBJECTIVE

Provide proof-of-concept for development of a Pediatric Functional Status eScore (PFSeS). Demonstrate that expert clinicians rank billing codes as relevant to patient functional status and identify the domains that codes inform in a way that reliably matches analytical modeling.

DESIGN

Retrospective chart review, modified Delphi, and nominal group techniques.

SETTING

Large, urban, quaternary care children's hospital in the Midwestern United States.

PARTICIPANTS

Data from 1955 unique patients and 2029 hospital admissions (2000-2020); 12 expert consultants representing the continuum of rehabilitation care reviewed 2893 codes (procedural, diagnostic, pharmaceutical, durable medical equipment).

MAIN OUTCOME MEASURES

Consensus voting to determine whether codes were associated with functional status at discharge and, if so, what domains they informed (self-care, mobility, cognition/ communication).

RESULTS

The top 250 and 500 codes identified by statistical modeling were mostly composed of codes selected by the consultant panel (78%-80% of the top 250 and 71%-78% of the top 500). The results provide evidence that clinical experts' selection of functionally meaningful codes corresponds with codes selected by statistical modeling as most strongly associated with WeeFIM domain scores. The top 5 codes most strongly related to functional independence ratings from a domain-specific assessment indicate clinically sensible relationships, further supporting the use of billing data in modeling to create a PFSeS.

CONCLUSIONS

Development of a PFSeS that is predicated on billing data would improve researchers' ability to assess the functional status of children who receive inpatient rehabilitation care for a neurologic injury or illness. An expert clinician panel, representing the spectrum of medical and rehabilitative care, indicated that proposed statistical modeling identifies relevant codes mapped to 3 important domains: self-care, mobility, and cognition/communication.

Novel autoantibodies help diagnose anti-SSA antibody negative Sjögren's disease and predict abnormal labial salivary gland pathology

Objectives

Sj□gren's disease (SjD) diagnosis requires either positive anti-SSA antibodies or a labial salivary gland biopsy with a positive focus score (FS). One-third of SjD patients lack anti-SSA antibodies (SSA-), requiring a positive FS for diagnosis. Our objective was to identify novel autoantibodies to diagnose 'seronegative' SjD.

Methods

IgG binding to a high density whole human peptidome array was quantified using sera from SSA- SjD cases and matched non-autoimmune controls. We identified the highest bound peptides using empirical Bayesian statistical filters, which we confirmed in an independent cohort comprising SSA- SjD (n=76), sicca controls without autoimmunity (n=75), and autoimmune controls (SjD features but not meeting SjD criteria; n=41). In this external validation, we used non-parametric methods for peptide abundance and controlled false discovery rate in group comparisons. For predictive modeling, we used logistic regression, model selection methods, and cross-validation to identify clinical and peptide variables that predict SSA- SjD and FS positivity.

Results

IgG against a peptide from D-aminoacyl-tRNA deacylase (DTD2) was bound more in SSA- SjD than sicca controls (p=.004) and more than combined controls (sicca and autoimmune controls combined; p=0.003). IgG against peptides from retroelement silencing factor-1 (RESF1) and DTD2, were bound more in FS-positive than FS-negative participants (p=.010; p=0.012). A predictive model incorporating clinical variables showed good discrimination between SjD versus control (AUC 74%) and between FS-positive versus FS-negative (AUC 72%).

Conclusion

We present novel autoantibodies in SSA- SjD that have good predictive value for SSA- SjD and FS-positivity.

KEY MESSAGES

What is already known on this topic - Seronegative (anti-SSA antibody negative [SSA-]) Sjögren's disease (SjD) requires a labial salivary gland biopsy for diagnosis, which is challenging to obtain and interpret. What this study adds - We identified novel autoantibodies in SSA- SjD that, when combined with readily available clinical variables, provide good predictive ability to discriminate 1) SSA- SjD from control participants and 2) abnormal salivary gland biopsies from normal salivary gland biopsies. How this study might affect research, practice or policy - This study provides novel diagnostic antibodies addressing the critical need for improvement of SSA- SjD diagnostic tools.

A Bayesian hierarchical model for signal extraction from protein microarrays

Protein microarrays are a promising technology that measure protein levels in serum or plasma samples. Due to their high technical variability and high variation in protein levels across serum samples in any population, directly answering biological questions of interest using protein microarray measurements is challenging. Analyzing preprocessed data and within-sample ranks of protein levels can mitigate the impact of between-sample variation. As for any analysis, ranks are sensitive to preprocessing, but loss function based ranks that accommodate major structural relations and components of uncertainty are very effective. Bayesian modeling with full posterior distributions for quantities of interest produce the most effective ranks. Such Bayesian models have been developed for other assays, for example, DNA microarrays, but modeling assumptions for these assays are not appropriate for protein microarrays. Consequently, we develop and evaluate a Bayesian model to extract the full posterior distribution of normalized protein levels and associated ranks for protein microarrays, and show that it fits well to data from two studies that use protein microarrays produced by different manufacturing processes. We validate the model via simulation and demonstrate the downstream impact of using estimates from this model to obtain optimal ranks.

An empirical Bayesian ranking method, with applications to high throughput biology

MOTIVATION

In bioinformatics, genome-wide experiments look for important biological differences between two groups at a large number of locations in the genome. Often, the final analysis focuses on a P-value-based ranking of locations which might then be investigated further in follow-up experiments. However, this strategy may result in small effect sizes, with low P-values, being ranked more favorably than larger more scientifically important effects. Bayesian ranking techniques may offer a solution to this problem provided a good prior distribution for the collective distribution of effect sizes is available.

RESULTS

We develop an Empirical Bayes ranking algorithm, using the marginal distribution of the data over all locations to estimate an appropriate prior. In simulations and analysis using real datasets, we demonstrate favorable performance compared to ordering P-values and a number of other competing ranking methods. The algorithm is computationally efficient and can be used to rank the entirety of genomic locations or to rank a subset of locations, pre-selected via traditional FWER/FDR methods in a 2-stage analysis.

AVAILABILITY AND IMPLEMENTATION

An R-package, EBrank, implementing the ranking algorithm is available on CRAN.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

ncdDetect2: improved models of the site-specific mutation rate in cancer and driver detection with robust significance evaluation

Motivation

Understanding the mutational processes that act during cancer development is a key topic of cancer biology. Nevertheless, much remains to be learned, as a complex interplay of processes with dependencies on a range of genomic features creates highly heterogeneous cancer genomes. Accurate driver detection relies on unbiased models of the mutation rate that also capture rate variation from uncharacterized sources.

Results

Here, we analyse patterns of observed-to-expected mutation counts across 505 whole cancer genomes, and find that genomic features missing from our mutation-rate model likely operate on a megabase length scale. We extend our site-specific model of the mutation rate to include the additional variance from these sources, which leads to robust significance evaluation of candidate cancer drivers. We thus present ncdDetect v.2, with greatly improved cancer driver detection specificity. Finally, we show that ranking candidates by their posterior mean value of their effect sizes offers an equivalent and more computationally efficient alternative to ranking by their P-values.

Availability and implementation

ncdDetect v.2 is implemented as an R-package and is freely available at http://github.com/TobiasMadsen/ncdDetect2

Supplementary information

Supplementary data are available at Bioinformatics online.

Detecting strong signals in gene perturbation experiments: An adaptive approach with power guarantee and FDR control

The perturbation of a transcription factor should affect the expression levels of its direct targets. However, not all genes showing changes in expression are direct targets. To increase the chance of detecting direct targets, we propose a modified two-group model where the null group corresponds to genes which are not direct targets, but can have small non-zero effects. We model the behavior of genes from the null set by a Gaussian distribution with unknown variance τ ². To estimate τ ², we focus on a simple estimation approach, the iterated empirical Bayes estimation. We conduct a detailed analysis of the properties of the iterated EB estimate and provide theoretical guarantee of its good performance under mild conditions. We provide simulations comparing the new modeling approach with existing methods, and the new approach shows more stable and better performance under different situations. We also apply it to a real data set from gene knock-down experiments and obtained better results compared with the original two-group model testing for non-zero effects.

Optimal Bayesian point estimates and credible intervals for ranking with application to county health indices

It is fairly common to rank different geographic units, e.g. counties in the USA, based on health indices. In a typical application, point estimates of the health indices are obtained for each county, and the indices are then simply ranked as if they were known constants. Several authors have considered optimal rank estimators under squared error loss on the rank scale as a default method for general purpose ranking, e.g. situations where ranking units across the full spectrum of performance (low, medium, high) is important. While computationally convenient, squared error loss on the rank scale may not represent the true inferential goals of rank consumers. We construct alternative loss functions based on three components: (1) the inferential goal (rank position or pairwise comparisons), (2) the scale (original, log-transformed or rank) and (3) the (positional or pairwise) loss function (0/1, squared error or absolute error). We can obtain optimal ranks for loss functions based on rank positions and nearly optimal ranks for loss functions based on pairwise comparisons paired with highest posterior density (HPD) credible intervals. We compare inferences produced by the various ranking methods, both optimal and heuristic, using low birth weight data for counties in the Midwestern United States, from 2006 to 2012.