Multiple-instance ensemble for construction of deep heterogeneous committees for high-dimensional low-sample-size data

Deep ensemble learning, where we combine knowledge learned from multiple individual neural networks, has been widely adopted to improve the performance of neural networks in deep learning. This field can be encompassed by committee learning, which includes the construction of neural network cascades. This study focuses on the high-dimensional low-sample-size (HDLS) domain and introduces multiple instance ensemble (MIE) as a novel stacking method for ensembles and cascades. In this study, our proposed approach reformulates the ensemble learning process as a multiple-instance learning problem. We utilise the multiple-instance learning solution of pooling operations to associate feature representations of base neural networks into joint representations as a method of stacking. This study explores various attention mechanisms and proposes two novel committee learning strategies with MIE. In addition, we utilise the capability of MIE to generate pseudo-base neural networks to provide a proof-of-concept for a "growing" neural network cascade that is unbounded by the number of base neural networks. We have shown that our approach provides (1) a class of alternative ensemble methods that performs comparably with various stacking ensemble methods and (2) a novel method for the generation of high-performing "growing" cascades. The approach has also been verified across multiple HDLS datasets, achieving high performance for binary classification tasks in the low-sample size regime.

Computational portraits of the tumoral microenvironment in human breast cancer

Breast cancer is the most diagnosed cancer in humans. In recent years, myxoid and proportionated stroma have been described as clinically significant in many cancer subtypes. Here computational portraits of tumor-associated stromata were created from a machine learning (ML) classifier using QuPath to evaluate proportionated stromal area (PSA), myxoid stromal ratio (MSR), and immune stroma proportion (ISP) from whole slide images (WSI). The ML classifier was validated in independent training (n = 40) and validation (n = 109) cohorts finding MSR, PSA, and ISP to be associated with tumor stage, lymph node status, Nottingham grade, stromal differentiation (SD), tumor size, estrogen receptor (ER), progesterone receptor (PR), and receptor tyrosine-protein kinase erbB-2 (HER-2). Overall, MSR correlated better with the clinicopathologic profile than PSA and ISP. High MSR was found to be associated with high tumor stage, low ISP, and high Nottingham histologic score. As a computational biomarker, high MSR was more likely to be associated with luminal B like, Her-2 enriched, and triple-negative biomarker status when compared to luminal A like. The supervised ML superpixel approach demonstrated here can be performed by a trained pathologist to provide a faster and more uniformed approach to the analysis to the tumoral microenvironment (TME). The TME may be relevant for clinical decision-making, determining chemotherapeutic efficacy, and guiding a more overall precision-based breast cancer care.

An empirical investigation of bias and variance in time series forecasting: modeling considerations and error evaluation

Bias and variance play an important role in understanding the fundamental issue of learning and generalization in neural network modeling. Several studies on bias and variance effects have been published in classification and regression related research of neural networks. However, little research has been done in this area for time-series modeling and forecasting. We consider modeling issues related to understanding error components given the common practices associated with neural-network time-series forecasting. We point out the key difference between classification and time-series problems in consideration of the bias-plus-variance decomposition. A Monte Carlo study on the role of bias and variance in neural networks time-series forecasting is conducted. We find that both input lag structure and hidden nodes are important in contributing to the overall forecasting performance. The results also suggest that overspecification of input nodes in neural network modeling does not impact the model bias, but has significant effect on the model variance. Methods such as neural ensembles that focus on reducing the model variance, therefore, can be valuable and effective in time-series forecasting modeling.

Novel machine learning methods for ERP analysis: a validation from research on infants at risk for autism

Machine learning and other computer intensive pattern recognition methods are successfully applied to a variety of fields that deal with high-dimensional data and often small sample sizes such as genetic microarray, functional magnetic resonance imaging (fMRI) and, more recently, electroencephalogram (EEG) data. The aim of this article is to discuss the use of machine learning and discrimination methods and their possible application to the analysis of infant event-related potential (ERP) data. The usefulness of two methods, regularized discriminant function analyses and support vector machines, will be demonstrated by reanalyzing an ERP dataset from infants ( Elsabbagh et al., 2009 ). Using cross-validation, both methods successfully discriminated above chance between groups of infants at high and low risk of a later diagnosis of autism. The suitability of machine learning methods for the use of single trial or averaged ERP data is discussed.

Estimation of particulate mass and manganese exposure levels among welders

BACKGROUND

Welders are frequently exposed to Manganese (Mn), which may increase the risk of neurological impairment. Historical exposure estimates for welding-exposed workers are needed for epidemiological studies evaluating the relationship between welding and neurological or other health outcomes. The objective of this study was to develop and validate a multivariate model to estimate quantitative levels of welding fume exposures based on welding particulate mass and Mn concentrations reported in the published literature.

METHODS

Articles that described welding particulate and Mn exposures during field welding activities were identified through a comprehensive literature search. Summary measures of exposure and related determinants such as year of sampling, welding process performed, type of ventilation used, degree of enclosure, base metal, and location of sampling filter were extracted from each article. The natural log of the reported arithmetic mean exposure level was used as the dependent variable in model building, while the independent variables included the exposure determinants. Cross-validation was performed to aid in model selection and to evaluate the generalizability of the models.

RESULTS

A total of 33 particulate and 27 Mn means were included in the regression analysis. The final model explained 76% of the variability in the mean exposures and included welding process and degree of enclosure as predictors. There was very little change in the explained variability and root mean squared error between the final model and its cross-validation model indicating the final model is robust given the available data.

CONCLUSIONS

This model may be improved with more detailed exposure determinants; however, the relatively large amount of variance explained by the final model along with the positive generalizability results of the cross-validation increases the confidence that the estimates derived from this model can be used for estimating welder exposures in absence of individual measurement data.

Quality prevails over identity in the sexually selected vocalisations of an ageing mammal

BACKGROUND

Male sexually selected vocalisations generally contain both individuality and quality cues that are crucial in intra- as well as inter-sexual communication. As individuality is a fixed feature whereas male phenotypic quality changes with age, individuality and quality cues may be subjected to different selection pressures over time. Individuality (for example, morphology of the vocal apparatus) and quality (for example, body size and dominance status) can both affect the vocal production mechanism, inducing the same components of vocalisations to convey both kinds of information. In this case, do quality-related changes to the acoustic structure of calls induce a modification of vocal cues to identity from year to year? We investigated this question in fallow deer (Dama dama), in which some acoustic parameters of vocalisations (groans) code for both individuality and quality.

RESULTS

We carried out a longitudinal analysis of groan individuality, examining the effects of age and dominance rank on the acoustic structure of groans of the same males recorded during consecutive years. We found both age- and rank-related changes to groans; the minimum values of the highest formant frequencies and the fundamental frequency increased with the age of males and they decreased when males became more dominant. Both age- and rank-related acoustic parameters contributed to individuality. Male quality changed with age, inducing a change in quality-related parameters and thus, a modification of vocal cues to male individuality between years.

CONCLUSIONS

The encoding of individuality and quality information in the same components of vocalisations induces a tradeoff between these two kinds of signals over time. Fallow deer vocalisations are honest signals of quality that are not fixed over time but are modified dynamically according to male quality. As they are more reliable cues to quality than to individuality, they may not be used by conspecifics to recognize a given male from one year to another, but potentially used by both sexes to assess male quality during each breeding season.

Boosting the prediction and understanding of DNA-binding domains from sequence

DNA-binding proteins perform vital functions related to transcription, repair and replication. We have developed a new sequence-based machine learning protocol to identify DNA-binding proteins. We compare our method with an extensive benchmark of previously published structure-based machine learning methods as well as a standard sequence alignment technique, BLAST. Furthermore, we elucidate important feature interactions found in a learned model and analyze how specific rules capture general mechanisms that extend across DNA-binding motifs. This analysis is carried out using the malibu machine learning workbench available at http://proteomics.bioengr.uic.edu/malibu and the corresponding data sets and features are available at http://proteomics.bioengr.uic.edu/dna.

QSAR modeling of mono- and bis-quaternary ammonium salts that act as antagonists at neuronal nicotinic acetylcholine receptors mediating dopamine release

Back-propagation artificial neural networks (ANNs) were trained on a dataset of 42 molecules with quantitative IC50 values to model structure-activity relationships of mono- and bis-quaternary ammonium salts as antagonists at neuronal nicotinic acetylcholine receptors (nAChR) mediating nicotine-evoked dopamine release. The ANN QSAR models produced a reasonable level of correlation between experimental and calculated log(1/IC50) (r2=0.76, r(cv)2=0.64). An external test for the models was performed on a dataset of 18 molecules with IC50 values >1 microM. Fourteen of these were correctly classified. Classification ability of various models, including self-organizing maps (SOM), for all 60 molecules was also evaluated. A detailed analysis of the modeling results revealed the following relative contributions of the used descriptors to the trained ANN QSAR model: approximately 44.0% from the length of the N-alkyl chain attached to the quaternary ammonium head group, approximately 20.0% from Moriguchi octanol-water partition coefficient of the molecule, approximately 13.0% from molecular surface area, approximately 12.6% from the first component shape directional WHIM index/unweighted, approximately 7.8% from Ghose-Crippen molar refractivity, and 2.6% from the lowest unoccupied molecular orbital energy. The ANN QSAR models were also evaluated using a set of 13 newly synthesized compounds (11 biologically active antagonists and two biologically inactive compounds) whose structures had not been previously utilized in the training set. Twelve among 13 compounds were predicted to be active which further supports the robustness of the trained models. Other insights from modeling include a structural modification in the bis-quinolinium series that involved replacing the 5 and/or 8 as well as the 5' and/or 8' carbon atoms with nitrogen atoms, predicting inactive compounds. Such data can be effectively used to reduce synthetic and in vitro screening activities by eliminating compounds of predicted low activity from the pool of candidate molecules for synthesis. The application of the ANN QSAR model has led to the successful discovery of six new compounds in this study with experimental IC50 values of less than 0.1 microM at nAChR subtypes responsible for mediating nicotine-evoked dopamine release, demonstrating that the ANN QSAR model is a valuable aid to drug discovery.

On different facets of regularization theory

This review provides a comprehensive understanding of regularization theory from different perspectives, emphasizing smoothness and simplicity principles. Using the tools of operator theory and Fourier analysis, it is shown that the solution of the classical Tikhonov regularization problem can be derived from the regularized functional defined by a linear differential (integral) operator in the spatial (Fourier) domain. State-of-the-art research relevant to the regularization theory is reviewed, covering Occam's razor, minimum length description, Bayesian theory, pruning algorithms, informational (entropy) theory, statistical learning theory, and equivalent regularization. The universal principle of regularization in terms of Kolmogorov complexity is discussed. Finally, some prospective studies on regularization theory and beyond are suggested.

Bias/Variance Decompositions for Likelihood-Based Estimators

The bias/variance decomposition of mean-squared error is well understood and relatively straightforward. In this note, a similar simple decomposition is derived, valid for any kind of error measure that, when using the appropriate probability model, can be derived from a Kullback-Leibler divergence or log-likelihood.