Multichannel Fusion Models for the Parametric Classification of Differential Brain Activity

Power transformer fault diagnosis method based on multi source signal fusion and fast spectral correlation

Addressing the issues that signal measured by a single sensor can not provide a complete description of power transformer fault states and the problems that selection of signal features relies on manual experience, a method based on multi source signal fusion and Fast Spectral Correlation is produced for power transformer fault diagnosis. At first, the vibration signals from different locations on the surface of the transformer case are collected by a sensor array synchronously, and Correlation Function Weighting is proposed to fuse multi-source signals from multiple sensors in order to obtain the fused signal; then, the fused signals are subjected to Fast Spectral Correlation belonging to cyclic smooth theory in order to construct a sample set of images; finally, the Fast Spectral Correlation image samples are fed into MobileNetV3 model for training of transfer learning to obtain the fine-tuned neural network model, which completes power transformer fault diagnosis. Experimental results showed that the overall recognition accuracy of the method proposed reached 98.75%, which was 10.52% higher than the diagnosis of single sensor signal, and 10.86% higher than the diagnosis of other classical images, providing a new tool for transformer fault diagnosis based on vibration signals.

Multidomain Convolution Neural Network Models for Improved Event-Related Potential Classification

Two convolution neural network (CNN) models are introduced to accurately classify event-related potentials (ERPs) by fusing frequency, time, and spatial domain information acquired from the continuous wavelet transform (CWT) of the ERPs recorded from multiple spatially distributed channels. The multidomain models fuse the multichannel Z-scalograms and the V-scalograms, which are generated from the standard CWT scalogram by zeroing-out and by discarding the inaccurate artifact coefficients that are outside the cone of influence (COI), respectively. In the first multidomain model, the input to the CNN is generated by fusing the Z-scalograms of the multichannel ERPs into a frequency-time-spatial cuboid. The input to the CNN in the second multidomain model is formed by fusing the frequency-time vectors of the V-scalograms of the multichannel ERPs into a frequency-time-spatial matrix. Experiments are designed to demonstrate (a) customized classification of ERPs, where the multidomain models are trained and tested with the ERPs of individual subjects for brain-computer interface (BCI)-type applications, and (b) group-based ERP classification, where the models are trained on the ERPs from a group of subjects and tested on single subjects not included in the training set for applications such as brain disorder classification. Results show that both multidomain models yield high classification accuracies for single trials and small-average ERPs with a small subset of top-ranked channels, and the multidomain fusion models consistently outperform the best unichannel classifiers.

Exploiting the Cone of Influence for Improving the Performance of Wavelet Transform-Based Models for ERP/EEG Classification

Features extracted from the wavelet transform coefficient matrix are widely used in the design of machine learning models to classify event-related potential (ERP) and electroencephalography (EEG) signals in a wide range of brain activity research and clinical studies. This novel study is aimed at dramatically improving the performance of such wavelet-based classifiers by exploiting information offered by the cone of influence (COI) of the continuous wavelet transform (CWT). The COI is a boundary that is superimposed on the wavelet scalogram to delineate the coefficients that are accurate from those that are inaccurate due to edge effects. The features derived from the inaccurate coefficients are, therefore, unreliable. In this study, it is hypothesized that the classifier performance would improve if unreliable features, which are outside the COI, are zeroed out, and the performance would improve even further if those features are cropped out completely. The entire, zeroed out, and cropped scalograms are referred to as the "same" (S)-scalogram, "zeroed out" (Z)-scalogram, and the "valid" (V)-scalogram, respectively. The strategy to validate the hypotheses is to formulate three classification approaches in which the feature vectors are extracted from the (a) S-scalogram in the standard manner, (b) Z-scalogram, and (c) V-scalogram. A subsampling strategy is developed to generate small-sample ERP ensembles to enable customized classifier design for single subjects, and a strategy is developed to select a subset of channels from multiple ERP channels. The three scalogram approaches are implemented using support vector machines, random forests, k-nearest neighbor, multilayer perceptron neural networks, and deep learning convolution neural networks. In order to validate the performance hypotheses, experiments are designed to classify the multi-channel ERPs of five subjects engaged in distinguishing between synonymous and non-synonymous word pairs. The results confirm that the classifiers using the Z-scalogram features outperform those using the S-scalogram features, and the classifiers using the V-scalogram features outperform those using the Z-scalogram features. Most importantly, the relative improvement of the V-scalogram classifiers over the standard S-scalogram classifiers is dramatic. Additionally, enabling the design of customized classifiers for individual subjects is an important contribution to ERP/EEG-based studies and diagnoses of patient-specific disorders.

Decision fusion in healthcare and medicine: a narrative review

OBJECTIVE

To provide an overview of the decision fusion (DF) technique and describe the applications of the technique in healthcare and medicine at prevention, diagnosis, treatment and administrative levels.

BACKGROUND

The rapid development of technology over the past 20 years has led to an explosion in data growth in various industries, like healthcare. Big data analysis within the healthcare systems is essential for arriving to a value-based decision over a period of time. Diversity and uncertainty in big data analytics have made it impossible to analyze data by using conventional data mining techniques and thus alternative solutions are required. DF is a form of data fusion techniques that could increase the accuracy of diagnosis and facilitate interpretation, summarization and sharing of information.

METHODS

We conducted a review of articles published between January 1980 and December 2020 from various databases such as Google Scholar, IEEE, PubMed, Science Direct, Scopus and web of science using the keywords decision fusion (DF), information fusion, healthcare, medicine and big data. A total of 141 articles were included in this narrative review.

CONCLUSIONS

Given the importance of big data analysis in reducing costs and improving the quality of healthcare; along with the potential role of DF in big data analysis, it is recommended to know the full potential of this technique including the advantages, challenges and applications of the technique before its use. Future studies should focus on describing the methodology and types of data used for its applications within the healthcare sector.

Fusion Models for Generalized Classification of Multi-Axial Human Movement: Validation in Sport Performance

We introduce a set of input models for fusing information from ensembles of wearable sensors supporting human performance and telemedicine. Veracity is demonstrated in action classification related to sport, specifically strikes in boxing and taekwondo. Four input models, formulated to be compatible with a broad range of classifiers, are introduced and two diverse classifiers, dynamic time warping (DTW) and convolutional neural networks (CNNs) are implemented in conjunction with the input models. Seven classification models fusing information at the input-level, output-level, and a combination of both are formulated. Action classification for 18 boxing punches and 24 taekwondo kicks demonstrate our fusion classifiers outperform the best DTW and CNN uni-axial classifiers. Furthermore, although DTW is ostensibly an ideal choice for human movements experiencing non-linear variations, our results demonstrate deep learning fusion classifiers outperform DTW. This is a novel finding given that CNNs are normally designed for multi-dimensional data and do not specifically compensate for non-linear variations within signal classes. The generalized formulation enables subject-specific movement classification in a feature-blind fashion with trivial computational expense for trained CNNs. A commercial boxing system, 'Corner', has been produced for real-world mass-market use based on this investigation providing a basis for future telemedicine translation.

Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain

Two multimodal classification models aimed at enhancing object classification through the integration of semantically congruent unimodal stimuli are introduced. The feature-integrating model, inspired by multisensory integration in the subcortical superior colliculus, combines unimodal features which are subsequently classified by a multimodal classifier. The decision-integrating model, inspired by integration in primary cortical areas, classifies unimodal stimuli independently using unimodal classifiers and classifies the combined decisions using a multimodal classifier. The multimodal classifier models are implemented using multilayer perceptrons and multivariate statistical classifiers. Experiments involving the classification of noisy and attenuated auditory and visual representations of ten digits are designed to demonstrate the properties of the multimodal classifiers and to compare the performances of multimodal and unimodal classifiers. The experimental results show that the multimodal classification systems exhibit an important aspect of the "inverse effectiveness principle" by yielding significantly higher classification accuracies when compared with those of the unimodal classifiers. Furthermore, the flexibility offered by the generalized models enables the simulations and evaluations of various combinations of multimodal stimuli and classifiers under varying uncertainty conditions.

Pairwise diversity ranking of polychotomous features for ensemble physiological signal classifiers

It is well known that fusion classifiers for physiological signal classification with diverse components (classifiers or data sets) outperform those with less diverse components. Determining component diversity, therefore, is of the utmost importance in the design of fusion classifiers that are often employed in clinical diagnostic and numerous other pattern recognition problems. In this article, a new pairwise diversity-based ranking strategy is introduced to select a subset of ensemble components, which when combined will be more diverse than any other component subset of the same size. The strategy is unified in the sense that the components can be classifiers or data sets. Moreover, the classifiers and data sets can be polychotomous. Classifier-fusion and data-fusion systems are formulated based on the diversity-based selection strategy, and the application of the two fusion strategies are demonstrated through the classification of multichannel event-related potentials. It is observed that for both classifier and data fusion, the classification accuracy tends to increase/decrease when the diversity of the component ensemble increases/decreases. For the four sets of 14-channel event-related potentials considered, it is shown that data fusion outperforms classifier fusion. Furthermore, it is demonstrated that the combination of data components that yield the best performance, in a relative sense, can be determined through the diversity-based selection strategy.

Detecting missing signals in multichannel recordings by using higher order statistics

In real world applications, a multichannel acquisition system is susceptible of having one or many of its sensors displaced or detached, leading therefore to the loss or corruption of the recorded signals. In this paper, we present a technique for detecting missing or corrupted signals in multichannel recordings. Our approach is based on Higher Order Statistics (HOS) analysis. Our approach is tested on real uterine electromyogram (EMG) signals recorded by 4×4 electrode grid. Results have shown that HOS descriptors can discriminate between the two classes of signals (missing vs. non-missing). These results are supported by statistical analysis using the t-test which indicated good statistical significance of 95% confidence level.

Classification of multichannel uterine EMG signals

Classification of multichannel uterine electromyogram (EMG) signals is addressed. Signals were recorded by a matrix of 16 electrodes. First, signals corresponding to each channel were individually classified using an artificial neural network (ANN) based on radial basis functions (RBF). The results have shown that the classification performance varies from one channel to another. Then, a decision fusion method based on these classification performances was tested. After fusion, the network yielded better classification accuracy than any individual channel could provide. The high percentage of correctly classified labor/non-labor events proves the efficiency of multichannel recordings in detecting labor. These findings can be very useful for the aim of classifying antepartum versus labor patients.

Classification of multichannel EEG patterns using parallel hidden Markov models

In this paper, a parallel hidden-Markov-model (PHMM)-based approach is proposed for the problem of multichannel electroencephalogram (EEG) patterns classification. The approach is based on multi-channel representation of the EEG signals using a parallel combination of HMMs, where each model represents a particular channel. The performance of the proposed algorithm is studied using an artificial EEG database, and two real EEG databases: a database of two classes of EEGs elicited during a task of imagery of hand upward and downward movements of a computer screen cursor (db Ia), and a database of two classes of sensorimotor EEGs elicited during a feedback-regulated left-right motor imagery task (db III). The results show that the proposed algorithm outperforms other commonly used methods with classification rate improvement of 2 and 10% for db Ia and db III, respectively. In addition, the proposed method outperforms a support vector machine classifier with a linear kernel, when both classifiers utilize the same feature set. The results also show that a model architecture which includes a left-to-right scheme with no skips, five states and three Gaussians, outperforms the other tested architectures due to the fact that it allows a better modeling of the temporal sequencing of the EEG components.

Diversity-based selection of components for fusion classifiers

Fusion classifiers with diverse components (classifiers or data sets) outperform those with less diverse components. Determining component diversity, therefore, is of the utmost importance in the design of fusion classifiers which are often employed in clinical diagnostic and numerous other pattern recognition problems. In this paper, a new pairwise diversity-based ranking strategy is introduced to select a subset of ensemble components, which when combined, will be more diverse than any other component subset of the same size. The strategy is unified in the sense that the components can be either polychotomous classifiers or polychotomous data sets. Classifier fusion and data fusion systems are formulated based on the diversity selection strategy and the application of the two fusion strategies are demonstrated through the classification of multi-channel event related potentials (ERPs). From the results it is concluded that data fusion outperforms classifier fusion. It is also shown that the diversity-based data fusion system outperforms the system using randomly selected data components. Furthermore, it is demonstrated that the combination of data components that yield the best performance, in a relative sense, can be determined through the diversity selection strategy.

Central-tendency estimation and nearest-estimate classification of multi-channel evoked potentials

By modeling evoked potentials (EPs) as random vectors in which the EP samples are random variables, a generalized strategy is introduced to determine multivariate central-tendency estimates such as the arithmetic mean, geometric mean, harmonic mean, median, tri-mean, and trimmed-mean. Additionally, a generalized strategy is introduced to develop minimum-distance classifiers based on central tendency estimates. Furthermore, procedures are developed to fuse the decisions of the nearest-estimate classifiers for multi-channel EP classification. The central-tendency estimates of real EPs are compared and it is shown that although the mathematical operations to compute the estimates are quite different, the EP estimates are similar with respect to their overall waveform shapes and latencies. It is also shown that by fusing the classifier decisions across multiple channels, the classification accuracy can be improved significantly when compared with the accuracies of individual channel classifiers.

A dynamic channel selection strategy for dense-array ERP classification

The goal of this paper is to introduce a new strategy to accurately classify event-related potentials (ERPs), recorded using dense electrode arrays, into predefined brain activity categories. The challenge is to exploit the enhanced spatial information offered by dense arrays while overcoming the significant increase in the dimensionality problem introduced by the large increase in the number of channels. These conflicting objectives are achieved by introducing a spatiotemporal-array model to observe the dense-array ERP amplitude variations across channels and time, simultaneously. To account for latency variations and EEG noise in the array elements, each spatiotemporal element in the array is initially modeled as a Gaussian random variable. A two-step process that uses the Kolmogrov-Smirnov test and the Lilliefors test is formulated to select the array elements that have different Gaussian densities across all ERP categories. Selecting spatiotemporal elements that fit the assumed model and also statistically differ across the ERP categories not only ensures high classification accuracies but also decreases the dimensionality significantly. The selection is dynamic in the sense that selecting spatiotemporal-array elements corresponds to selecting ERP samples of different channels at different time instants. Each selected array element is classified using a univariate Gaussian classifier, and the resulting decisions are fused into a decision fusion vector that is classified using a discrete Bayes classifier. By converting an inherently multivariate classification problem into a simpler problem involving only univariate classifications, the dimensionality problem that plagues the design of practical multivariate ERP classifiers is circumvented. Consequently, classifiers can be designed to classify the ERPs that are unique to an individual without having to collect a prohibitively large ERP dataset from him/her. The application of the resulting dynamic-channel-selection-based classification strategy is demonstrated by designing and testing classifiers for eight subjects using ERPs from a Stroop color test and it is shown that the strategy yields high classification accuracies. Finally, it is noted that because of the generalized formulation of the strategy, it can be applied to various other problems involving the classification of multivariate signals acquired from multiple identical or multiple heterogeneous sensors.

Spatio-temporal modeling for dense array ERP classification

A new strategy is introduced to exploit the enhanced spatial resolution offered by dense electrode arrays and to solve the dimensionality problem that plagues the design and evaluation of practical dense array event-related potential (ERP) classifiers. A spatio-temporal model is introduced to observe the dense array ERP amplitude variations across channels and time, simultaneously. Dimensionality reduction is achieved by selecting elements of the spatio-temporal arrays which differ in their probability distributions across the brain activity classes. Each selected spatio-temporal element is classified using an univariate Gaussian classifier and the resulting decisions are fused into a decision fusion vector which is classified using a discrete Bayes vector classifier. Using ERPs from a Stroop color test, it is shown that the performance improves significantly when the strategy is applied to normalized spatio-temporal ERP arrays. The main advantage of the new strategy is that it is not constrained by the dimensionality of the ERP vector. Consequently, it can be used to design ERP classifiers specialized for individual test subjects without having to collect a large number of ERPs from groups of subjects in order to solve the dimensionality problem.

An ensemble-based incremental learning approach to data fusion

This paper introduces Learn++, an ensemble of classifiers based algorithm originally developed for incremental learning, and now adapted for information/data fusion applications. Recognizing the conceptual similarity between incremental learning and data fusion, Learn++ follows an alternative approach to data fusion, i.e., sequentially generating an ensemble of classifiers that specifically seek the most discriminating information from each data set. It was observed that Learn++ based data fusion consistently outperforms a similarly configured ensemble classifier trained on any of the individual data sources across several applications. Furthermore, even if the classifiers trained on individual data sources are fine tuned for the given problem, Learn++ can still achieve a statistically significant improvement by combining them, if the additional data sets carry complementary information. The algorithm can also identify-albeit indirectly-those data sets that do not carry such additional information. Finally, it was shown that the algorithm can consecutively learn both the supplementary novel information coming from additional data of the same source, and the complementary information coming from new data sources without requiring access to any of the previously seen data.