Lyapunov Theory-Based Fusion Neural Networks for the Identification of Dynamic Nonlinear Systems

A Bidirectional Feedforward Neural Network Architecture Using the Discretized Neural Memory Ordinary Differential Equation

Deep Feedforward Neural Networks (FNNs) with skip connections have revolutionized various image recognition tasks. In this paper, we propose a novel architecture called bidirectional FNN (BiFNN), which utilizes skip connections to aggregate features between its forward and backward paths. The BiFNN accepts any FNN as a plugin that can incorporate any general FNN model into its forward path, introducing only a few additional parameters in the cross-path connections. The backward path is implemented as a nonparameter layer, utilizing a discretized form of the neural memory Ordinary Differential Equation (nmODE), which is named [Formula: see text]-net. We provide a proof of convergence for the [Formula: see text]-net and evaluate its initial value problem. Our proposed architecture is evaluated on diverse image recognition datasets, including Fashion-MNIST, SVHN, CIFAR-10, CIFAR-100, and Tiny-ImageNet. The results demonstrate that BiFNNs offer significant improvements compared to embedded models such as ConvMixer, ResNet, ResNeXt, and Vision Transformer. Furthermore, BiFNNs can be fine-tuned to achieve comparable performance with embedded models on Tiny-ImageNet and ImageNet-1K datasets by loading the same pretrained parameters.

Hysteresis Modeling in Iron-Dominated Magnets Based on a Multi-Layered Narx Neural Network Approach

A full-fledged neural network modeling, based on a Multi-layered Nonlinear Autoregressive Exogenous Neural Network (NARX) architecture, is proposed for quasi-static and dynamic hysteresis loops, one of the most challenging topics for computational magnetism. This modeling approach overcomes drawbacks in attaining better than percent-level accuracy of classical and recent approaches for accelerator magnets, that combine hybridization of standard hysteretic models and neural network architectures. By means of an incremental procedure, different Deep Neural Network Architectures are selected, fine-tuned and tested in order to predict magnetic hysteresis in the context of electromagnets. Tests and results show that the proposed NARX architecture best fits the measured magnetic field behavior of a reference quadrupole at CERN. In particular, the proposed modeling framework leads to a percent error below 0.02% for the magnetic field prediction, thus outperforming state of the art approaches and paving a very promising way for future real time applications.

Decoding Multiple Sound-Categories in the Auditory Cortex by Neural Networks: An fNIRS Study

This study aims to decode the hemodynamic responses (HRs) evoked by multiple sound-categories using functional near-infrared spectroscopy (fNIRS). The six different sounds were given as stimuli (English, non-English, annoying, nature, music, and gunshot). The oxy-hemoglobin (HbO) concentration changes are measured in both hemispheres of the auditory cortex while 18 healthy subjects listen to 10-s blocks of six sound-categories. Long short-term memory (LSTM) networks were used as a classifier. The classification accuracy was 20.38 ± 4.63% with six class classification. Though LSTM networks' performance was a little higher than chance levels, it is noteworthy that we could classify the data subject-wise without feature selections.

A Novel Method for Sleep-Stage Classification Based on Sonification of Sleep Electroencephalogram Signals Using Wavelet Transform and Recurrent Neural Network

INTRODUCTION

Visual sleep-stage scoring is a time-consuming technique that cannot extract the nonlinear characteristics of electroencephalogram (EEG). This article presents a novel method for sleep-stage differentiation based on sonification of sleep-EEG signals using wavelet transform and recurrent neural network (RNN).

METHODS

Two RNNs were designed and trained separately based on a database of classical guitar pieces and Kurdish tanbur Makams using a long short-term memory model. Moreover, discrete wavelet transform and wavelet packet decomposition were used to determine the association between the EEG signals and musical pitches. Continuous wavelet transform was applied to extract musical beat-based features from the EEG. Then, the pretrained RNN was used to generate music. To test the proposed model, 11 sleep EEGs were mapped onto the guitar and tanbur frequency intervals and presented to the pretrained RNN. Next, the generated music was randomly presented to 2 neurologists.

RESULTS

The proposed model classified the sleep stages with an accuracy of >81% for tanbur and more than 93% for guitar musical pieces. The inter-rater reliability measured by Cohen's kappa coefficient (κ) revealed good reliability for both tanbur (κ = 0.64, p < 0.001) and guitar musical pieces (κ = 0.85, p < 0.001).

CONCLUSION

The present EEG sonification method leads to valid sleep staging by clinicians. The method could be used on various EEG databases for classification, differentiation, diagnosis, and treatment purposes. Real-time EEG sonification can be used as a feedback tool for replanning of neurophysiological functions for the management of many neurological and psychiatric disorders in the future.

Mechanisms Contributing to the Generation of Mayer Waves

Mayer waves may synchronize overlapping propriobulbar interneuronal microcircuits constituting the respiratory rhythm and pattern generator, sympathetic oscillators, and cardiac vagal preganglionic neurons. Initially described by Sir Sigmund Mayer in the year 1876 in the arterial pressure waveform of anesthetized rabbits, authors have since extensively observed these oscillations in recordings of hemodynamic variables, including arterial pressure waveform, peripheral resistance, and blood flow. Authors would later reveal the presence of these oscillations in sympathetic neural efferent discharge and brainstem and spinal zones corresponding with sympathetic oscillators. Mayer wave central tendency proves highly consistent within, though the specific frequency band varies extensively across, species. Striking resemblance of the Mayer wave central tendency to the species-specific baroreflex resonant frequency has led the majority of investigators to comfortably presume, and generate computational models premised upon, a baroreflex origin of these oscillations. Empirical interrogation of this conjecture has generated variable results and derivative interpretations. Sinoaortic denervation and effector sympathectomy variably reduces or abolishes spectral power contained within the Mayer wave frequency band. Refractorines of Mayer wave generation to barodeafferentation lends credence to the hypothesis these waves are chiefly generated by brainstem propriobulbar and spinal cord propriospinal interneuronal microcircuit oscillators and likely modulated by the baroreflex. The presence of these waves in unitary discharge of medullary lateral tegmental field and rostral ventrolateral medullary neurons (contemporaneously exhibiting fast sympathetic rhythms [2-6 and 10 Hz bands]) in spectral variability in vagotomized pentobarbital-anesthetized and unanesthetized midcollicular (i.e., intercollicular) decerebrate cats supports genesis of Mayer waves by supraspinal sympathetic microcircuit oscillators. Persistence of these waves following high cervical transection in vagotomized unanesthetized midcollicular decerebrate cats would seem to suggest spinal sympathetic microcircuit oscillators generate these waves. The widespread presence of Mayer waves in brainstem sympathetic-related and non-sympathetic-related cells would seem to betray a general tendency of neurons to oscillate at this frequency. We have thus presented an extensive and, hopefully cohesive, discourse evaluating, and evolving the interpretive consideration of, evidence seeking to illumine our understanding of origins of, and insight into mechanisms contributing to, the genesis of Mayer waves. We have predicated our arguments and conjectures in the substance and matter of empirical data, though we have occasionally waxed philosophical beyond these traditional confines in suggesting interpretations exceeding these limits. We believe our synthesis and interpretation of the relevant literature will fruitfully inspire future studies from the perspective of a more intimate appreciation and conceptualization of network mechanisms generating oscillatory variability in neuronal and neural outputs. Our evaluation of Mayer waves informs a novel set of disciplines we term quantum neurophysics extendable to describing subatomic reality. Beyond informing our appreciation of mechanisms generating sympathetic oscillations, Mayer waves may constitute an intrinsic property of neurons extant throughout the cerebrum, brainstem, and spinal cord or reflect an emergent property of interactions between arteriogenic and neuronal oscillations.

Neural Computing Enhanced Parameter Estimation for Multi-Input and Multi-Output Total Non-Linear Dynamic Models

In this paper, a gradient descent algorithm is proposed for the parameter estimation of multi-input and multi-output (MIMO) total non-linear dynamic models. Firstly, the MIMO total non-linear model is mapped to a non-completely connected feedforward neural network, that is, the parameters of the total non-linear model are mapped to the connection weights of the neural network. Then, based on the minimization of network error, a weight-updating algorithm, that is, an estimation algorithm of model parameters, is proposed with the convergence conditions of a non-completely connected feedforward network. In further determining the variables of the model set, a method of model structure detection is proposed for selecting a group of important items from the whole variable candidate set. In order to verify the usefulness of the parameter identification process, we provide a virtual bench test example for the numerical analysis and user-friendly instructions for potential applications.