Generalization characteristics of complex-valued feedforward neural networks in relation to signal coherence

Perturbation dimensional reduction theory assisted low complexity convolutional neural network equalizer for optical communications

The integration of neural networks with perturbation theory has been recognized as a pivotal factor contributing to the enhancement of transmission efficiency in modern optical communication systems. Research focuses on balancing superior equalization performance against system complexity minimization. This paper presents a novel and efficient data dimensionality reduction compression methodology for developing low-complexity neural network-based equalizers intended to mitigate nonlinear impairments in optical fiber communication systems. Firstly, we propose a dimensionality reduction (DR) method exploiting the spatial symmetry of perturbation coefficients, enabling low-complexity DR feature map construction at the input end. Then, a complex-valued CNN (CvCNN) equalizer is then designed, utilizing single-channel complex-valued feature maps to preserve data characteristics while further reducing complexity. The effectiveness of the proposed equalizer is verified through numerical studies, and further performance comparisons and complexity analyses with other equalizers are conducted in a 20 GBaud PDM 64-QAM coherent optical transmission experimental system. The research and analysis are grounded in a fair performance comparison of Q-factors, accompanied by thorough analyses of time and space complexity under equivalent performance conditions. Finally, The DR feature map achieves an 82.64% reduction in effective feature units. When deployed in CvCNN, the equalizer reduces space and time complexity by 66.90% and 70.55%, respectively.

Complex-Valued Convolutional Gated Recurrent Neural Network for Ultrasound Beamforming

Ultrasound detection is a potent tool for the clinical diagnosis of various diseases due to its real-time, convenient, and noninvasive qualities. Yet, existing ultrasound beamforming and related methods face a big challenge to improve both the quality and speed of imaging for the required clinical applications. The most notable characteristic of ultrasound signal data is its spatial and temporal features. Because most signals are complex-valued, directly processing them by using real-valued networks leads to phase distortion and inaccurate output. In this study, for the first time, we propose a complex-valued convolutional gated recurrent (CCGR) neural network to handle ultrasound analytic signals with the aforementioned properties. The complex-valued network operations proposed in this study improve the beamforming accuracy of complex-valued ultrasound signals over traditional real-valued methods. Further, the proposed deep integration of convolution and recurrent neural networks makes a great contribution to extracting rich and informative ultrasound signal features. Our experimental results reveal its outstanding imaging quality over existing state-of-the-art methods. More significantly, its ultrafast processing speed of only 0.07 s per image promises considerable clinical application potential. The code is available at https://github.com/zhangzm0128/CCGR.

Image restoration in frequency space using complex-valued CNNs

Real-valued convolutional neural networks (RV-CNNs) in the spatial domain have outperformed classical approaches in many image restoration tasks such as image denoising and super-resolution. Fourier analysis of the results produced by these spatial domain models reveals the limitations of these models in properly processing the full frequency spectrum. This lack of complete spectral information can result in missing textural and structural elements. To address this limitation, we explore the potential of complex-valued convolutional neural networks (CV-CNNs) for image restoration tasks. CV-CNNs have shown remarkable performance in tasks such as image classification and segmentation. However, CV-CNNs for image restoration problems in the frequency domain have not been fully investigated to address the aforementioned issues. Here, we propose several novel CV-CNN-based models equipped with complex-valued attention gates for image denoising and super-resolution in the frequency domains. We also show that our CV-CNN-based models outperform their real-valued counterparts for denoising super-resolution structured illumination microscopy (SR-SIM) and conventional image datasets. Furthermore, the experimental results show that our proposed CV-CNN-based models preserve the frequency spectrum better than their real-valued counterparts in the denoising task. Based on these findings, we conclude that CV-CNN-based methods provide a plausible and beneficial deep learning approach for image restoration in the frequency domain.

Complex Residual Attention U-Net for Fast Ultrasound Imaging from a Single Plane-Wave Equivalent to Diverging Wave Imaging

Plane wave imaging persists as a focal point of research due to its high frame rate and low complexity. However, in spite of these advantages, its performance can be compromised by several factors such as noise, speckle, and artifacts that affect the image quality and resolution. In this paper, we propose an attention-based complex convolutional residual U-Net to reconstruct improved in-phase/quadrature complex data from a single insonification acquisition that matches diverging wave imaging. Our approach introduces an attention mechanism to the complex domain in conjunction with complex convolution to incorporate phase information and improve the image quality matching images obtained using coherent compounding imaging. To validate the effectiveness of this method, we trained our network on a simulated phased array dataset and evaluated it using in vitro and in vivo data. The experimental results show that our approach improved the ultrasound image quality by focusing the network's attention on critical aspects of the complex data to identify and separate different regions of interest from background noise.

Complex-valued recurrent neural network equalizer with low complexity for a 120-Gbps 50-km optical PAM-4 IM/DD system

This paper introduces a novel complex-valued recurrent neural networks equalizer (RNNE) designed for a 120-Gbps, 50-km optical 4-level pulse-amplitude modulation (PAM-4) intensity modulation and direct detection (IM/DD) system. By mapping adjacent symbols of PAM-4 signals onto the complex domain, the correlation between two adjacent symbols of PAM-4 signals can be preserved. Based on experimental results, the proposed complex-valued RNNE outperforms the traditional real-valued RNNE with a 1.38-dB system power budget gain at the 7% overhead forward error correction BER threshold of 3.8 × 10-3. We believe that complex-valued RNNE has an advantage over real-valued RNNE in processing real-valued signals in IM/DD systems.

A Sparse Model-Inspired Deep Thresholding Network for Exponential Signal Reconstruction-Application in Fast Biological Spectroscopy

The nonuniform sampling (NUS) is a powerful approach to enable fast acquisition but requires sophisticated reconstruction algorithms. Faithful reconstruction from partially sampled exponentials is highly expected in general signal processing and many applications. Deep learning (DL) has shown astonishing potential in this field, but many existing problems, such as lack of robustness and explainability, greatly limit its applications. In this work, by combining the merits of the sparse model-based optimization method and data-driven DL, we propose a DL architecture for spectra reconstruction from undersampled data, called MoDern. It follows the iterative reconstruction in solving a sparse model to build the neural network, and we elaborately design a learnable soft-thresholding to adaptively eliminate the spectrum artifacts introduced by undersampling. Extensive results on both synthetic and biological data show that MoDern enables more robust, high-fidelity, and ultrafast reconstruction than the state-of-the-art methods. Remarkably, MoDern has a small number of network parameters and is trained on solely synthetic data while generalizing well to biological data in various scenarios. Furthermore, we extend it to an open-access and easy-to-use cloud computing platform (XCloud-MoDern), contributing a promising strategy for further development of biological applications.

Exponential Signal Reconstruction With Deep Hankel Matrix Factorization

Exponential function is a basic form of temporal signals, and how to fast acquire this signal is one of the fundamental problems and frontiers in signal processing. To achieve this goal, partial data may be acquired but result in severe artifacts in its spectrum, which is the Fourier transform of exponentials. Thus, reliable spectrum reconstruction is highly expected in the fast data acquisition in many applications, such as chemistry, biology, and medical imaging. In this work, we propose a deep learning method whose neural network structure is designed by imitating the iterative process in the model-based state-of-the-art exponentials' reconstruction method with the low-rank Hankel matrix factorization. With the experiments on synthetic data and realistic biological magnetic resonance signals, we demonstrate that the new method yields much lower reconstruction errors and preserves the low-intensity signals much better than compared methods.

Fully Complex-Valued Dendritic Neuron Model

A single dendritic neuron model (DNM) that owns the nonlinear information processing ability of dendrites has been widely used for classification and prediction. Complex-valued neural networks that consist of a number of multiple/deep-layer McCulloch-Pitts neurons have achieved great successes so far since neural computing was utilized for signal processing. Yet no complex value representations appear in single neuron architectures. In this article, we first extend DNM from a real-value domain to a complex-valued one. Performance of complex-valued DNM (CDNM) is evaluated through a complex XOR problem, a non-minimum phase equalization problem, and a real-world wind prediction task. Also, a comparative analysis on a set of elementary transcendental functions as an activation function is implemented and preparatory experiments are carried out for determining hyperparameters. The experimental results indicate that the proposed CDNM significantly outperforms real-valued DNM, complex-valued multi-layer perceptron, and other complex-valued neuron models.

Automatic Modulation Classification for Underwater Acoustic Communication Signals Based on Deep Complex Networks

Automatic modulation classification (AMC) is an important method for monitoring and identifying any underwater communication interference. Since the underwater acoustic communication scenario is full of multi-path fading and ocean ambient noise (OAN), coupled with the application of modern communication technology, which is usually susceptible to environmental influences, automatic modulation classification (AMC) becomes particularly difficult when it comes to an underwater scenario. Motivated by the deep complex networks (DCN), which have an innate ability to process complex data, we explore DCN for AMC of underwater acoustic communication signals. To integrate the signal processing method with deep learning and overcome the influences of underwater acoustic channels, we propose two complex physical signal processing layers based on DCN. The proposed layers include a deep complex matched filter (DCMF) and deep complex channel equalizer (DCCE), which are designed to remove noise and reduce the influence of multi-path fading for the received signals, respectively. Hierarchical DCN is constructed using the proposed method to achieve better performance of AMC. The influence of the real-world underwater acoustic communication scenario is taken into account; two underwater acoustic multi-path fading channels are conducted using the real-world ocean observation dataset, white Gaussian noise, and real-world OAN are used as the additive noise, respectively. Contrastive experiments show that the AMC based on DCN can achieve better performance than the traditional deep neural network based on real value (the average accuracy of the DCN is 5.3% higher than real-valued DNN). The proposed method based on DCN can effectively reduce the influence of underwater acoustic channels and improve the AMC performance in different underwater acoustic channels. The performance of the proposed method was verified on the real-world dataset. In the underwater acoustic channels, the proposed method outperforms a series of advanced AMC method.

Throughput Maximization Using Deep Complex Networks for Industrial Internet of Things

The high-density Industrial Internet of Things needs to meet the requirements of high-density device access and massive data transmission, which requires the support of multiple-input multiple-output (MIMO) antenna cognitive systems to keep high throughput. In such a system, spectral efficiency (SE) optimization based on dynamic power allocation is an effective way to enhance the network throughput as the channel quality variations significantly affect the spectral efficiency performance. Deep learning methods have illustrated the ability to efficiently solve the non-convexity of resource allocation problems induced by the channel multi-path and inter-user interference effects. However, current real-valued deep-learning-based power allocation methods have failed to utilize the representational capacity of complex-valued data as they regard the complex-valued channel data as two parts: real and imaginary data. In this paper, we propose a complex-valued power allocation network (AttCVNN) with cross-channel and in-channel attention mechanisms to improve the model performance where the former considers the relationship between cognitive users and the primary user, i.e., inter-network users, while the latter focuses on the relationship among cognitive users, i.e., intra-network users. Comparison experiments indicate that the proposed AttCVNN notably outperforms both the equal power allocation method (EPM) and the real-valued and the complex-valued fully connected network (FNN, CVFNN) and shows a better convergence rate in the training phase than the real-valued convolutional neural network (AttCNN).

Complex-valued Neural Network-based Quantum Language Models

Language modeling is essential in Natural Language Processing and Information Retrieval related tasks. After the statistical language models, Quantum Language Model (QLM) has been proposed to unify both single words and compound terms in the same probability space without extending term space exponentially. Although QLM achieved good performance in ad hoc retrieval, it still has two major limitations: (1) QLM cannot make use of supervised information, mainly due to the iterative and non-differentiable estimation of the density matrix, which represents both queries and documents in QLM. (2) QLM assumes the exchangeability of words or word dependencies, neglecting the order or position information of words.

This article aims to generalize QLM and make it applicable to more complicated matching tasks (e.g., Question Answering) beyond ad hoc retrieval. We propose a complex-valued neural network-based QLM solution called C-NNQLM to employ an end-to-end approach to build and train density matrices in a light-weight and differentiable manner, and it can therefore make use of external well-trained word vectors and supervised labels. Furthermore, C-NNQLM adopts complex-valued word vectors whose phase vectors can directly encode the order (or position) information of words. Note that complex numbers are also essential in the quantum theory. We show that the real-valued NNQLM (R-NNQLM) is a special case of C-NNQLM.

The experimental results on the QA task show that both R-NNQLM and C-NNQLM achieve much better performance than the vanilla QLM, and C-NNQLM’s performance is on par with state-of-the-art neural network models. We also evaluate the proposed C-NNQLM on text classification and document retrieval tasks. The results on most datasets show that the C-NNQLM can outperform R-NNQLM, which demonstrates the usefulness of the complex representation for words and sentences in C-NNQLM.

Nonblind Image Deblurring via Deep Learning in Complex Field

Nonblind image deblurring is about recovering the latent clear image from a blurry one generated by a known blur kernel, which is an often-seen yet challenging inverse problem in imaging. Its key is how to robustly suppress noise magnification during the inversion process. Recent approaches made a breakthrough by exploiting convolutional neural network (CNN)-based denoising priors in the image domain or the gradient domain, which allows using a CNN for noise suppression. The performance of these approaches is highly dependent on the effectiveness of the denoising CNN in removing magnified noise whose distribution is unknown and varies at different iterations of the deblurring process for different images. In this article, we introduce a CNN-based image prior defined in the Gabor domain. The prior not only utilizes the optimal space-frequency resolution and strong orientation selectivity of the Gabor transform but also enables using complex-valued (CV) representations in intermediate processing for better denoising. A CV CNN is developed to exploit the benefits of the CV representations, with better generalization to handle unknown noises over the real-valued ones. Combining our Gabor-domain CV CNN-based prior with an unrolling scheme, we propose a deep-learning-based approach to nonblind image deblurring. Extensive experiments have demonstrated the superior performance of the proposed approach over the state-of-the-art ones.

The Loss Surface of Deep Linear Networks Viewed Through the Algebraic Geometry Lens

By using the viewpoint of modern computational algebraic geometry, we explore properties of the optimization landscapes of deep linear neural network models. After providing clarification on the various definitions of "flat" minima, we show that the geometrically flat minima, which are merely artifacts of residual continuous symmetries of the deep linear networks, can be straightforwardly removed by a generalized L₂-regularization. Then, we establish upper bounds on the number of isolated stationary points of these networks with the help of algebraic geometry. Combining these upper bounds with a method in numerical algebraic geometry, we find all stationary points for modest depth and matrix size. We demonstrate that, in the presence of the non-zero regularization, deep linear networks can indeed possess local minima which are not global minima. Finally, we show that even though the number of stationary points increases as the number of neurons (regularization parameters) increases (decreases), higher index saddles are surprisingly rare.

All-optical graph representation learning using integrated diffractive photonic computing units

Photonic neural networks perform brain-inspired computations using photons instead of electrons to achieve substantially improved computing performance. However, existing architectures can only handle data with regular structures but fail to generalize to graph-structured data beyond Euclidean space. Here, we propose the diffractive graph neural network (DGNN), an all-optical graph representation learning architecture based on the diffractive photonic computing units (DPUs) and on-chip optical devices to address this limitation. Specifically, the graph node attributes are encoded into strip optical waveguides, transformed by DPUs, and aggregated by optical couplers to extract their feature representations. DGNN captures complex dependencies among node neighborhoods during the light-speed optical message passing over graph structures. We demonstrate the applications of DGNN for node and graph-level classification tasks with benchmark databases and achieve superior performance. Our work opens up a new direction for designing application-specific integrated photonic circuits for high-efficiency processing large-scale graph data structures using deep learning.

Event-Triggered Exponential Stabilization for State-Based Switched Inertial Complex-Valued Neural Networks With Multiple Delays

This article explores the exponential stabilization issue of a class of state-based switched inertial complex-valued neural networks with multiple delays via event-triggered control. First, the state-based switched inertial complex-valued neural networks with multiple delays are modeled. Second, by separating the real and imaginary parts of complex values, the state-based switched inertial complex-valued neural networks are transformed into two state-based switched inertial real-valued neural networks. Through the variable substitution method, the model of the second-order inertial neural networks is transformed into a model of the first-order neural networks. Third, an event-triggered controller with the transmission sequence is designed to study the exponential stabilization issue of neural networks constructed above. Then, by constructing the Lyapunov functions and based on some inequalities, we obtain sufficient conditions for exponential stabilization of the proposed neural networks. Furthermore, it is proved that the Zeno phenomenon cannot happen under the designed event-triggered controller. Finally, a simulation example is given to illustrate the correctness of the results.

SurReal: Complex-Valued Learning as Principled Transformations on a Scaling and Rotation Manifold

Complex-valued data are ubiquitous in signal and image processing applications, and complex-valued representations in deep learning have appealing theoretical properties. While these aspects have long been recognized, complex-valued deep learning continues to lag far behind its real-valued counterpart. We propose a principled geometric approach to complex-valued deep learning. Complex-valued data could often be subject to arbitrary complex-valued scaling; as a result, real and imaginary components could covary. Instead of treating complex values as two independent channels of real values, we recognize their underlying geometry: we model the space of complex numbers as a product manifold of nonzero scaling and planar rotations. Arbitrary complex-valued scaling naturally becomes a group of transitive actions on this manifold. We propose to extend the property instead of the form of real-valued functions to the complex domain. We define convolution as the weighted Fréchet mean on the manifold that is equivariant to the group of scaling/rotation actions and define distance transform on the manifold that is invariant to the action group. The manifold perspective also allows us to define nonlinear activation functions, such as tangent ReLU and G -transport, as well as residual connections on the manifold-valued data. We dub our model SurReal, as our experiments on MSTAR and RadioML deliver high performance with only a fractional size of real- and complex-valued baseline models.

Complex Convolutional Neural Networks for Ultrafast Ultrasound Imaging Reconstruction From In-Phase/Quadrature Signal

Ultrafast ultrasound imaging remains an active area of interest in the ultrasound community due to its ultrahigh frame rates. Recently, a wide variety of studies based on deep learning have sought to improve ultrafast ultrasound imaging. Most of these approaches have been performed on radio frequency (RF) signals. However, in- phase/quadrature (I/Q) digital beamformers are now widely used as low-cost strategies. In this work, we used complex convolutional neural networks for reconstruction of ultrasound images from I/Q signals. We recently described a convolutional neural network architecture called ID-Net, which exploited an inception layer designed for reconstruction of RF diverging-wave ultrasound images. In the present study, we derive the complex equivalent of this network, i.e., complex-valued inception for diverging-wave network (CID-Net) that operates on I/Q data. We provide experimental evidence that CID-Net provides the same image quality as that obtained from RF-trained convolutional neural networks, i.e., using only three I/Q images, CID-Net produces high-quality images that can compete with those obtained by coherently compounding 31 RF images. Moreover, we show that CID-Net outperforms the straightforward architecture that consists of processing real and imaginary parts of the I/Q signal separately, which thereby indicates the importance of consistently processing the I/Q signals using a network that exploits the complex nature of such signals.

Proposal of higher-order tensor independent component analysis for signal separation in multiple-input multiple-output respiration/heartbeat remote sensing. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021;2021:325-328. [PMID: 34891301 DOI: 10.1109/embc46164.2021.9630656] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

This paper proposes higher-order tensor independent component analysis (HOT-ICA). HOT-ICA is a tensor ICA that makes effective use of the relationships among the axes of a separating tensor. We deal with multiple-target signal separation in a multiple-input multiple-output (MIMO) radar system to detect respiration and heartbeat. Numerical physical experiments demonstrate the significance of the HOT-ICA which keeps the tensor structure unchanged to fully utilizes the high-dimensionality of the separation tensor.

Optical random phase dropout in a diffractive deep neural network

Unitary learning is a backpropagation (BP) method that serves to update unitary weights in fully connected deep complex-valued neural networks, meeting a prior unitary in an active modulation diffractive deep neural network. However, the square matrix characteristic of unitary weights in each layer results in its learning belonging to a small-sample training, which produces an almost useless network that has a fairly poor generalization capability. To alleviate such a serious over-fitting problem, in this Letter, optical random phase dropout is formulated and designed. The equivalence between unitary forward and diffractive networks deduces a synthetic mask that is seamlessly compounded with a computational modulation and a random sampling comb called dropout. The dropout is filled with random phases in its zero positions that satisfy the Bernoulli distribution, which could slightly deflect parts of transmitted optical rays in each output end to generate statistical inference networks. The enhancement of generalization benefits from the fact that massively parallel full connection with different optical links is involved in the training. The random phase comb introduced into unitary BP is in the form of conjugation, which indicates the significance of optical BP.

Reverse engineering gene regulatory network based on complex-valued ordinary differential equation model

BACKGROUND

The growing researches of molecular biology reveal that complex life phenomena have the ability to demonstrating various types of interactions in the level of genomics. To establish the interactions between genes or proteins and understand the intrinsic mechanisms of biological systems have become an urgent need and study hotspot.

RESULTS

In order to forecast gene expression data and identify more accurate gene regulatory network, complex-valued version of ordinary differential equation (CVODE) is proposed in this paper. In order to optimize CVODE model, a complex-valued hybrid evolutionary method based on Grammar-guided genetic programming and complex-valued firefly algorithm is presented.

CONCLUSIONS

When tested on three real gene expression datasets from E. coli and Human Cell, the experiment results suggest that CVODE model could improve 20-50% prediction accuracy of gene expression data, which could also infer more true-positive regulatory relationships and less false-positive regulations than ordinary differential equation.

Adaptive deep-learning equalizer based on constellation partitioning scheme with reduced computational complexity in UVLC system

Visible light communication (VLC) system has emerged as a promising solution for high-speed underwater data transmission. To tackle with the linear and nonlinear impairments, deep learning inspired equalization is introduced into VLC. Despite their success in accuracy, deep learning approaches often come with high computational budget. In this paper, we propose an adaptive deep-learning equalizer based on complex-valued neural network and constellation partitioning scheme for 64 QAM-CAP modulated underwater VLC (UVLC) system. Inspired by the fact that symbols modulated at different levels experience various extent of nonlinear distortion, we adaptively partition the received symbols in constellation and design compact equalization networks for specific regions to reduce computation consumption. Experiments demonstrate that the partitioned equalizer can achieve the bit error rate below the 7% hard-decision forward error correction (HD-FEC) limit of 3.8 × 10^-3 at 2.85 Gbps similar to the standard complex-valued network, yet with 56.1% total computational complexity reduction. This work paves the path for online data processing in high speed UVLC system.

A Quaternion Gated Recurrent Unit Neural Network for Sensor Fusion

Recurrent Neural Networks (RNNs) are known for their ability to learn relationships within temporal sequences. Gated Recurrent Unit (GRU) networks have found use in challenging time-dependent applications such as Natural Language Processing (NLP), financial analysis and sensor fusion due to their capability to cope with the vanishing gradient problem. GRUs are also known to be more computationally efficient than their variant, the Long Short-Term Memory neural network (LSTM), due to their less complex structure and as such, are more suitable for applications requiring more efficient management of computational resources. Many of such applications require a stronger mapping of their features to further enhance the prediction accuracy. A novel Quaternion Gated Recurrent Unit (QGRU) is proposed in this paper, which leverages the internal and external dependencies within the quaternion algebra to map correlations within and across multidimensional features. The QGRU can be used to efficiently capture the inter- and intra-dependencies within multidimensional features unlike the GRU, which only captures the dependencies within the sequence. Furthermore, the performance of the proposed method is evaluated on a sensor fusion problem involving navigation in Global Navigation Satellite System (GNSS) deprived environments as well as a human activity recognition problem. The results obtained show that the QGRU produces competitive results with almost 3.7 times fewer parameters compared to the GRU. The QGRU code is available at.

Shadow-Cuts Minimization/Maximization and Complex Hopfield Neural Networks

In this article, we continue our very recent work by extending it to the complex case. Having been inspired by the real Hopfield neural network (HNN) results, our investigations here yield various novel results, some of which are as follows. First, extending the "biased pseudo-cut" concept to the complex HNN (CHNN) case, we introduce a "shadow-cut" that is defined as the sum of intercluster phased edges. Second, while the discrete-time real HNN strictly minimizes the "biased pseudo-cut" in each neuron state change, the CHNN "tends" to minimize the shadow-cut (as the CHNN energy function is minimized). Third, these definitions pose a novel L-phased graph clustering (partitioning) problem in which the sum of the shadow-cuts is minimized (or maximized) for the Hermitian complex and the directed graphs whose edges are (possibly arbitrary positive/negative) complex numbers. Finally, combining the CHNN and the pioneering algorithm GADIA of Babadi and Tarokh and their modified versions, we propose simple indirect algorithms to solve the defined shadow-cuts minimization/maximization problem. The proposed algorithms naturally include the CHNN as well as the GADIA as its special cases. The computer simulations confirm the findings.

Phase-shift determination for a 4 × 4 intelligent photonic neural network with compatible learning

Intelligent photonic circuits (IPCs) tuned with an appropriate phase-shift vector could enable a photonic intelligent matrix possibly implemented in multiple neural layers for a task-oriented topologies. A photonic Mach-Zehnder Interferometer (MZI) is a fundamental photonic component in IPCs, whose matrix representation could be broadcasted into an arbitrary matrix that is equipped with an optimized phase-shift vector. The initialized MZIs' phases are tentatively probed between analytical elements and a digital weight matrix that is learned from samples with efficient compatible learning for complex-valued neural networks. Nonlinear least squares is utilized to formulate a phase determination system to refine the optimal phase-shift solutions. The robustness of phase determination system for photonic neural networks is discussed in detail. For a preliminary implementation, a basic 4×4 intelligent photonic neural network is utilized to verify the proof of concept on phase-shift determination in IPC through numerical experiments.

Scheme for automatic differentiation of complex loss functions with applications in quantum physics

The past few years have seen a significant transfer of tools from machine learning to solve quantum physics problems. Automatic differentiation is one standard algorithm used to efficiently compute gradients of loss functions for generic neural networks. In this work we show how to extend automatic differentiation to the case of complex loss function in a way that can be readily implemented in existing frameworks and which is compatible with the common case of real loss functions. We then combine this tool with matrix product states and apply it to compute the ground state and the steady state of a close and an open quantum system. Compared to the traditional density matrix renormalization group algorithm, complex automatic differentiation allows both straightforward GPU accelerations as well as generalizations to different types of tensor and neural networks.

An optical neural chip for implementing complex-valued neural network

Complex-valued neural networks have many advantages over their real-valued counterparts. Conventional digital electronic computing platforms are incapable of executing truly complex-valued representations and operations. In contrast, optical computing platforms that encode information in both phase and magnitude can execute complex arithmetic by optical interference, offering significantly enhanced computational speed and energy efficiency. However, to date, most demonstrations of optical neural networks still only utilize conventional real-valued frameworks that are designed for digital computers, forfeiting many of the advantages of optical computing such as efficient complex-valued operations. In this article, we highlight an optical neural chip (ONC) that implements truly complex-valued neural networks. We benchmark the performance of our complex-valued ONC in four settings: simple Boolean tasks, species classification of an Iris dataset, classifying nonlinear datasets (Circle and Spiral), and handwriting recognition. Strong learning capabilities (i.e., high accuracy, fast convergence and the capability to construct nonlinear decision boundaries) are achieved by our complex-valued ONC compared to its real-valued counterpart.

Circular Complex-Valued GMDH-Type Neural Network for Real-Valued Classification Problems

Recently, applications of complex-valued neural networks (CVNNs) to real-valued classification problems have attracted significant attention. However, most existing CVNNs are black-box models with poor explanation performance. This study extends the real-valued group method of data handling (RGMDH)-type neural network to the complex field and constructs a circular complex-valued group method of data handling (C-CGMDH)-type neural network, which is a white-box model. First, a complex least squares method is proposed for parameter estimation. Second, a new complex-valued symmetric regularity criterion is constructed with a logarithmic function to represent explicitly the magnitude and phase of the actual and predicted complex output to evaluate and select the middle candidate models. Furthermore, the property of this new complex-valued external criterion is proven to be similar to that of the real external criterion. Before training this model, a circular transformation is used to transform the real-valued input features to the complex field. Twenty-five real-valued classification data sets from the UCI Machine Learning Repository are used to conduct the experiments. The results show that both RGMDH and C-CGMDH models can select the most important features from the complete feature space through a self-organizing modeling process. Compared with RGMDH, the C-CGMDH model converges faster and selects fewer features. Furthermore, its classification performance is statistically significantly better than the benchmark complex-valued and real-valued models. Regarding time complexity, the C-CGMDH model is comparable with other models in dealing with the data sets that have few features. Finally, we demonstrate that the GMDH-type neural network can be interpretable.

State-Feedback Filtering for Delayed Discrete-Time Complex-Valued Neural Networks

This article explores a new filtering problem for the class of delayed discrete-time complex-valued neural networks (CVNNs) via state-feedback control design. The novelty of this article comes from the consideration of the newly developed complex-valued reciprocal convex matrix inequality as well as the complex-valued Jensen-based summation inequalities (JSIs). By employing an appropriate Lyapunov-Krasovskii functional (LKF) and by using newly proposed complex-valued inequalities, attention is concentrated on the design of a state-feedback filter such that the associated filtering error system is asymptotically stable with prescribed filter and control gain matrices. The proposed theoretical results are presented in terms of complex-valued linear matrix inequalities (LMIs) that can be solved numerically by using the YALMIP toolbox in MATLAB software. Additionally, one numerical example is given to confirm the validity of the resulting sufficient conditions with the availability of the suitable control and filter design.

Millimeter-wave Adaptive Glucose Concentration Estimation with Complex-Valued Neural Networks

In this paper, we propose an adaptive glucose concentration estimation system. The system estimates glucose concentration values non-invasively by making full use of transmission magnitude and phase data. Debye relaxation model indicates that, in millimeter wave frequency range, we can acquire both a high sensitivity and a sufficient penetration depth. Based on the physical model, we choose 60-80 GHz frequency band millimeter wave. We build a single output-neuron complex-valued neural network (CVNN) for adaptive concentration estimation. Glucose water solution samples ranging from 0 to 300 mg/dL are measured. Transmission magnitude and phase data with teacher signals are fed to a CVNN for training and validation. The change in the glucose concentration presents a variation of both transmission magnitude and phase. The CVNN learns the relationship between the transmission data and the glucose concentrations. We find that the system shows a good generalization ability to estimate the concentration for unknown samples. It is effective in the estimation of the glucose concentration in the clinically practical range. Non-invasive methods usually suffer from instability in measurement condition. Our proposed method has the adaptability to different measurement conditions through the learning process based on a set of sample transmission magnitude and phase data with corresponding teacher signals.

Singularities of Three-Layered Complex-Valued Neural Networks With Split Activation Function

There are three important concepts related to learning processes in neural networks: reducibility, nonminimality, and singularity. Although the definitions of these three concepts differ, they are equivalent in real-valued neural networks. This is also true of complex-valued neural networks (CVNNs) with hidden neurons not employing biases. The situation of CVNNs with hidden neurons employing biases, however, is very complicated. Exceptional reducibility was found, and it was shown that reducibility and nonminimality are not the same. Irreducibility consists of minimality and exceptional reducibility. The relationship between minimality and singularity has not yet been established. In this paper, we describe our surprising finding that minimality and singularity are independent. We also provide several examples based on exceptional reducibility.

Complex Neural Network Models for Time-Varying Drazin Inverse

Two complex Zhang neural network (ZNN) models for computing the Drazin inverse of arbitrary time-varying complex square matrix are presented. The design of these neural networks is based on corresponding matrix-valued error functions arising from the limit representations of the Drazin inverse. Two types of activation functions, appropriate for handling complex matrices, are exploited to develop each of these networks. Theoretical results of convergence analysis are presented to show the desirable properties of the proposed complex-valued ZNN models. Numerical results further demonstrate the effectiveness of the proposed models.