An Efficient Memristor-Based Circuit Implementation of Squeeze-and-Excitation Fully Convolutional Neural Networks

Study on real-time warning system of blind path for the visually impaired based on improved deep residual shrinkage network

Visually impaired individuals often face various obstacles when navigating blind roads, such as road disconnections, obstructions, and more complex road emergencies, which can leave them in difficult situations. Traditional early warning methods suffer from low accuracy and lack real-time warning capabilities. Therefore, this study proposes a novel real-time warning system for traffic jams on blind roads. By analyzing the emotional state (normal, mild anxiety, extreme anxiety) from the electroencephalogram (EEG) signals of visually impaired individuals when they are trapped, the system can determine whether they are in distress and require assistance. Additionally, considering the complexity of the road environment and the fact that EEG signals are prone to external interference during acquisition, this study introduces an improved deep residual shrinkage network based on dense blocks (DB-DRSN). DB-DRSN replaces the convolutional hidden layer in the original residual shrinkage module with dense blocks and integrates dense connections to optimize the use of both shallow and deep features. The results show that the system achieves an accuracy of 96.72% in recognizing the difficulties faced by the visually impaired, significantly outperforming traditional models. Compared to other warning methods, the proposed system offers quicker assistance to visually impaired individuals. The real-time warning system based on DB-DRSN demonstrated strong performance in detecting and warning about blind road jams, greatly enhancing the safety of visually impaired individuals and enabling timely detection and intervention.

Latent representation learning for classification of the Doppler ultrasound images

The classification of Doppler ultrasound images plays an important role in the diagnosis of pregnancy. However, it is a challenging problem that suffers from a variable length of these images with a dimension gap between them. In this study, we propose a latent representation weights learning method (LRWL) for pregnancy prediction using Doppler ultrasound images. Unlike most existing methods, LRWL can handle a variable length of multiple images, especially with an irregular multi-image issue. Furthermore, a spatial interaction measurement (SIM) method is proposed to verify the hypothesis that LRWL can more accurately capture relationships among the images. The images, along with diagnostic indices and weights, are integrated as inputs to a deep learning (DL) model for pregnancy prediction. The study conducts comprehensive experiments involving classification tasks on real irregular reproduction datasets and two synthetic regular datasets. Results demonstrate that LRWL surpasses existing methods and is well-suited for irregular multi-image datasets. The proposed method can be effectively implemented using the limited-memory Broyden-Fletcher-Goldfarb-Shanno bound constraint (L-BFGS-B) and the alternating direction minimization (ADM) framework, exhibiting strong performance in terms of accuracy and convergence.

Memristor-Based Neural Network Circuit of Associative Memory With Overshadowing and Emotion Congruent Effect

Most memristor-based neural network circuits consider only a single pattern of overshadowing or emotion, but the relationship between overshadowing and emotion is ignored. In this article, a memristor-based neural network circuit of associative memory with overshadowing and emotion congruent effect is designed, and overshadowing under multiple emotions is taken into account. The designed circuit mainly consists of an emotion module, a memory module, an inhibition module, and a feedback module. The generation and recovery of different emotions are realized by the emotion module. The functions of overshadowing under different emotions and recovery from overshadowing are achieved by the inhibition module and the memory module. Finally, the blocking caused by long-term overshadowing is implemented by the feedback module. The proposed circuit can be applied to bionic emotional robots and offers some references for brain-like systems.

AutoGMap: Learning to Map Large-Scale Sparse Graphs on Memristive Crossbars

The sparse representation of graphs has shown great potential for accelerating the computation of graph applications (e.g., social networks and knowledge graphs) on traditional computing architectures (CPU, GPU, or TPU). But, the exploration of large-scale sparse graph computing on processing-in-memory (PIM) platforms (typically with memristive crossbars) is still in its infancy. To implement the computation or storage of large-scale or batch graphs on memristive crossbars, a natural assumption is that a large-scale crossbar is demanded, but with low utilization. Some recent works question this assumption; to avoid the waste of storage and computational resource, the fixed-size or progressively scheduled "block partition" schemes are proposed. However, these methods are coarse-grained or static and are not effectively sparsity-aware. This work proposes the dynamic sparsity-aware mapping scheme generating method that models the problem with a sequential decision-making model, and optimizes it by reinforcement learning (RL) algorithm (REINFORCE). Our generating model [long short-term memory (LSTM), combined with the dynamic-fill scheme] generates remarkable mapping performance on the small-scale graph/matrix data (complete mapping costs 43% area of the original matrix) and two large-scale matrix data (costing 22.5% area on qh882 and 17.1% area on qh1484). Our method may be extended to sparse graph computing on other PIM architectures, not limited to the memristive device-based platforms.

Soft Memristor at a Microbubble Interface

Memristors show promising features for neuromorphic computing. Here we report a soft memristor based on the liquid-vapor surface of a microbubble. The thickness of the liquid film was modulated by electrostatic and interfacial forces, enabling resistance switches. We found a pinched current hysteresis at scanning periods between 1.6 and 51.2 s, while representing a resistor below 1.6 s and a diode-like behavior above 51.2 s. We approximate the thickening/thinning dynamics of liquid film by pressure-driven flow at the interface and derived the impacts of salt concentration and voltage amplitude on the memory effects. Our work opens a new approach to building nanofluidic memristors by a soft interface, which may be useful for new types of neuromorphic computing in the future.

Lung image segmentation via generative adversarial networks

Introduction

Lung image segmentation plays an important role in computer-aid pulmonary disease diagnosis and treatment.

Methods

This paper explores the lung CT image segmentation method by generative adversarial networks. We employ a variety of generative adversarial networks and used their capability of image translation to perform image segmentation. The generative adversarial network is employed to translate the original lung image into the segmented image.

Results

The generative adversarial networks-based segmentation method is tested on real lung image data set. Experimental results show that the proposed method outperforms the state-of-the-art method.

Discussion

The generative adversarial networks-based method is effective for lung image segmentation.

Toward Intelligent Display with Neuromorphic Technology

In the era of the Internet and the Internet of Things, display technology has evolved significantly toward full-scene display and realistic display. Incorporating "intelligence" into displays is a crucial technical approach to meet the demands of this development. Traditional display technology relies on distributed hardware systems to achieve intelligent displays but encounters challenges stemming from the physical separation of sensing, processing, and light-emitting modules. The high energy consumption and data transformation delays limited the development of intelligence display, breaking the physical separation is crucial to overcoming the bottlenecks of intelligence display technology. Inspired by the biological neural system, neuromorphic technology with all-in-one features is widely employed across various fields. It proves effective in reducing system power consumption, facilitating frequent data transformation, and enabling cross-scene integration. Neuromorphic technology shows great potential to overcome display technology bottlenecks, realizing the full-scene display and realistic display with high efficiency and low power consumption. This review offers a comprehensive summary of recent advancements in the application of neuromorphic technology in displays, with a focus on interoperability. This work delves into its state-of-the-art designs and potential future developments aimed at revolutionizing display technology.

Pruning and quantization algorithm with applications in memristor-based convolutional neural network

The human brain's ultra-low power consumption and highly parallel computational capabilities can be accomplished by memristor-based convolutional neural networks. However, with the rapid development of memristor-based convolutional neural networks in various fields, more complex applications and heavier computations lead to the need for a large number of memristors, which makes power consumption increase significantly and the network model larger. To mitigate this problem, this paper proposes an SBT-memristor-based convolutional neural network architecture and a hybrid optimization method combining pruning and quantization. Firstly, SBT-memristor-based convolutional neural network is constructed by using the good thresholding property of the SBT memristor. The memristive in-memory computing unit, activation unit and max-pooling unit are designed. Then, the hybrid optimization method combining pruning and quantization is used to improve the SBT-memristor-based convolutional neural network architecture. This hybrid method can simplify the memristor-based neural network and represent the weights at the memristive synapses better. Finally, the results show that the SBT-memristor-based convolutional neural network reduces a large number of memristors, decreases the power consumption and compresses the network model at the expense of a little precision loss. The SBT-memristor-based convolutional neural network obtains faster recognition speed and lower power consumption in MNIST recognition. It provides new insights for the complex application of convolutional neural networks.

Performance estimation for the memristor-based computing-in-memory implementation of extremely factorized network for real-time and low-power semantic segmentation

Nowadays many semantic segmentation algorithms have achieved satisfactory accuracy on von Neumann platforms (e.g., GPU), but the speed and energy consumption have not meet the high requirements of certain edge applications like autonomous driving. To tackle this issue, it is of necessity to design an efficient lightweight semantic segmentation algorithm and then implement it on emerging hardware platforms with high speed and energy efficiency. Here, we first propose an extremely factorized network (EFNet) which can learn multi-scale context information while preserving rich spatial information with reduced model complexity. Experimental results on the Cityscapes dataset show that EFNet achieves an accuracy of 68.0% mean intersection over union (mIoU) with only 0.18M parameters, at a speed of 99 frames per second (FPS) on a single RTX 3090 GPU. Then, to further improve the speed and energy efficiency, we design a memristor-based computing-in-memory (CIM) accelerator for the hardware implementation of EFNet. It is shown by the simulation in DNN+NeuroSim V2.0 that the memristor-based CIM accelerator is ∼63× (∼4.6×) smaller in area, at most ∼9.2× (∼1000×) faster, and ∼470× (∼2400×) more energy-efficient than the RTX 3090 GPU (the Jetson Nano embedded development board), although its accuracy slightly decreases by 1.7% mIoU. Therefore, the memristor-based CIM accelerator has great potential to be deployed at the edge to implement lightweight semantic segmentation models like EFNet. This study showcases an algorithm-hardware co-design to realize real-time and low-power semantic segmentation at the edge.

A Rare Hernia Mimicking Implant in a Patient with Rectal Adenocarcinoma: Internal Herniation

Internal herniation may be seen more frequently in patients with intra-abdominal surgery and malignancy history. We presented a 58-year-old male patient diagnosed with rectal adenocarcinoma seven years ago with a history of surgery and pelvic radiotherapy. When the abdominal computed tomography (CT) image was taken during routine oncology follow-up, a lesion mimicking a serosal implant on the anterior abdominal wall was detected. ¹⁸F-fluorodeoxyglucose (FDG) positron emission tomography (PET)/CT imaging was performed the suspicion of recurrence. It was concluded that the lesion, which was evaluated as an implant in abdominal CT with ¹⁸F-FDG PET/CT imaging, was a spontaneously reducing internal herniation. ¹⁸F-FDG PET/CT imaging in cancer patients is crucial in illuminating the suspicion of recurrent lesions in these patients and sheds light on the course of the patients in oncology practice.

Convolutional Neural Network Based on Crossbar Arrays of (Co-Fe-B)x(LiNbO3)100-x Nanocomposite Memristors

Convolutional neural networks (CNNs) have been widely used in image recognition and processing tasks. Memristor-based CNNs accumulate the advantages of emerging memristive devices, such as nanometer critical dimensions, low power consumption, and functional similarity to biological synapses. Most studies on memristor-based CNNs use either software models of memristors for simulation analysis or full hardware CNN realization. Here, we propose a hybrid CNN, consisting of a hardware fixed pre-trained and explainable feature extractor and a trainable software classifier. The hardware part was realized on passive crossbar arrays of memristors based on nanocomposite (Co-Fe-B)_x(LiNbO₃)_100-x structures. The constructed 2-kernel CNN was able to classify the binarized Fashion-MNIST dataset with ~ 84% accuracy. The performance of the hybrid CNN is comparable to the other reported memristor-based systems, while the number of trainable parameters for the hybrid CNN is substantially lower. Moreover, the hybrid CNN is robust to the variations in the memristive characteristics: dispersion of 20% leads to only a 3% accuracy decrease. The obtained results pave the way for the efficient and reliable realization of neural networks based on partially unreliable analog elements.

Segmentation of liver tumors with abdominal computed tomography using fully convolutional networks

BACKGROUND

Dividing liver organs or lesions depicting on computed tomography (CT) images could be applied to help tumor staging and treatment. However, most existing image segmentation technologies use manual or semi-automatic analysis, making the analysis process costly and time-consuming.

OBJECTIVE

This research aims to develop and apply a deep learning network architecture to segment liver tumors automatically after fine tuning parameters.

METHODS AND MATERIALS

The medical imaging is obtained from the International Symposium on Biomedical Imaging (ISBI), which includes 3D abdominal CT scans of 131 patients diagnosed with liver tumors. From these CT scans, there are 7,190 2D CT images along with the labeled binary images. The labeled binary images are regarded as gold standard for evaluation of the segmented results by FCN (Fully Convolutional Network). The backbones of FCN are extracted from Xception, InceptionresNetv2, MobileNetv2, ResNet18, ResNet50 in this study. Meanwhile, the parameters including optimizers (SGDM and ADAM), size of epoch, and size of batch are investigated. CT images are randomly divided into training and testing sets using a ratio of 9:1. Several evaluation indices including Global Accuracy, Mean Accuracy, Mean IoU (Intersection over Union), Weighted IoU and Mean BF Score are applied to evaluate tumor segmentation results in the testing images.

RESULTS

The Global Accuracy, Mean Accuracy, Mean IoU, Weighted IoU, and Mean BF Scores are 0.999, 0.969, 0.954, 0.998, 0.962 using ResNet50 in FCN with optimizer SGDM, batch size 12, and epoch 9. It is important to fine tuning the parameters in FCN model. Top 20 FNC models enable to achieve higher tumor segmentation accuracy with Mean IoU over 0.900. The occurred frequency of InceptionresNetv2, MobileNetv2, ResNet18, ResNet50, and Xception are 9, 6, 3, 5, and 2 times. Therefore, the InceptionresNetv2 has higher performance than others.

CONCLUSIONS

This study develop and test an automated liver tumor segmentation model based on FCN. Study results demonstrate that many deep learning models including InceptionresNetv2, MobileNetv2, ResNet18, ResNet50, and Xception have high potential to segment liver tumors from CT images with accuracy exceeding 90%. However, it is still difficult to accurately segment tiny and small size tumors by FCN models.

Single-Channel Blind Source Separation of Spatial Aliasing Signal Based on Stacked-LSTM

Aiming at the problem of insufficient separation accuracy of aliased signals in space Internet satellite-ground communication scenarios, a stacked long short-term memory network (Stacked-LSTM) separation method based on deep learning is proposed. First, the coding feature representation of the mixed signal is extracted. Then, the long sequence input is divided into smaller blocks through the Stacked-LSTM network with the attention mechanism of the SE module, and the deep feature mask of the source signal is trained to obtain the Hadamard product of the mask of each source and the coding feature of the mixed signal, which is the encoding feature representation of the source signal. Finally, characteristics of the source signal is decoded by 1-D convolution to to obtain the original waveform. The negative scale-invariant source-to-noise ratio (SISNR) is used as the loss function of network training, that is, the evaluation index of single-channel blind source separation performance. The results show that in the single-channel separation of spatially aliased signals, the Stacked-LSTM method improves SISNR by 10.09∼38.17 dB compared with the two classic separation algorithms of ICA and NMF and the three deep learning separation methods of TasNet, Conv-TasNet and Wave-U-Net. The Stacked-LSTM method has better separation accuracy and noise robustness.