An Extreme Learning Machine-Based Neuromorphic Tactile Sensing System for Texture Recognition

Fibrosis and inflammatory activity diagnosis of chronic hepatitis C based on extreme learning machine

The traditional diagnosis of chronic hepatitis C usually relies on liver biopsy. Diagnosing chronic hepatitis C based on serum indices provides a non-invasive way to determine the stage of chronic hepatitis C without liver biopsy. In this paper, we proposed two automatic diagnosis systems for non-invasive diagnosis of chronic hepatitis C based on serum indices, an extreme learning machine (ELM) based auto-diagnosis method and a hybrid method using k-means clustering and ELM. The two proposed systems were used to predict the fibrosis stage and inflammatory activity grade of patients with chronic hepatitis C by analyzing their serum index observations. ELM has superiorities such as simple structure and fast calculation speed and can provide good diagnosis performance. To overcome the problem of class-imbalance, outliers and small sample size, we also proposed a method hybridizing k-means and ELM. It employed the k-means clustering to generate new robust training samples and then employed the new generated training samples to train an ELM for chronic hepatitis C diagnosis. The proposed methods were tested on 123 real clinical cases. Experimental results show that the proposed methods outperform the state-of-the-art methods for the fibrosis stage and inflammatory activity grade diagnosis tasks.

A Fingertip-Mimicking 1216 200 m-Resolution e-Skin Taxel Readout Chip With Per-Taxel Spiking Readout and Embedded Receptive Field Processing

This paper presents an electronic skin (e-skin) taxel array readout chip in 0.18m CMOS technology, achieving the highest reported spatial resolution of 200m, comparable to human fingertips. A key innovation is the integration on chip of a 1216 polyvinylidene fluoride (PVDF)-based piezoelectric sensor array with per-taxel signal conditioning frontend and spiking readout combined with local embedded neuromorphic first-order processing through Complex Receptive Fields (CRFs). Experimental results show that Spiking Neural Network (SNN)-based classification of the chip's spatiotemporal spiking output for input tactile stimuli such as texture and flutter frequency achieves excellent accuracies up to 97.1 and 99.2, respectively. SNN-based classification of the indentation period applied to the on-chip PVDF sensors achieved 95.5 classification accuracy, despite using only a small 256-neuron SNN classifier, a low equivalent spike encoding resolution of 3-5 bits, and a sub-Nyquist 2.2kevent/s population spiking rate, a state-of-the-art power consumption of 12.33nW per-taxel, and 75W-5mW for the entire chip is obtained. Finally, a comparison of the texture classification accuracies between two on-chip spike encoder outputs shows that the proposed neuromorphic level-crossing sampling (N-LCS) architecture with a decaying threshold outperforms the conventional bipolar level-crossing sampling (LCS) architecture with fixed threshold.

Tactile-GAT: tactile graph attention networks for robot tactile perception classification

As one of the most important senses in human beings, touch can also help robots better perceive and adapt to complex environmental information, improving their autonomous decision-making and execution capabilities. Compared to other perception methods, tactile perception needs to handle multi-channel tactile signals simultaneously, such as pressure, bending, temperature, and humidity. However, directly transferring deep learning algorithms that work well on temporal signals to tactile signal tasks does not effectively utilize the physical spatial connectivity information of tactile sensors. In this paper, we propose a tactile perception framework based on graph attention networks, which incorporates explicit and latent relation graphs. This framework can effectively utilize the structural information between different tactile signal channels. We constructed a tactile glove and collected a dataset of pressure and bending tactile signals during grasping and holding objects, and our method achieved 89.58% accuracy in object tactile signal classification. Compared to existing time-series signal classification algorithms, our graph-based tactile perception algorithm can better utilize and learn sensor spatial information, making it more suitable for processing multi-channel tactile data. Our method can serve as a general strategy to improve a robot's tactile perception capabilities.

A Systematic Review of AI-Driven Prediction of Fabric Properties and Handfeel

Artificial intelligence (AI) is revolutionizing the textile industry by improving the prediction of fabric properties and handfeel, which are essential for assessing textile quality and performance. However, the practical application and translation of AI-predicted results into real-world textile production remain unclear, posing challenges for widespread adoption. This paper systematically reviews AI-driven techniques for predicting these characteristics by focusing on model mechanisms, dataset diversity, and prediction accuracy. Among 899 papers initially identified, 39 were selected for in-depth analysis through both bibliometric and content analysis. The review categorizes and evaluates various AI approaches, including machine learning, deep learning, and hybrid models, across different types of fabric. Despite significant advances, challenges remain, such as ensuring model generalization and managing complex fabric behavior. Future research should focus on developing more robust models, integrating sustainability, and refining feature extraction techniques. This review highlights the critical gaps in the literature and provides practical insights to enhance AI-driven prediction of fabric properties, thus guiding future textile innovations.

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

A Butterfly-Inspired Multisensory Neuromorphic Platform for Integration of Visual and Chemical Cues

Unisensory cues are often insufficient for animals to effectively engage in foraging, mating, and predatory activities. In contrast, integration of cues collected from multiple sensory organs enhances the overall perceptual experience and thereby facilitates better decision-making. Despite the importance of multisensory integration in animals, the field of artificial intelligence (AI) and neuromorphic computing has primarily focused on processing unisensory information. This lack of emphasis on multisensory integration can be attributed to the absence of a miniaturized hardware platform capable of co-locating multiple sensing modalities and enabling in-sensor and near-sensor processing. In this study, this limitation is addressed by utilizing the chemo-sensing properties of graphene and the photo-sensing capability of monolayer molybdenum disulfide (MoS2 ) to create a multisensory platform for visuochemical integration. Additionally, the in-memory-compute capability of MoS2 memtransistors is leveraged to develop neural circuits that facilitate multisensory decision-making. The visuochemical integration platform is inspired by intricate courtship of Heliconius butterflies, where female species rely on the integration of visual cues (such as wing color) and chemical cues (such as pheromones) generated by the male butterflies for mate selection. The butterfly-inspired visuochemical integration platform has significant implications in both robotics and the advancement of neuromorphic computing, going beyond unisensory intelligence and information processing.

An approximate randomization-based neural network with dedicated digital architecture for energy-constrained devices

Variable energy constraints affect the implementations of neural networks on battery-operated embedded systems. This paper describes a learning algorithm for randomization-based neural networks with hard-limit activation functions. The approach adopts a novel cost function that balances accuracy and network complexity during training. From an energy-specific perspective, the new learning strategy allows to adjust, dynamically and in real time, the number of operations during the network’s forward phase. The proposed learning scheme leads to efficient predictors supported by digital architectures. The resulting digital architecture can switch to approximate computing at run time, in compliance with the available energy budget. Experiments on 10 real-world prediction testbeds confirmed the effectiveness of the learning scheme. Additional tests on limited-resource devices supported the implementation efficiency of the overall design approach.

Texture Identification and Object Recognition Using a Soft Robotic Hand Innervated Bio-Inspired Proprioception

In this study, we innervated bio-inspired proprioception into a soft hand, facilitating a robust perception of textures and object shapes. The tendon-driven soft finger with three joints, inspired by the human finger, was detailed. With tension sensors embedded in the tendon that simulate the Golgi tendon organ of the human body, 17 types of textures can be identified under uncertain rotation angles and actuator displacements. Four classifiers were used and the highest identification accuracy was 98.3%. A three-fingered soft hand based on the bionic finger was developed. Its basic grasp capability was tested experimentally. The soft hand can distinguish 10 types of objects that vary in shape with top grasp and side grasp, with the highest accuracies of 96.33% and 96.00%, respectively. Additionally, for six objects with close shapes, the soft hand obtained an identification accuracy of 97.69% with a scan-grasp method. This study offers a novel bionic solution for the texture identification and object recognition of soft manipulators.

Multi-IMF Sample Entropy Features with Machine Learning for Surface Texture Recognition Based on Robot Tactile Perception

Discrimination of surface textures using tactile sensors has attracted increasing attention. Intelligent robotics with the ability to recognize and discriminate the surface textures of grasped objects are crucial. In this paper, a novel method for surface texture classification based on tactile signals is proposed. For the proposed method, first, the tactile signals of each channel (X, Y, Z, and S) are decomposed based on empirical mode decomposition (EMD). Then, the intrinsic mode functions (IMFs) are obtained. Second, based on the multiple IMFs, the sample entropy is calculated for each IMF. Therefore, the multi-IMF sample entropy (MISE) features are obtained. Last but not least, based on the two public datasets, a variety of machine learning algorithms are used to recognize different textures. The results show that the SVM classification method, with the proposed MISE features, achieves the highest classification accuracy. Undeniably, the MISE features with the SVM method, proposed in this paper, provide a novel idea for the recognition of surface texture based on tactile perception.

Texture Discrimination with a Soft Biomimetic Finger Using a Flexible Neuromorphic Tactile Sensor Array That Provides Sensory Feedback

The compliant nature of soft fingers allows for safe and dexterous manipulation of objects by humans in an unstructured environment. A soft prosthetic finger design with tactile sensing capabilities for texture discrimination and subsequent sensory stimulation has the potential to create a more natural experience for an amputee. In this work, a pneumatically actuated soft biomimetic finger is integrated with a textile neuromorphic tactile sensor array for a texture discrimination task. The tactile sensor outputs were converted into neuromorphic spike trains, which emulate the firing pattern of biological mechanoreceptors. Spike-based features from each taxel compressed the information and were then used as inputs for the support vector machine classifier to differentiate the textures. Our soft biomimetic finger with neuromorphic encoding was able to achieve an average overall classification accuracy of 99.57% over 16 independent parameters when tested on 13 standardized textured surfaces. The 16 parameters were the combination of 4 angles of flexion of the soft finger and 4 speeds of palpation. To aid in the perception of more natural objects and their manipulation, subjects were provided with transcutaneous electrical nerve stimulation to convey a subset of four textures with varied textural information. Three able-bodied subjects successfully distinguished two or three textures with the applied stimuli. This work paves the way for a more human-like prosthesis through a soft biomimetic finger with texture discrimination capabilities using neuromorphic techniques that provide sensory feedback; furthermore, texture feedback has the potential to enhance user experience when interacting with their surroundings.

Embodied tactile perception and learning

Various living creatures exhibit embodiment intelligence, which is reflected by a collaborative interaction of the brain, body, and environment. The actual behavior of embodiment intelligence is generated by a continuous and dynamic interaction between a subject and the environment through information perception and physical manipulation. The physical interaction between a robot and the environment is the basis for realizing embodied perception and learning. Tactile information plays a critical role in this physical interaction process. It can be used to ensure safety, stability, and compliance, and can provide unique information that is difficult to capture using other perception modalities. However, due to the limitations of existing sensors and perception and learning methods, the development of robotic tactile research lags significantly behind other sensing modalities, such as vision and hearing, thereby seriously restricting the development of robotic embodiment intelligence. This paper presents the current challenges related to robotic tactile embodiment intelligence and reviews the theory and methods of robotic embodied tactile intelligence. Tactile perception and learning methods for embodiment intelligence can be designed based on the development of new large‐scale tactile array sensing devices, with the aim to make breakthroughs in the neuromorphic computing technology of tactile intelligence.

Self healable neuromorphic memtransistor elements for decentralized sensory signal processing in robotics

Sensory information processing in robot skins currently rely on a centralized approach where signal transduction (on the body) is separated from centralized computation and decision-making, requiring the transfer of large amounts of data from periphery to central processors, at the cost of wiring, latency, fault tolerance and robustness. We envision a decentralized approach where intelligence is embedded in the sensing nodes, using a unique neuromorphic methodology to extract relevant information in robotic skins. Here we specifically address pain perception and the association of nociception with tactile perception to trigger the escape reflex in a sensorized robotic arm. The proposed system comprises self-healable materials and memtransistors as enabling technologies for the implementation of neuromorphic nociceptors, spiking local associative learning and communication. Configuring memtransistors as gated-threshold and -memristive switches, the demonstrated system features in-memory edge computing with minimal hardware circuitry and wiring, and enhanced fault tolerance and robustness.

Neuromorphic Processing of Pressure Signal Using Integrated Sensor-Synaptic Device Capable of Selective and Reversible Short- and Long-Term Plasticity Operation

To mimic the tactile sensing properties of the human skin, signals from tactile sensors need to be processed in an efficient manner. The integration of the tactile sensor with a neuromorphic device can potentially address this issue, as the neuromorphic device has both signal processing and memory capability through which parallel and efficient processing of information is possible. In this article, an intelligent haptic perception device (IHPD) is presented that combines pressure sensing with an organic electrochemical transistor-based synaptic device into a simple device architecture. More importantly, the IHPD is capable of rapid and reversible switching between short-term plasticity (STP) and long-term plasticity (LTP) operation through which accelerated learning, processing of new information, and distinctive operation of STP and LTP are possible. Various types of pressure information such as magnitude, rate, and duration were processed utilizing STP by which error-tolerant perception was demonstrated. Meanwhile, memorization and learning of pressure through a stepwise change in a conductive state was demonstrated using LTP. These demonstrations present unique approaches to process and learn tactile information, which can potentially be utilized in various electronic skin applications in the future.

A Survey on Industrial Information Integration 2016–2019

Industrial information integration engineering (IIIE) is a set of foundational concepts and techniques that facilitate the industrial information integration process. In recent years, many applications of the integration between Internet of Things (IoT) and IIIE have become available, including industrial Internet of Things (IIoT), cyber-physical systems, smart grids, and smart manufacturing. In order to investigate the latest achievements of studies on IIIE, this paper reviews literatures from 2016 to 2019 in IEEEXplore and Web of Science. Altogether, 970 papers related to IIIE are grouped into 27 research categories and reviewed. The results present up-to-date development of IIIE and provide directions for future research on IIIE.

Soft eSkin: distributed touch sensing with harmonized energy and computing

Inspired by biology, significant advances have been made in the field of electronic skin (eSkin) or tactile skin. Many of these advances have come through mimicking the morphology of human skin and by distributing few touch sensors in an area. However, the complexity of human skin goes beyond mimicking few morphological features or using few sensors. For example, embedded computing (e.g. processing of tactile data at the point of contact) is centric to the human skin as some neuroscience studies show. Likewise, distributed cell or molecular energy is a key feature of human skin. The eSkin with such features, along with distributed and embedded sensors/electronics on soft substrates, is an interesting topic to explore. These features also make eSkin significantly different from conventional computing. For example, unlike conventional centralized computing enabled by miniaturized chips, the eSkin could be seen as a flexible and wearable large area computer with distributed sensors and harmonized energy. This paper discusses these advanced features in eSkin, particularly the distributed sensing harmoniously integrated with energy harvesters, storage devices and distributed computing to read and locally process the tactile sensory data. Rapid advances in neuromorphic hardware, flexible energy generation, energy-conscious electronics, flexible and printed electronics are also discussed. This article is part of the theme issue 'Harmonizing energy-autonomous computing and intelligence'.

Texture Discrimination using a Soft Biomimetic Finger for Prosthetic Applications

Soft robotic fingers have shown great potential for use in prostheses due to their inherent compliant, light, and dexterous nature. Recent advancements in sensor technology for soft robotic systems showcase their ability to perceive and respond to static cues. However, most of the soft fingers for use in prosthetic applications are not equipped with sensors which have the ability to perceive texture like humans can. In this work, we present a dexterous, soft, biomimetic solution which is capable of discrimination of textures. We fabricated a soft finger with two individually controllable degrees of freedom with a tactile sensor embedded at the fingertip. The output of the tac- tile sensor, as texture plates were palpated, was converted into spikes, mimicking the behavior of a biological mechanoreceptor. We explored the spatial properties of the textures captured in the form of spiking patterns by generating spatial event plots and analyzing the similarity between spike trains generated for each texture. Unique features representative of the different textures were then extracted from the spikes and input to a classifier. The textures were successfully classified with an accuracy of 94% when palpating at a rate of 42 mm/s. This work demonstrates the potential of providing amputees with a soft finger with sensing capabilities, which could potentially help discriminate between different objects and surfaces during activities of daily living (ADL) through palpation.

Memristive Circuit Design of Emotional Generation and Evolution Based on Skin-Like Sensory Processor

Sensory processor in human skin is used for processing and transmitting sensations to the brain, which leads to body actions and emotional responses. In this paper, a memristive circuit of emotional generation and evolution based on skin-like sensory processor is proposed. The circuit includes: first, memristive skin-like sensory processor module; second, emotional generation and evolution module; and third, emotional expression module. The first module consists of four single-memristor skin-like sensory processors, which correspond to process sensations of pain, cold, warm, and tactile. It will automatically return to its initial state if sensory signals disappear. But if sensory signals are much strong, it will not automatically return to initial state unless applied "restoring signal" just like a surgical operation. The second module realizes a conversion mechanism from sensations to emotions using memristor as emotional synapse. Given signals from skin-like sensory processor, the memristance will decrease, which means the extent of emotion will increase, such as more happy. This is the emotional generation. The extent of emotion will be changed if the same sensation is applied to skin-like sensory processor repeatedly, which is the emotional evolution. The third module can show the generated emotions visually. The simulation results in PSPICE show that the proposed circuit can generate and evolve emotions like human beings after processing sensory signals from skin. The proposed circuit can be applied in a perceptual robot platform to realize the conversion from sensations to emotions, enabling the robot to have the ability to sense and process information.