Silent Speech Recognition with Strain Sensors and Deep Learning Analysis of Directional Facial Muscle Movement

Deep Learning-Assisted 3D Pressure Sensors for Control of Unmanned Aerial Vehicles

Accurately and reliably detecting and recognizing human body movements in real time, relaying appropriate commands to the machine, have substantial implications for virtual reality, remote control, and robotics applications. Nonetheless, most contemporary wearable analysis and control systems attain action recognition by setting sensor thresholds. In routine usage, the stringent trigger conditions facilitate inadvertent contact, resulting in a poorer user experience. Here, we have created a wearable intelligent gesture recognition control system utilizing a multilayer microstructure composite thin film piezoresistive sensing array and deep learning techniques. The system exhibits ultrahigh sensitivity (ranging from 0-6 kPa to 412.2 kPa-1) and rapid response times (loading at 40 ms, recovery at 30 ms). The detected gestures are classified and recognized via a convolutional neural network, achieving a recognition accuracy of 97.5%. Ultimately, the altitude control of an unmanned aerial vehicle is accomplished through wireless signal transmission and reception. To achieve the visualization of the complete gesture-controlled flight process, we developed an intuitive user interface for the real-time display of flight altitude and video surveillance. The implementation of this recognition system introduces a novel control mechanism for human-machine interaction, expands the applications of robotic technology, and offers innovative concepts and practical pathways for virtual reality.

A Flexible and Adhesive Strain Sensor Based on Deep Eutectic Solvents for Deep Learning-Assisted Signal Recognition

Flexible wearable electronic devices have garnered significant interest due to their inherent properties, serving as replacements for traditional rigid metal conductors in personal healthcare monitoring, human motion detection, and sensory skin applications. Here, we report a preparation strategy for a self-adhesive, ultrahigh stretchable DGel based on poly(acrylic acid) (PAA). The resulting DGel exhibits a high tensile strength (approximately 2.16 MPa) and an ultrahigh stretchability (approximately 5622.14%). More importantly, these meticulously designed DES gels demonstrate high signal recognition capabilities under strains ranging from 1 to 500%. DGel also shows excellent cyclic stability and durability (5000 cycles at 100% strain), exhibiting a superior electromechanical performance as a strain sensor. The ultrahigh strength of DGel is attributed to the synergistic effects of chemical and physical cross-linking within the gel. Additionally, DGel can be effortlessly assembled into wearable sensors. By integration of flexible sensing with deep learning, the fabricated touch recognition system achieves an identification accuracy of up to 99.33%. This advancement offers new insights into designing novel gels for a variety of applications, including tissue engineering, sensing, and wearable electronic devices.

Survey of Sustainable Wearable Strain Sensors Enabled by Biopolymers and Conductive Organic Polymers

The field of wearable sensors has evolved with operating devices capable of measuring biomechanics and biometrics, and detecting speech. The transduction, being the conversion of the biosignal to a measurable and quantifiable electrical signal, is governed by a conductive organic polymer. Meanwhile, the conformality of skin to the substrate is quintessential. Both the substrate and the conductive polymer must work in concert to reversibly deform with the user's movements for motion tracking. While polydimethylsiloxane shows mechanical compliance as a sensor substrate, it is of environmental interest to replace it with sustainable and degradable alternatives. As both the bulk of the weight and area of the sensor consist of the substrate, using renewable and biodegradable materials for its preparation would be an important step toward improving the lifecycle of wearable sensors. This review highlights wearable resistive sensors that are prepared from naturally occurring polymers that are both sustainable and biodegradable. Conductive polythiophenes are also presented, as well as how they are integrated into the biopolymer for sensors showing mechanical compliance with skin. This polymer is highlighted because of its structural conformality, conductivity, and processability, ensuring it fulfils the requirements for its use in sensors without adversely affecting the overall sustainability and biodegradability of resistive sensors. Different sustainable resistive sensors are also presented, and their performance is compared to conventional sensors to illustrate the successful integration of the biosourced polymers into sensors without comprising the desired elasticity and sensitivity to movement. The current state-of-the-art in sustainable resistive sensors is presented, along with knowledge of how biopolymers from different fields can be leveraged in the rational design of the next generation of sustainable sensors that can potentially be composted after their use.

Emerging Wearable Acoustic Sensing Technologies

Sound signals not only serve as the primary communication medium but also find application in fields such as medical diagnosis and fault detection. With public healthcare resources increasingly under pressure, and challenges faced by disabled individuals on a daily basis, solutions that facilitate low-cost private healthcare hold considerable promise. Acoustic methods have been widely studied because of their lower technical complexity compared to other medical solutions, as well as the high safety threshold of the human body to acoustic energy. Furthermore, with the recent development of artificial intelligence technology applied to speech recognition, speech recognition devices, and systems capable of assisting disabled individuals in interacting with scenes are constantly being updated. This review meticulously summarizes the sensing mechanisms, materials, structural design, and multidisciplinary applications of wearable acoustic devices applied to human health and human-computer interaction. Further, the advantages and disadvantages of the different approaches used in flexible acoustic devices in various fields are examined. Finally, the current challenges and a roadmap for future research are analyzed based on existing research progress to achieve more comprehensive and personalized healthcare.

Ti3C2Tx Composite Aerogels Enable Pressure Sensors for Dialect Speech Recognition Assisted by Deep Learning

Wearable pressure sensors capable of adhering comfortably to the skin hold great promise in sound detection. However, current intelligent speech assistants based on pressure sensors can only recognize standard languages, which hampers effective communication for non-standard language people. Here, we prepare an ultralight Ti3C2Tx MXene/chitosan/polyvinylidene difluoride composite aerogel with a detection range of 6.25 Pa-1200 kPa, rapid response/recovery time, and low hysteresis (13.69%). The wearable aerogel pressure sensor can detect speech information through the throat muscle vibrations without any interference, allowing for accurate recognition of six dialects (96.2% accuracy) and seven different words (96.6% accuracy) with the assistance of convolutional neural networks. This work represents a significant step forward in silent speech recognition for human-machine interaction and physiological signal monitoring.

Ultrathin Bioelectrode Array with Improved Electrochemical Performance for Electrophysiological Sensing and Modulation

To achieve high accuracy and effectiveness in sensing and modulating neural activity, efficient charge-transfer biointerfaces and a high spatiotemporal resolution are required. Ultrathin bioelectrode arrays exhibiting mechanical compliance with biological tissues offer such biointerfaces. However, their thinness often leads to a lack of mechano-electrical stability or sufficiently high electrochemical capacitance, thus deteriorating their overall performance. Here, we report ultrathin (∼115 nm) bioelectrode arrays that simultaneously enable ultraconformability, mechano-electrical stability and high electrochemical performance. These arrays show high opto-electrical conductivity (2060 S cm-1@88% transparency), mechanical stretchability (110% strain), and excellent electrochemical properties (24.5 mC cm-2 charge storage capacity and 3.5 times lower interfacial impedance than commercial electrodes). The improved mechano-electrical and electrochemical performance is attributed to the synergistic interactions within the poly(3,4-ethylenedioxythiophene) sulfonate (PEDOT:PSS)/graphene oxide (GO) interpenetrating network (PGIN), where π-π and hydrogen bonding interactions improve conductive pathways between PEDOT chains and enhance the charge-transfer mobility. This ultrathin bioelectrode is compatible with photolithography processing and provides spatiotemporally precise signal mapping capabilities for sensing and modulating neuromuscular activity. By capturing weak multichannel facial electromyography signals and applying machine learning algorithms, we achieve high accuracy in silent speech recognition. Moreover, the high transparency of the bioelectrode allows simultaneous recording of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals, facilitating dual-mode brain activity analysis with both high temporal and high spatial resolution.

Combining 2.5D deep learning and conventional features in a joint model for the early detection of sICH expansion

The study aims to investigate the potential of training efficient deep learning models by using 2.5D (2.5-Dimension) masks of sICH. Furthermore, it intends to evaluate and compare the predictive performance of a joint model incorporating four types of features with standalone 2.5D deep learning, radiomics, radiology, and clinical models for early expansion in sICH. A total of 254 sICH patients were enrolled retrospectively and divided into two groups according to whether the hematoma was enlarged or not. The 2.5D mask of sICH is constructed with the maximum axial, coronal and sagittal planes of the hematoma, which is used to train the deep learning model and extract deep learning features. Predictive models were built on clinic, radiology, radiomics and deep learning features separately and four type features jointly. The diagnostic performance of each model was measured using the receiver operating characteristic curve (AUC), Accuracy, Recall, F1 and decision curve analysis (DCA). The AUCs of the clinic model, radiology model, radiomics model, deep learning model, joint model, and nomogram model on the train set (training and Cross-validation) were 0.639, 0.682, 0.859, 0.807, 0.939, and 0.942, respectively, while the AUCs on the test set (external validation) were 0.680, 0.758, 0.802, 0.857, 0.929, and 0.926. Decision curve analysis showed that the joint model was superior to the other models and demonstrated good consistency between the predicted probability of early hematoma expansion and the actual occurrence probability. Our study demonstrates that the joint model is a more efficient and robust prediction model, as verified by multicenter data. This finding highlights the potential clinical utility of a multifactorial prediction model that integrates various data sources for prognostication in patients with intracerebral hemorrhage. The Critical Relevance Statement: Combining 2.5D deep learning features with clinic features, radiology markers, and radiomics signatures to establish a joint model enabling physicians to conduct better-individualized assessments the risk of early expansion of sICH.

Toward an AI Era: Advances in Electronic Skins

Electronic skins (e-skins) have seen intense research and rapid development in the past two decades. To mimic the capabilities of human skin, a multitude of flexible/stretchable sensors that detect physiological and environmental signals have been designed and integrated into functional systems. Recently, researchers have increasingly deployed machine learning and other artificial intelligence (AI) technologies to mimic the human neural system for the processing and analysis of sensory data collected by e-skins. Integrating AI has the potential to enable advanced applications in robotics, healthcare, and human-machine interfaces but also presents challenges such as data diversity and AI model robustness. In this review, we first summarize the functions and features of e-skins, followed by feature extraction of sensory data and different AI models. Next, we discuss the utilization of AI in the design of e-skin sensors and address the key topic of AI implementation in data processing and analysis of e-skins to accomplish a range of different tasks. Subsequently, we explore hardware-layer in-skin intelligence before concluding with an analysis of the challenges and opportunities in the various aspects of AI-enabled e-skins.

Research on predicting hematoma expansion in spontaneous intracerebral hemorrhage based on deep features of the VGG-19 network

PURPOSE

To construct a clinical noncontrastive computed tomography (NCCT) deep learning joint model for predicting early hematoma expansion (HE) after cerebral hemorrhage (sICH) and evaluate its predictive performance.

METHODS

All 254 patients with primary cerebral hemorrhage from January 2017 to December 2022 in the General Hospital of the Western Theater Command were included. According to the criteria of hematoma enlargement exceeding 33% or the volume exceeding 6 ml, the patients were divided into the HE group and the hematoma non-enlargement (NHE) group. Multiple models and the 10-fold cross-validation method were used to screen the most valuable features and model the probability of predicting HE. The area under the curve (AUC) was used to analyze the prediction efficiency of each model for HE.

RESULTS

They were randomly divided into a training set of 204 cases in an 8:2 ratio and 50 cases of the test set. The clinical imaging deep feature joint model (22 features) predicted the area under the curve of HE as follows: clinical Navie Bayes model AUC 0.779, traditional radiology logistic regression (LR) model AUC 0.818, deep learning LR model AUC 0.873, and clinical NCCT deep learning multilayer perceptron model AUC 0.921.

CONCLUSION

The combined clinical imaging deep learning model has a high predictive effect for early HE in sICH patients, which is helpful for clinical individualized assessment of the risk of early HE in sICH patients.

Decoding silent speech commands from articulatory movements through soft magnetic skin and machine learning

Silent speech interfaces have been pursued to restore spoken communication for individuals with voice disorders and to facilitate intuitive communications when acoustic-based speech communication is unreliable, inappropriate, or undesired. However, the current methodology for silent speech faces several challenges, including bulkiness, obtrusiveness, low accuracy, limited portability, and susceptibility to interferences. In this work, we present a wireless, unobtrusive, and robust silent speech interface for tracking and decoding speech-relevant movements of the temporomandibular joint. Our solution employs a single soft magnetic skin placed behind the ear for wireless and socially acceptable silent speech recognition. The developed system alleviates several concerns associated with existing interfaces based on face-worn sensors, including a large number of sensors, highly visible interfaces on the face, and obtrusive interconnections between sensors and data acquisition components. With machine learning-based signal processing techniques, good speech recognition accuracy is achieved (93.2% accuracy for phonemes, and 87.3% for a list of words from the same viseme groups). Moreover, the reported silent speech interface demonstrates robustness against noises from both ambient environments and users' daily motions. Finally, its potential in assistive technology and human-machine interactions is illustrated through two demonstrations - silent speech enabled smartphone assistants and silent speech enabled drone control.

Design of 3D Controller Using Nanocracking Structure-Based Stretchable Strain Sensor

In this study, we introduce a novel design for a three-dimensional (3D) controller, which incorporates the omni-purpose stretchable strain sensor (OPSS sensor). This sensor exhibits both remarkable sensitivity, with a gauge factor of approximately 30, and an extensive working range, accommodating strain up to 150%, thereby enabling accurate 3D motion sensing. The 3D controller is structured such that its triaxial motion can be discerned independently along the X, Y, and Z axes by quantifying the deformation of the controller through multiple OPSS sensors affixed to its surface. To ensure precise and real-time 3D motion sensing, a machine learning-based data analysis technique was implemented for the effective interpretation of the multiple sensor signals. The outcomes reveal that the resistance-based sensors successfully and accurately track the 3D controller's motion. We believe that this innovative design holds the potential to augment the performance of 3D motion sensing devices across a diverse range of applications, encompassing gaming, virtual reality, and robotics.

A Piezoelectric MEMS Speaker with a Combined Function of a Silent Alarm

To explore the versatility of speakers, a piezoelectric micro-electro-mechanical system (MEMS) speaker combining the function of a silent alarm is proposed, which mainly comprises a lead zirconate titanate (PZT) actuation layer and a rigid-flexible coupling supporting layer. Measurements performed on encapsulated prototypes mounted to an artificial ear simulator have revealed that, compared to a speaker with a rigid supporting layer, the sound pressure level (SPL) of the proposed piezoelectric MEMS speaker with a rigid-flexible coupling supporting layer is significantly higher and is especially higher by 4.1-20.1 dB in the frequency range from 20 Hz to 4.2 kHz, indicating that the rigid-flexible coupling supporting layer can improve the SPL significantly in low frequency. Moreover, the spectral distribution characteristic of its playback audio is similar to that of the commercial electromagnetic type. The device can also function as a silent alarm based on oral airflows in dangerous situations, as it performs well at recognizing words according to their unique voltage-signal characteristics, and can avoid the effects of external sound noise, body movement, long distance, and occlusion. This strategy provides inspiration for functional diversification of piezoelectric MEMS speakers.