AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove

Integrated Fibrous Iontronic Pressure Sensors with High Sensitivity and Reliability for Human Plantar Pressure and Gait Analysis

Flexible sensing systems (FSSs) designed to measure plantar pressure can deliver instantaneous feedback on human movement and posture. This feedback is crucial not only for preventing and controlling diseases associated with abnormal plantar pressures but also for optimizing athletes' postures to minimize injuries. The development of an optimal plantar pressure sensor hinges on key metrics such as a wide sensing range, high sensitivity, and long-term stability. However, the effectiveness of current flexible sensors is impeded by numerous challenges, including limitations in structural deformability, mechanical incompatibility between multifunctional layers, and instability under complex stress conditions. Addressing these limitations, we have engineered an integrated pressure sensing system with high sensitivity and reliability for human plantar pressure and gait analysis. It features a high-modulus, porous laminated ionic fiber structure with robust self-bonded interfaces, utilizing a unified polyimide material system. This system showcases a high sensitivity (156.6 kPa-1), an extensive sensing range (up to 4000 kPa), and augmented interfacial toughness and durability (over 150,000 cycles). Additionally, our FSS is capable of real-time monitoring of plantar pressure distribution across various sports activities. Leveraging deep learning, the flexible sensing system achieves a high-precision, intelligent recognition of different plantar types with a 99.8% accuracy rate. This approach provides a strategic advancement in the field of flexible pressure sensors, ensuring prolonged stability and accuracy even amidst complex pressure dynamics and providing a feasible solution for long-term gait monitoring and analysis.

Displacement-pressure biparametrically regulated softness sensory system for intraocular pressure monitoring

High intraocular pressure (IOP) is one of the high-risk pathogenic factors of glaucoma. Existing methods of IOP measurement are based on the direct interaction with the cornea. Commercial ophthalmic tonometers based on snapshot measurements are expensive, bulky, and their operation requires trained personnel. Theranostic contact lenses are easy to use, but they may block vision and cause infection. Here, we report a sensory system for IOP assessment that uses a soft indentor with two asymmetrically deployed iontronic flexible pressure sensors to interact with the eyelid-eyeball in an eye-closed situation. Inspired by human fingertip assessment of softness, the sensory system extracts displacement-pressure information for soft evaluation, achieving high accuracy IOP monitoring (>96%). We further design and custom-make a portable and wearable ophthalmic tonometer based on the sensory system and demonstrate its high efficacy in IOP screening. This sensory system paves a way towards cost-effective, robust, and reliable IOP monitoring.

Predicting Concrete Strength Early Age using a Combination of Machine Learning and Electromechanical Impedance with Nano-enhanced Sensors

To ensure the structural integrity of concrete and prevent unanticipated fracturing, real-time monitoring of early-age concrete's strength development is essential, mainly through advanced techniques such as nano-enhanced sensors. The piezoelectric-based electro-mechanical impedance (EMI) method with nano-enhanced sensors is emerging as a practical solution for such monitoring requirements. This study presents a strength estimation method based on Non-Destructive Testing (NDT) Techniques and Long Short-Term Memory (LSTM) and artificial neural networks (ANNs) as hybrid (NDT-LSTMs-ANN), including several types of concrete strength-related agents. Input data includes water-to-cement rate, temperature, curing time, and maturity based on interior temperature, allowing experimentally monitoring the development of concrete strength from the early steps of hydration and casting to the last stages of hardening 28 days after the casting. The study investigated the impact of various factors on concrete strength development, utilizing a cutting-edge approach that combines traditional models with nano-enhanced piezoelectric sensors and NDT-LSTMs-ANN enhanced with nanotechnology. The results demonstrate that the hybrid provides highly accurate concrete strength estimation for construction safety and efficiency. Adopting the piezoelectric-based EMI technique with these advanced sensors offers a viable and effective monitoring solution, presenting a significant leap forward for the construction industry's structural health monitoring practices.

Directional Moisture-Wicking Triboelectric Materials Enabled by Laplace Pressure Differences

Wearable sensors are experiencing vibrant growth in the fields of health monitoring systems and human motion detection, with comfort becoming a significant research direction for wearable sensing devices. However, the weak moisture-wicking capability of sensor materials leads to liquid retention, severely restricting the comfort of the wearable sensors. This study employs a pattern-guided alignment strategy to construct microhill arrays, endowing triboelectric materials with directional moisture-wicking capability. Within 2.25 s, triboelectric materials can quickly and directionally remove the droplets, driven by the Laplace pressure differences and the wettability gradient. The directional moisture-wicking triboelectric materials exhibit excellent pressure sensing performance, enabling rapid response/recovery (29.1/37.0 ms), thereby achieving real-time online monitoring of human respiration and movement states. This work addresses the long-standing challenge of insufficient moisture-wicking driving force in flexible electronic sensing materials, holding significant implications for enhancing the comfort and application potential of electronic skin and wearable electronic devices.

Deep Learning Model Coupling Wearable Bioelectric and Mechanical Sensors for Refined Muscle Strength Assessment

Muscle strength (MS) is related to our neural and muscle systems, essential for clinical diagnosis and rehabilitation evaluation. Although emerging wearable technology seems promising for MS assessment, problems still exist, including inaccuracy, spatiotemporal differences, and analyzing methods. In this study, we propose a wearable device consisting of myoelectric and strain sensors, synchronously acquiring surface electromyography and mechanical signals at the same spot during muscle activities, and then employ a deep learning model based on temporal convolutional network (TCN) + Transformer (Tcnformer), achieving accurate grading and prediction of MS. Moreover, by combining with deep clustering, named Tcnformer deep cluster (TDC), we further obtain a 25-level classification for MS assessment, refining the conventional 5 levels. Quantification and validation showcase a patient's postoperative recovery from level 3.2 to level 3.6 in the first few days after surgery. We anticipate that this system will importantly advance precise MS assessment, potentially improving relevant clinical diagnosis and rehabilitation outcomes.

Sign language recognition based on dual-path background erasure convolutional neural network

Sign language is an important way to provide expression information to people with hearing and speaking disabilities. Therefore, sign language recognition has always been a very important research topic. However, many sign language recognition systems currently require complex deep models and rely on expensive sensors, which limits the application scenarios of sign language recognition. To address this issue, based on computer vision, this study proposed a lightweight, dual-path background erasing deep convolutional neural network (DPCNN) model for sign language recognition. The DPCNN consists of two paths. One path is used to learn the overall features, while the other path learns the background features. The background features are gradually subtracted from the overall features to obtain an effective representation of hand features. Then, these features are flatten into a one-dimensional layer, and pass through a fully connected layer with an output unit of 128. Finally, use a fully connected layer with an output unit of 24 as the output layer. Based on the ASL Finger Spelling dataset, the total accuracy and Macro-F1 scores of the proposed method is 99.52% and 0.997, respectively. More importantly, the proposed method can be applied to small terminals, thereby improving the application scenarios of sign language recognition. Through experimental comparison, the dual path background erasure network model proposed in this paper has better generalization ability.

Multimodal Sensors Enabled Autonomous Soft Robotic System with Self-Adaptive Manipulation

Human hands are amazingly skilled at recognizing and handling objects of different sizes and shapes. To date, soft robots rarely demonstrate autonomy equivalent to that of humans for fine perception and dexterous operation. Here, an intelligent soft robotic system with autonomous operation and multimodal perception ability is developed by integrating capacitive sensors with triboelectric sensor. With distributed multiple sensors, our robot system can not only sense and memorize multimodal information but also enable an adaptive grasping method for robotic positioning and grasp control, during which the multimodal sensory information can be captured sensitively and fused at feature level for crossmodally recognizing objects, leading to a highly enhanced recognition capability. The proposed system, combining the performance and physical intelligence of biological systems (i.e., self-adaptive behavior and multimodal perception), will greatly advance the integration of soft actuators and robotics in many fields.

Haptic Sensing and Feedback Techniques toward Virtual Reality

Haptic interactions between human and machines are essential for information acquisition and object manipulation. In virtual reality (VR) system, the haptic sensing device can gather information to construct virtual elements, while the haptic feedback part can transfer feedbacks to human with virtual tactile sensation. Therefore, exploring high-performance haptic sensing and feedback interface imparts closed-loop haptic interaction to VR system. This review summarizes state-of-the-art VR-related haptic sensing and feedback techniques based on the hardware parts. For the haptic sensor, we focus on mechanism scope (piezoresistive, capacitive, piezoelectric, and triboelectric) and introduce force sensor, gesture translation, and touch identification in the functional view. In terms of the haptic feedbacks, methodologies including mechanical, electrical, and elastic actuators are surveyed. In addition, the interactive application of virtual control, immersive entertainment, and medical rehabilitation is also summarized. The challenges of virtual haptic interactions are given including the accuracy, durability, and technical conflicts of the sensing devices, bottlenecks of various feedbacks, as well as the closed-loop interaction system. Besides, the prospects are outlined in artificial intelligence of things, wise information technology of medicine, and multimedia VR areas.

Assessing the Role of Yarn Placement in Plated Knit Strain Sensors: A Detailed Study of Their Electromechanical Properties and Applicability in Bending Cycle Monitoring

In this study, we explore how the strategic positioning of conductive yarns influences the performance of plated knit strain sensors fabricated using commercial knitting machines with both conductive and non-conductive yarns. Our study reveals that sensors with conductive yarns located at the rear, referred to as 'purl plated sensors', exhibit superior performance in comparison to those with conductive yarns at the front, or 'knit plated sensors'. Specifically, purl plated sensors demonstrate a higher sensitivity, evidenced by a gauge factor ranging from 3 to 18, and a minimized strain delay, indicated by a 1% strain in their electromechanical response. To elucidate the mechanisms behind these observations, we developed an equivalent circuit model. This model examines the role of contact resistance within varying yarn configurations on the sensors' sensitivity, highlighting the critical influence of contact resistance in conductive yarns subjected to wale-wise stretching on sensor responsiveness. Furthermore, our findings illustrate that the purl plated sensors benefit from the vertical movement of non-conductive yarns, which promotes enhanced contact between adjacent conductive yarns, thereby improving both the stability and sensitivity of the sensors. The practicality of these sensors is confirmed through bending cycle tests with an in situ monitoring system, showcasing the purl plated sensors' exceptional reproducibility, with a standard deviation of 0.015 across 1000 cycles, and their superior sensitivity, making them ideal for wearable devices designed for real-time joint movement monitoring. This research highlights the critical importance of conductive yarn placement in sensor efficacy, providing valuable guidance for crafting advanced textile-based strain sensors.

Soft Sensors and Actuators for Wearable Human-Machine Interfaces

Haptic human-machine interfaces (HHMIs) combine tactile sensation and haptic feedback to allow humans to interact closely with machines and robots, providing immersive experiences and convenient lifestyles. Significant progress has been made in developing wearable sensors that accurately detect physical and electrophysiological stimuli with improved softness, functionality, reliability, and selectivity. In addition, soft actuating systems have been developed to provide high-quality haptic feedback by precisely controlling force, displacement, frequency, and spatial resolution. In this Review, we discuss the latest technological advances of soft sensors and actuators for the demonstration of wearable HHMIs. We particularly focus on highlighting material and structural approaches that enable desired sensing and feedback properties necessary for effective wearable HHMIs. Furthermore, promising practical applications of current HHMI technology in various areas such as the metaverse, robotics, and user-interactive devices are discussed in detail. Finally, this Review further concludes by discussing the outlook for next-generation HHMI technology.

Smart Printed Triboelectric Wearable Sensor with High Performance for Glove-Based Motion Detection

In a world increasingly driven by data, wearable triboelectric nanogenerators (TENGs) offer a convenient way to monitor and collect information about human body motions. To meet the demands of the large-scale production of wearable TENGs, material selection to realize a high conversion efficiency and simplify the fabrication process remains a challenge. To address these issues, we present a simple-structured wearable printed arc-shaped triboelectric sensor (PATS) for finger motion detection by leveraging inkjet printing technology. In this regard, pressure sensors composed of diverse materials based on dielectric-dielectric and metal-dielectric structures in contact-separation mode were fabricated and compared. Thanks to the unique characteristics of the silver nanoparticle (Ag-NP)-printed layer and silicon rubber (SR), the SR-Ag PATS shows a high peak-to-peak voltage of 14.15 V and a short-circuit current of 0.78 μA. The proposed sensor with the capability of accurately identifying finger motions at various bending angles suggests promising application potential in glove-based human-machine interface (HMI) systems.

Computer-Vision Based Gesture-Metasurface Interaction System for Beam Manipulation and Wireless Communication

Hand gesture plays an important role in many circumstances, which is one of the most common interactive methods in daily life, especially for disabled people. Human-machine interaction is another popular research topic to realize direct and efficient control, making machines intelligent and maneuverable. Here, a special human-machine interaction system is proposed and namedas computer-vision (CV) based gesture-metasurface interaction (GMI) system, which can be used for both direct beam manipulations and real-time wireless communications. The GMI system first needs to select its working mode according to the gesture command to determine whether to perform beam manipulations or wireless communications, and then validate the permission for further operation by recognizing unlocking gesture to ensure security. Both beam manipulation and wireless communication functions are validated experimentally, which show that the GMI system can not only realize real-time switching and remote control of different beams through gesture command, but also communicate with a remote computer in real time by translating the gesture language to text message. The proposed non-contact GMI system has the advantages of good interactivity, high flexibility, and multiple functions, which can find potential applications in community security, gesture-command smart home, barrier-free communications, and so on.

The Inheritance of Hearing Loss and Deafness: A Historical Perspective

If the term "genetics" is a relatively recent proposition, introduced in 1905 by English biologist William Bateson, who rediscovered and spread in the scientific community Mendel's principles of inheritance, since the dawn of human civilization the influence of heredity has been recognized, especially in agricultural crops and animal breeding. And, later, in familial dynasties. In this concise review, we outline the evolution of the idea of hereditary hearing loss, up to the current knowledge of molecular genetics and epigenetics.

Synergizing Machine Learning Algorithm with Triboelectric Nanogenerators for Advanced Self-Powered Sensing Systems

The advancement of the Internet of Things (IoT) has increased the demand for large-scale intelligent sensing systems. The periodic replacement of power sources for ubiquitous sensing systems leads to significant resource waste and environmental pollution. Human staffing costs associated with replacement also increase the economic burden. The triboelectric nanogenerators (TENGs) provide both an energy harvesting scheme and the possibility of self-powered sensing. Based on contact electrification from different materials, TENGs provide a rich material selection to collect complex and diverse data. As the data collected by TENGs become increasingly numerous and complex, different approaches to machine learning (ML) and deep learning (DL) algorithms have been proposed to efficiently process output signals. In this paper, the latest advances in ML algorithms assisting solid-solid TENG and liquid-solid TENG sensors are reviewed based on the sample size and complexity of the data. The pros and cons of various algorithms are analyzed and application scenarios of various TENG sensing systems are presented. The prospects of synergizing hardware (TENG sensors) with software (ML algorithms) in a complex environment and their main challenges for future developments are discussed.

American Sign Language Recognition and Translation Using Perception Neuron Wearable Inertial Motion Capture System

Sign language is designed as a natural communication method to convey messages among the deaf community. In the study of sign language recognition through wearable sensors, the data sources are limited, and the data acquisition process is complex. This research aims to collect an American sign language dataset with a wearable inertial motion capture system and realize the recognition and end-to-end translation of sign language sentences with deep learning models. In this work, a dataset consisting of 300 commonly used sentences is gathered from 3 volunteers. In the design of the recognition network, the model mainly consists of three layers: convolutional neural network, bi-directional long short-term memory, and connectionist temporal classification. The model achieves accuracy rates of 99.07% in word-level evaluation and 97.34% in sentence-level evaluation. In the design of the translation network, the encoder-decoder structured model is mainly based on long short-term memory with global attention. The word error rate of end-to-end translation is 16.63%. The proposed method has the potential to recognize more sign language sentences with reliable inertial data from the device.

Graphene-based Two-Stage Enhancement Pressure Sensor for Subtle Mechanical Force Monitoring

The development of pressure sensors with high sensitivity and a low detection limit for subtle mechanical force monitoring and the understanding of the sensing mechanism behind subtle mechanical force monitoring are of great significance for intelligent technology. Here, we proposed a graphene-based two-stage enhancement pressure sensor (GTEPS), and we analyzed the difference between subtle mechanical force monitoring and conventional mechanical force monitoring. The GTEPS exhibited a high sensitivity of 62.2 kPa-1 and a low detection limit of 0.1 Pa. Leveraging its excellent performance, the GTEPS was successfully applied in various subtle mechanical force monitoring applications, including acoustic wave detection, voice-print recognition, and pulse wave monitoring. In acoustic wave detection, the GTEPS achieved a 100% recognition accuracy for six words. In voiceprint recognition, the sensor exhibited accurate identification of distinct voiceprints among individuals. Furthermore, in pulse wave monitoring, GTEPS demonstrated effective detection of pulse waves. By combination of the pulse wave signals with electrocardiogram (ECG) signals, it enabled the assessment of blood pressure. These results demonstrate the excellent performance of GTEPS and highlight its great potential for subtle mechanical force monitoring and its various applications. The current results indicate that GTEPS shows great potential for applications in subtle mechanical force monitoring.

Fiber-Based Noncontact Sensor with Stretchability for Underwater Wearable Sensing and VR Applications

The rapid development of artificial intelligent wearable devices has led to an increasing need for seamless information exchange between humans, machines, and virtual spaces, often relying on touch sensors as the primary interaction medium. Additionally, the demand for underwater detection technologies is on the rise owing to the prevalent wet and submerged environment. Here, a fiber-based capacitive sensor with superior stretchability and hydrophobicity is proposed, designed to cater to noncontact and underwater applications. The sensor is constructed using bacterial cellulose (BC)@BC/carbon nanotubes (CNTs) (BBT) helical fiber as the matrix and methyltrimethoxysilane (MTMS) as the hydrophobic modified agent, forming a hydrophobic silylated BC@BC/CNT (SBBT) helical fiber by the chemical vapor deposition (CVD) technique. These fibers exhibit an impressive contact angle of 132.8°. The SBBT helicalfiber-based capacitive sensor presents capabilities for both noncontact and underwater sensing, which exhibits a significant capacitance change of -0.27 (at a distance of 0.5 cm). We have achieved interactive control between real space and virtual space through intelligent data analysis technology with minimal interference from the presence of water. This work has laid a solid foundation of noncontact sensing with attributes such as degradability, stretchability, and hydrophobicity. Moreover, it offers promising solutions for barrier-free communication in virtual reality (VR) and underwater applications, providing avenues for smart human-machine interfaces for submerged use.

Artificial Intelligence Meets Flexible Sensors: Emerging Smart Flexible Sensing Systems Driven by Machine Learning and Artificial Synapses

The recent wave of the artificial intelligence (AI) revolution has aroused unprecedented interest in the intelligentialize of human society. As an essential component that bridges the physical world and digital signals, flexible sensors are evolving from a single sensing element to a smarter system, which is capable of highly efficient acquisition, analysis, and even perception of vast, multifaceted data. While challenging from a manual perspective, the development of intelligent flexible sensing has been remarkably facilitated owing to the rapid advances of brain-inspired AI innovations from both the algorithm (machine learning) and the framework (artificial synapses) level. This review presents the recent progress of the emerging AI-driven, intelligent flexible sensing systems. The basic concept of machine learning and artificial synapses are introduced. The new enabling features induced by the fusion of AI and flexible sensing are comprehensively reviewed, which significantly advances the applications such as flexible sensory systems, soft/humanoid robotics, and human activity monitoring. As two of the most profound innovations in the twenty-first century, the deep incorporation of flexible sensing and AI technology holds tremendous potential for creating a smarter world for human beings.

Metaverse Wearables for Immersive Digital Healthcare: A Review

The recent exponential growth of metaverse technology has been instrumental in reshaping a myriad of sectors, not least digital healthcare. This comprehensive review critically examines the landscape and future applications of metaverse wearables toward immersive digital healthcare. The key technologies and advancements that have spearheaded the metamorphosis of metaverse wearables are categorized, encapsulating all-encompassed extended reality, such as virtual reality, augmented reality, mixed reality, and other haptic feedback systems. Moreover, the fundamentals of their deployment in assistive healthcare (especially for rehabilitation), medical and nursing education, and remote patient management and treatment are investigated. The potential benefits of integrating metaverse wearables into healthcare paradigms are multifold, encompassing improved patient prognosis, enhanced accessibility to high-quality care, and high standards of practitioner instruction. Nevertheless, these technologies are not without their inherent challenges and untapped opportunities, which span privacy protection, data safeguarding, and innovation in artificial intelligence. In summary, future research trajectories and potential advancements to circumvent these hurdles are also discussed, further augmenting the incorporation of metaverse wearables within healthcare infrastructures in the post-pandemic era.

Fabrication of Laser-Induced Graphene Based Flexible Sensors Using 355 nm Ultraviolet Laser and Their Application in Human-Computer Interaction System

In recent years, flexible sensors based on laser-induced graphene (LIG) have played an important role in areas such as smart healthcare, smart skin, and wearable devices. This paper presents the fabrication of flexible sensors based on LIG technology and their applications in human-computer interaction (HCI) systems. Firstly, LIG with a sheet resistance as low as 4.5 Ω per square was generated through direct laser interaction with commercial polyimide (PI) film. The flexible sensors were then fabricated through a one-step method using the as-prepared LIG. The applications of the flexible sensors were demonstrated by an HCI system, which was fabricated through the integration of the flexible sensors and a flexible glove. The as-prepared HCI system could detect the bending motions of different fingers and translate them into the movements of the mouse on the computer screen. At the end of the paper, a demonstration of the HCI system is presented in which words were typed on a computer screen through the bending motion of the fingers. The newly designed LIG-based flexible HCI system can be used by persons with limited mobility to control a virtual keyboard or mouse pointer, thus enhancing their accessibility and independence in the digital realm.

Recycled, Contaminated, Crumpled Aluminum Foil-Driven Triboelectric Nanogenerator

With rapid urbanization and global population growth, the amount of wasted aluminum foil is significantly increasing. Most deformed and contaminated foil is difficult to recycle; hence, it is landfilled or incinerated, causing environmental pollution. Therefore, using aluminum foil waste for electricity may be conducive to addressing environmental problems. In this regard, various literatures have explored the concept of energy generation using foil, while a crumple ball design for this purpose has not been studied. Thus, a recycled foil-based crumpled ball triboelectric nanogenerator (RFCB-TENG) is proposed. The crumpled ball design can minimize the effects of contamination on foil, ensuring efficient power output. Moreover, owing to novel crumpled design, the RFCB-TENG has some outstanding characteristics to become a sustainable power source, such as ultralight weight, low noise, and high durability. By introducing the air-breakdown model, the RFCB-TENG achieved an output peak voltage of 648 V, a current of 8.1 mA cm3 , and an optimum power of 162.7 mW cm3 . The structure of the RFCB-TENG is systemically optimized depending on the design parameters to realize the optimum output performance. Finally, the RFCB-TENG operated 500 LEDs and 30-W commercial lamps. This work paves the guideline for effectively fabricating the TENG using waste-materials while exhibiting outstanding characteristics.

Soft Bioelectronics for Therapeutics

Soft bioelectronics play an increasingly crucial role in high-precision therapeutics due to their softness, biocompatibility, clinical accuracy, long-term stability, and patient-friendliness. In this review, we provide a comprehensive overview of the latest representative therapeutic applications of advanced soft bioelectronics, ranging from wearable therapeutics for skin wounds, diabetes, ophthalmic diseases, muscle disorders, and other diseases to implantable therapeutics against complex diseases, such as cardiac arrhythmias, cancer, neurological diseases, and others. We also highlight key challenges and opportunities for future clinical translation and commercialization of soft therapeutic bioelectronics toward personalized medicine.

Artificial intelligence enhanced sensors - enabling technologies to next-generation healthcare and biomedical platform

The fourth industrial revolution has led to the development and application of health monitoring sensors that are characterized by digitalization and intelligence. These sensors have extensive applications in medical care, personal health management, elderly care, sports, and other fields, providing people with more convenient and real-time health services. However, these sensors face limitations such as noise and drift, difficulty in extracting useful information from large amounts of data, and lack of feedback or control signals. The development of artificial intelligence has provided powerful tools and algorithms for data processing and analysis, enabling intelligent health monitoring, and achieving high-precision predictions and decisions. By integrating the Internet of Things, artificial intelligence, and health monitoring sensors, it becomes possible to realize a closed-loop system with the functions of real-time monitoring, data collection, online analysis, diagnosis, and treatment recommendations. This review focuses on the development of healthcare artificial sensors enhanced by intelligent technologies from the aspects of materials, device structure, system integration, and application scenarios. Specifically, this review first introduces the great advances in wearable sensors for monitoring respiration rate, heart rate, pulse, sweat, and tears; implantable sensors for cardiovascular care, nerve signal acquisition, and neurotransmitter monitoring; soft wearable electronics for precise therapy. Then, the recent advances in volatile organic compound detection are highlighted. Next, the current developments of human-machine interfaces, AI-enhanced multimode sensors, and AI-enhanced self-sustainable systems are reviewed. Last, a perspective on future directions for further research development is also provided. In summary, the fusion of artificial intelligence and artificial sensors will provide more intelligent, convenient, and secure services for next-generation healthcare and biomedical applications.

Foam nickel-PDMS composite film based triboelectric nanogenerator for speed and acceleration sensing

As a new energy conversion technology, triboelectric nanogenerator (TENG) can use the coupling of triboelectrification and electrostatic induction effect to convert tiny mechanical energy into electrical energy, powering small electronic devices. In this paper, a vibration sensing triboelectric nanogenerator (V-TENG) based on a foam nickel-PDMS composite film was prepared, which can convert low frequency and small-amplitude mechanical energy into electrical energy, and the open circuit voltage of V-TENG can reach 3.6V at a vibration frequency of 4 Hz. In addition, the V-TENG can be used as a self-powered speed/acceleration sensor to detect speed changes in the range of 0.3 m/s to 1.5 m/s and acceleration changes in the range of 3 m/s2 to 13 m/s2.

Recent Advances in Flexible Piezoresistive Arrays: Materials, Design, and Applications

Spatial distribution perception has become an important trend for flexible pressure sensors, which endows wearable health devices, bionic robots, and human-machine interactive interfaces (HMI) with more precise tactile perception capabilities. Flexible pressure sensor arrays can monitor and extract abundant health information to assist in medical detection and diagnosis. Bionic robots and HMI with higher tactile perception abilities will maximize the freedom of human hands. Flexible arrays based on piezoresistive mechanisms have been extensively researched due to the high performance of pressure-sensing properties and simple readout principles. This review summarizes multiple considerations in the design of flexible piezoresistive arrays and recent advances in their development. First, frequently used piezoresistive materials and microstructures are introduced in which various strategies to improve sensor performance are presented. Second, pressure sensor arrays with spatial distribution perception capability are discussed emphatically. Crosstalk is a particular concern for sensor arrays, where mechanical and electrical sources of crosstalk issues and the corresponding solutions are highlighted. Third, several processing methods are also introduced, classified as printing, field-assisted and laser-assisted fabrication. Next, the representative application works of flexible piezoresistive arrays are provided, including human-interactive systems, healthcare devices, and some other scenarios. Finally, outlooks on the development of piezoresistive arrays are given.

Integrated intelligent tactile system for a humanoid robot

Tactile perception is the basis of human motion. Achieving artificial tactility is one of the challenges in the fields of smart robotics and artificial intelligence (AI), because touch emulation relies on high-performance pressure sensor arrays, signal reading, information processing, and feedback control. In this paper, we report an integrated intelligent tactile system (IITS) that is integrated with a humanoid robot to achieve human-like artificial tactile perception. The IITS is a closed-loop system that includes a multi-channel tactile sensing e-skin, a data acquisition and information processing chip, and a feedback control. With customized preset values of threshold pressures, the IITS-integrated robot can flexibly grasp various objects. The IITS has potential applications in the design of prosthetic hands, space manipulators, deep-sea exploration robots, and human-robot interactions.

A Wearable Multidimensional Motion Sensor for AI-Enhanced VR Sports

Regular exercise paves the way to a healthy life. However, conventional sports events are susceptible to weather conditions. Current motion sensors for home-based sports are mainly limited by operation power consumption, single-direction sensitivity, or inferior data analysis. Herein, by leveraging the 3-dimensional printing technique and triboelectric effect, a wearable self-powered multidimensional motion sensor has been developed to detect both the vertical and planar movement trajectory. By integrating with a belt, this sensor could be used to identify some low degree of freedom motions, e.g., waist or gait motion, with a high accuracy of 93.8%. Furthermore, when wearing the sensor at the ankle position, signals generated from shank motions that contain more abundant information could also be effectively collected. By means of a deep learning algorithm, the kicking direction and force could be precisely differentiated with an accuracy of 97.5%. Toward practical application, a virtual reality-enabled fitness game and a shooting game were successfully demonstrated. This work is believed to open up new insights for the development of future household sports or rehabilitation.

Stretchable and All-Directional Strain-Insensitive Electronic Glove for Robotic Skins and Human-Machine Interfacing

Electronic gloves (e-gloves), with their multifunctional sensing capability, hold a promising application in robotic skin and human-machine interfaces, endowing robots with a human sense of touch. Despite the progress in developing e-gloves by exploiting flexible or stretchable sensors, existing models have inherent rigidity in their sensing area, limiting their stretchability and sensing performance. Herein, we present an all-directional strain-insensitive stretchable e-glove that successfully extends sensing functionality such as pressure, temperature, humidity, and ECG with minimal crosstalk. A scalable and facile method is successfully demonstrated by combining low-cost CO₂ laser engraving and electrospinning technology to fabricate multimodal e-glove sensors with a vertical architecture. In comparison to other smart gloves, the proposed e-glove features a ripple-like meandering sensing area and interconnections that are designed to stretch in response to the applied deformation, without affecting the performance of the sensors offering full mechanical stretchability. Furthermore, CNT-coated laser-engraved graphene (CNT/LEG) is used as an active sensing material in which the cross-linking network of the CNT in the LEG minimizes the stress effect and maximizes the sensitivity of the sensors. The fabricated e-glove can detect hot/cold, moisture, and pain simultaneously and precisely, while also allowing for remote transmission of sensory data to the user.

Tendon-Inspired Anisotropic Hydrogels with Excellent Mechanical Properties for Strain Sensors

Anisotropic conductive hydrogels mimicking the natural tissues with high mechanical properties and intelligent sensing have played an important role in the field of flexible electronic devices. Herein, tensile remodeling, drying, and subsequent ion cross-linking methods were used to construct anisotropic hydrogels, which were inspired by the orientation and functionality of tendons. Due to the anisotropic arrangement of the polymer network, the mechanical performance and electrical conductivity were greatly improved in specific directions. The tensile stress and elastic modulus of the hydrogel along the network orientation were 29.82 and 28.53 MPa, which were higher than those along the vertical orientation, 9.63 and 11.7 MPa, respectively. Moreover, the hydrogels exhibited structure-dependent anisotropic sensing. The gauge factors (GFs) parallel to the prestretching direction were greater than the GF along the vertical direction. Thus, the tendon-inspired conductive hydrogels with anisotropy could be used as flexible sensors for joint motion detection and voice recognition. The anisotropic hydrogel-based sensors are highly expected to promote the great development of emerging soft electronics and medical detection.

Multichannel Gradient Piezoelectric Transducer Assisted with Deep Learning for Broadband Acoustic Sensing

As an important part of human-machine interfaces, piezoelectric voice recognition has received extensive attention due to its unique self-powered nature. However, conventional voice recognition devices exhibit a limited response frequency band due to the intrinsic hardness and brittleness of piezoelectric ceramics or the flexibility of piezoelectric fibers. Here, we propose a cochlear-inspired multichannel piezoelectric acoustic sensor (MAS) based on gradient PVDF piezoelectric nanofibers for broadband voice recognition by a programmable electrospinning technique. Compared with the common electrospun PVDF membrane-based acoustic sensor, the developed MAS demonstrates the greatly 300%-broadened frequency band and the substantially 334.6%-enhanced piezoelectric output. More importantly, this MAS can serve as a high-fidelity auditory platform for music recording and human voice recognition, in which the classification accuracy rate can reach up to 100% in coordination with deep learning. The programmable bionic gradient piezoelectric nanofiber may provide a universal strategy for the development of intelligent bioelectronics.

Recent Progress of Tactile and Force Sensors for Human-Machine Interaction

Human-Machine Interface (HMI) plays a key role in the interaction between people and machines, which allows people to easily and intuitively control the machine and immersively experience the virtual world of the meta-universe by virtual reality/augmented reality (VR/AR) technology. Currently, wearable skin-integrated tactile and force sensors are widely used in immersive human-machine interactions due to their ultra-thin, ultra-soft, conformal characteristics. In this paper, the recent progress of tactile and force sensors used in HMI are reviewed, including piezoresistive, capacitive, piezoelectric, triboelectric, and other sensors. Then, this paper discusses how to improve the performance of tactile and force sensors for HMI. Next, this paper summarizes the HMI for dexterous robotic manipulation and VR/AR applications. Finally, this paper summarizes and proposes the future development trend of HMI.

Recent Progress of Energy-Storage-Device-Integrated Sensing Systems

With the rapid prosperity of the Internet of things, intelligent human-machine interaction and health monitoring are becoming the focus of attention. Wireless sensing systems, especially self-powered sensing systems that can work continuously and sustainably for a long time without an external power supply have been successfully explored and developed. Yet, the system integrated by energy-harvester needs to be exposed to a specific energy source to drive the work, which provides limited application scenarios, low stability, and poor continuity. Integrating the energy storage unit and sensing unit into a single system may provide efficient ways to solve these above problems, promoting potential applications in portable and wearable electronics. In this review, we focus on recent advances in energy-storage-device-integrated sensing systems for wearable electronics, including tactile sensors, temperature sensors, chemical and biological sensors, and multifunctional sensing systems, because of their universal utilization in the next generation of smart personal electronics. Finally, the future perspectives of energy-storage-device-integrated sensing systems are discussed.

Anatomically Designed Triboelectric Wristbands with Adaptive Accelerated Learning for Human-Machine Interfaces

Recent advances in flexible wearable devices have boosted the remarkable development of devices for human-machine interfaces, which are of great value to emerging cybernetics, robotics, and Metaverse systems. However, the effectiveness of existing approaches is limited by the quality of sensor data and classification models with high computational costs. Here, a novel gesture recognition system with triboelectric smart wristbands and an adaptive accelerated learning (AAL) model is proposed. The sensor array is well deployed according to the wrist anatomy and retrieves hand motions from a distance, exhibiting highly sensitive and high-quality sensing capabilities beyond existing methods. Importantly, the anatomical design leads to the close correspondence between the actions of dominant muscle/tendon groups and gestures, and the resulting distinctive features in sensor signals are very valuable for differentiating gestures with data from 7 sensors. The AAL model realizes a 97.56% identification accuracy in training 21 classes with only one-third operands of the original neural network. The applications of the system are further exploited in real-time somatosensory teleoperations with a low latency of <1 s, revealing a new possibility for endowing cyber-human interactions with disruptive innovation and immersive experience.

Degradable MXene-Doped Polylactic Acid Textiles for Wearable Biomonitoring

Degradable wearable electronics offer a promising route to construct sustainable cities and reduced carbon society. However, the difficult functionalization and the poor stability of degradable sensitive materials dramatically restrict their application in personalized healthcare assessment. Herein, we developed a scalable, low-cost, and porosity degradable MXene-polylactic acid textile (DMPT) for on-body biomonitoring via electrospinning. A combination of polydimethylsiloxane templating and MXene flake impregnation methods endows the fabricated DMPT with a sensitivity of 5.37/kPa, a fast response time of 98 ms, and a good mechanical stability (over 6000 cycles). An efficient degradation of as-electrospun DMPTs was observed in 1 wt % sodium carbonate solution. It is found that the incorporation of MXene nanosheets boosts the hydrophilicity and degradation efficiency of active polylactic acid nanofibrous films in comparison with the pristine counterpart. Furthermore, the as-received DMPT demonstrates great capability in monitoring physiological activities of wrist pulse, knuckle bending, swallowing, and vocalization. This work opens up a new paradigm for developing and optimizing high-performance degradable on-body electronics.

Deep-Learning-Assisted Underwater 3D Tactile Tensegrity

The growth of underwater robotic applications in ocean exploration and research has created an urgent need for effective tactile sensing. Here, we propose an underwater 3-dimensional tactile tensegrity (U3DTT) based on soft self-powered triboelectric nanogenerators and deep-learning-assisted data analytics. This device can measure and distinguish the magnitude, location, and orientation of perturbations in real time from both flow field and interaction with obstacles and provide collision protection for underwater vehicles operation. It is enabled by the structure that mimics terrestrial animals' musculoskeletal systems composed of both stiff bones and stretchable muscles. Moreover, when successfully integrated with underwater vehicles, the U3DTT shows advantages of multiple degrees of freedom in its shape modes, an ultrahigh sensitivity, and fast response times with a low cost and conformability. The real-time 3-dimensional pose of the U3DTT has been predicted with an average root-mean-square error of 0.76 in a water pool, indicating that this developed U3DTT is a promising technology in vehicles with tactile feedback.

A Self-Powered, Highly Embedded and Sensitive Tribo-Label-Sensor for the Fast and Stable Label Printer

Label-sensor is an essential component of the label printer which is becoming a most significant tool for the development of Internet of Things (IoT). However, some drawbacks of the traditional infrared label-sensor make the printer fail to realize the high-speed recognition of labels as well as stable printing. Herein, we propose a self-powered and highly sensitive tribo-label-sensor (TLS) for accurate label identification, positioning and counting by embedding triboelectric nanogenerator into the indispensable roller structure of a label printer. The sensing mechanism, device parameters and deep comparison with infrared sensor are systematically studied both in theory and experiment. As the results, TLS delivers 6 times higher signal magnitude than traditional one. Moreover, TLS is immune to label jitter and temperature variation during fast printing and can also be used for transparent label directly and shows long-term robustness. This work may provide an alternative toolkit with outstanding advantages to improve current label printer and further promote the development of IoT.

Touch IoT enabled by wireless self-sensing and haptic-reproducing electronic skin

Tactile sensations are mainly transmitted to each other by physical touch. Wireless touch perception could be a revolution for us to interact with the world. Here, we report a wireless self-sensing and haptic-reproducing electronic skin (e-skin) to realize noncontact touch communications. A flexible self-sensing actuator was developed to provide an integrated function in both tactile sensing and haptic feedback. When this e-skin was dynamically pressed, the actuator generated an induced voltage as tactile information. Via wireless communication, another e-skin could receive this tactile data and run a synchronized haptic reproduction. Thus, touch could be wirelessly conveyed in bidirections between two users as a touch intercom. Furthermore, this e-skin could be connected with various smart devices to form a touch internet of things where one-to-one and one-to-multiple touch delivery could be realized. This wireless touch presents huge potentials in remote touch video, medical care/assistance, education, and many other applications.

Exploiting domain transformation and deep learning for hand gesture recognition using a low-cost dataglove

Hand gesture recognition is one of the most widely explored areas under the human-computer interaction domain. Although various modalities of hand gesture recognition have been explored in the last three decades, in recent years, due to the availability of hardware and deep learning algorithms, hand gesture recognition research has attained renewed momentum. In this paper, we evaluate the effectiveness of a low-cost dataglove for classifying hand gestures in the light of deep learning. We have developed a cost-effective dataglove using five flex sensors, an inertial measurement unit, and a powerful microcontroller for onboard processing and wireless connectivity. We have collected data from 25 subjects for 24 static and 16 dynamic American sign language gestures for validating our system. Moreover, we proposed a novel Spatial Projection Image-based technique for dynamic hand gesture recognition. We also explored a parallel-path neural network architecture for handling multimodal data more effectively. Our method produced an F1-score of 82.19% for static gestures and 97.35% for dynamic gestures from a leave-one-out-cross-validation approach. Overall, this study demonstrates the promising performance of a generalized hand gesture recognition technique in hand gesture recognition. The dataset used in this work has been made publicly available.

Machine-Learning Assisted Handwriting Recognition Using Graphene Oxide-Based Hydrogel

Machine-learning assisted handwriting recognition is crucial for development of next-generation biometric technologies. However, most of the currently reported handwriting recognition systems are lacking in flexible sensing and machine learning capabilities, both of which are essential for implementation of intelligent systems. Herein, assisted by machine learning, we develop a new handwriting recognition system, which can be applied as both a recognizer for written texts and an encryptor for confidential information. This flexible and intelligent handwriting recognition system combines a printed circuit board with graphene oxide-based hydrogel sensors. It offers fast response and good sensitivity and allows high-precision recognition of handwritten content from a single letter to words and signatures. By analyzing 690 acquired handwritten signatures obtained from seven participants, we successfully demonstrate a fast recognition time (less than 1 s) and a high recognition rate (∼91.30%). Our developed handwriting recognition system has great potential in advanced human-machine interactions, wearable communication devices, soft robotics manipulators, and augmented virtual reality.

Multifunctional Fiber-Enabled Intelligent Health Agents

The application of wearable devices is promoting the development toward digitization and intelligence in the field of health. However, the current smart devices centered on human health have disadvantages such as weak perception, high interference degree, and unfriendly interaction. Here, an intelligent health agent based on multifunctional fibers, with the characteristics of autonomy, activeness, intelligence, and perceptibility enabling health services, is proposed. According to the requirements for healthcare in the medical field and daily life, four major aspects driven by intelligent agents, including health monitoring, therapy, protection, and minimally invasive surgery, are summarized from the perspectives of materials science, medicine, and computer science. The function of intelligent health agents is realized through multifunctional fibers as sensing units and artificial intelligence technology as a cognitive engine. The structure, characteristics, and performance of fibers and analysis systems and algorithms are reviewed, while discussing future challenges and opportunities in healthcare and medicine. Finally, based on the above four aspects, future scenarios related to health protection of a person's life are presented. Intelligent health agents will have the potential to accelerate the realization of precision medicine and active health.

Imperceptible, designable, and scalable braided electronic cord

Flexible sensors, friendly interfaces, and intelligent recognition are important in the research of novel human-computer interaction and the development of smart devices. However, major challenges are still encountered in designing user-centered smart devices with natural, convenient, and efficient interfaces. Inspired by the characteristics of textile-based flexible electronic sensors, in this article, we report a braided electronic cord with a low-cost, and automated fabrication to realize imperceptible, designable, and scalable user interfaces. The braided electronic cord is in a miniaturized form, which is suitable for being integrated with various occasions in life. To achieve high-precision interaction, a multi-feature fusion algorithm is designed to recognize gestures of different positions, different contact areas, and different movements performed on a single braided electronic cord. The recognized action results are fed back to varieties of interactive terminals, which show the diversity of cord forms and applications. Our braided electronic cord with the features of user friendliness, excellent durability and rich interaction mode will greatly promote the development of human-machine integration in the future.

Human Machine Interface with Wearable Electronics Using Biodegradable Triboelectric Films for Calligraphy Practice and Correction

Letter handwriting, especially stroke correction, is of great importance for recording languages and expressing and exchanging ideas for individual behavior and the public. In this study, a biodegradable and conductive carboxymethyl chitosan-silk fibroin (CSF) film is prepared to design wearable triboelectric nanogenerator (denoted as CSF-TENG), which outputs of V_oc ≈ 165 V, I_sc ≈ 1.4 μA, and Q_sc ≈ 72 mW cm^-2. Further, in vitro biodegradation of CSF film is performed through trypsin and lysozyme. The results show that trypsin and lysozyme have stable and favorable biodegradation properties, removing 63.1% of CSF film after degrading for 11 days. Further, the CSF-TENG-based human-machine interface (HMI) is designed to promptly track writing steps and access the accuracy of letters, resulting in a straightforward communication media of human and machine. The CSF-TENG-based HMI can automatically recognize and correct three representative letters (F, H, and K), which is benefited by HMI system for data processing and analysis. The CSF-TENG-based HMI can make decisions for the next stroke, highlighting the stroke in advance by replacing it with red, which can be a candidate for calligraphy practice and correction. Finally, various demonstrations are done in real-time to achieve virtual and real-world controls including writing, vehicle movements, and healthcare.

Roles of Low-Dimensional Nanomaterials in Pursuing Human-Machine-Thing Natural Interaction

A wide variety of low-dimensional nanomaterials with excellent properties can meet almost all the requirements of functional materials for information sensing, processing, and feedback devices. Low-dimensional nanomaterials are becoming the star of hope on the road to pursuing human-machine-thing natural interactions, benefiting from the breakthroughs in precise preparation, performance regulation, structural design, and device construction in recent years. This review summarizes several types of low-dimensional nanomaterials commonly used in human-machine-thing natural interactions and outlines the differences in properties and application areas of different materials. According to the sequence of information flow in the human-machine-thing interaction process, the representative research progress of low-dimensional nanomaterials-based information sensing, processing, and feedback devices is reviewed and the key roles played by low-dimensional nanomaterials are discussed. Finally, the development trends and existing challenges of low-dimensional nanomaterials in the field of human-machine-thing natural interaction technology are discussed.

Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces

A wearable silent speech interface (SSI) is a promising platform that enables verbal communication without vocalization. The most widely studied methodology for SSI focuses on surface electromyography (sEMG). However, sEMG suffers from low scalability because of signal quality-related issues, including signal-to-noise ratio and interelectrode interference. Hence, here, we present a novel SSI by utilizing crystalline-silicon-based strain sensors combined with a 3D convolutional deep learning algorithm. Two perpendicularly placed strain gauges with minimized cell dimension (<0.1 mm²) could effectively capture the biaxial strain information with high reliability. We attached four strain sensors near the subject’s mouths and collected strain data of unprecedently large wordsets (100 words), which our SSI can classify at a high accuracy rate (87.53%). Several analysis methods were demonstrated to verify the system’s reliability, as well as the performance comparison with another SSI using sEMG electrodes with the same dimension, which exhibited a relatively low accuracy rate (42.60%).

Designing an efficient platform that enables verbal communication without vocalization remains a challenge. Here, the authors propose a silent speech interface by utilizing a deep learning algorithm combined with strain sensors attached near the subject’s mouth, able to collect 100 words and classify at a high accuracy rate.

Self-Powered Smart Gloves Based on Triboelectric Nanogenerators

The hands are used in all facets of daily life, from simple tasks such as grasping and holding to complex tasks such as communication and using technology. Finding a way to not only monitor hand movements and gestures but also to integrate that data with technology is thus a worthwhile task. Gesture recognition is particularly important for those who rely on sign language to communicate, but the limitations of current vision-based and sensor-based methods, including lack of portability, bulkiness, low sensitivity, highly expensive, and need for external power sources, among many others, make them impractical for daily use. To resolve these issues, smart gloves can be created using a triboelectric nanogenerator (TENG), a self-powered technology that functions based on the triboelectric effect and electrostatic induction and is also cheap to manufacture, small in size, lightweight, and highly flexible in terms of materials and design. In this review, an overview of the existing self-powered smart gloves will be provided based on TENGs, both for gesture recognition and human-machine interface, concluding with a discussion on the future outlook of these devices.

Debonding-On-Demand Polymeric Wound Patches for Minimal Adhesion and Clinical Communication

Herein, a multifunctional bilayer wound patch is developed by integrating a debonding-on-demand polymeric tissue adhesive (DDPTA) with an ionic conducting elastomer (ICE). As a skin adhesive layer, the DDPTA is soft and adherent at skin temperature but hard and non-tacky when cooled, so it provides unique temperature-triggered quick adhesion and non-forced detachment from the skin. During use, the dense surface of the DDPTA prevents blood infiltration and reduces unnecessary blood loss with gentle pressing. Moreover, its hydrophobic matrix helps to repel blood and prevents the formation of clots, thus precluding wound tearing during its removal. This unique feature enables the DDPTA to avoid the severe deficiencies of hydrophilic adhesives, providing a reliable solution for a wide range of secondary wound injuries. The DDPTA is versatile in that it can be covered with ICE to configure a DDPTA@ICE patch for initiating non-verbal communication systems by the fingers, leading toward sign language recognition and a remote clinical alarm system. This multifunctional wound patch with debonding-on-demand can promote a new style of tissue sealant for convenient clinical communication.

Soft Modular Glove with Multimodal Sensing and Augmented Haptic Feedback Enabled by Materials' Multifunctionalities

Immersive communications rely on smart perception based on diversified and augmented sensing and feedback technologies. However, the increasing of functional components also raises the issue of increased system complexity. Here, we propose a modular soft glove with multimodal sensing and feedback functions by exploring and utilizing the multiple properties of glove materials. With a single design of basic structure, the main functional unit possesses triboelectric-based sensing of static and dynamic contact, vibration, strain, and pneumatic actuation. Additionally, the same unit is also capable of offering pneumatic tactile haptic feedback and electroresistive thermal haptic feedback. Together with a machine learning algorithm, the proposed glove not only performs real-time detection of dexterous hand motion and direct feedback but also realizes intelligent object recognition and augmented feedback, which significantly enhance the communication and perception of more comprehensive information. In general, this glove utilizes a facile designed sensing and feedback device to achieve dual-way and multimodal communication among humans, machines, and the virtual world via smart perceptions.

Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions

Advancements of virtual reality technology pave the way for developing wearable devices to enable somatosensory sensation, which can bring more comprehensive perception and feedback in the metaverse-based virtual society. Here, we propose augmented tactile-perception and haptic-feedback rings with multimodal sensing and feedback capabilities. This highly integrated ring consists of triboelectric and pyroelectric sensors for tactile and temperature perception, and vibrators and nichrome heaters for vibro- and thermo-haptic feedback. All these components integrated on the ring can be directly driven by a custom wireless platform of low power consumption for wearable/portable scenarios. With voltage integration processing, high-resolution continuous finger motion tracking is achieved via the triboelectric tactile sensor, which also contributes to superior performance in gesture/object recognition with artificial intelligence analysis. By fusing the multimodal sensing and feedback functions, an interactive metaverse platform with cross-space perception capability is successfully achieved, giving people a face-to-face like immersive virtual social experience.

Current wearable solutions for Virtual Reality (VR) have limitations of complicated structures and large driven power. Here, the authors report a highly integrated ring consisting of multimodal sensing and feedback units for augmented interactions in metaverse.