C.DOT - Convolutional Deep Object Tracker for Augmented Reality Based Purely on Synthetic Data

A Feasibility Study of HoloLens Ear Image Guidance for Ear Reconstruction

BACKGROUND

An ideal 3D cartilage framework and accurate anatomical location are the most important factors to carry out a satisfactory reconstruction of the ear. To streamline this process, we developed an augmented reality assistance system, HoloLens Ear Image Guidance, which is based on computed tomography (CT) data, tailor-made navigation kits to guide reconstruction, data processing software, and HoloLens hardware. The objective of this study is to verify its feasibility in a clinical setting.

METHODS

This study first validated our system in healthy controls and then extrapolated data to test on patients with microtia. First, three healthy volunteers were recruited, and reconstructive navigation kits were made using 3D printing. CT data were collected for the head and neck and imported into the HoloLens Ear Image Guidance Application to generate a personalized 3D virtual ear image. Volunteers then wore the navigation kits while researchers observed them through the HoloLens to check accuracy, track delay, and view the ear image guide.Ten patients with unilateral microtia were recruited and CT data were collected, and reconstructive navigation kits were made to assist with surgery. The procedure was monitored to record the surgeon's experience wearing the HoloLens, the patients' complications associated with wearing navigation kit, and to measure the symmetry between the reconstructed ear and the reference counterpart.

RESULTS

In control patients, the deviation between the virtual image and the real ear was less than 2.4% (±0.22%); the tracking delay was less than 1.26 s (±0.09 s), the display effect was good, and surgeons did not report discomfort or dizziness while wearing the HoloLens. Volunteers did not report any pain from holding the navigation reference in their mouth during the test. Following validation, the HoloLens-assisted procedures were not associated with surgeon discomfort or dizziness. No complications were noted in patients including injury to the oral mucosa. Symmetry between the reconstructed ear and the contralateral ear was noted to be satisfactory in HoloLens-assisted surgery.

CONCLUSIONS

The HoloLens Ear Image Guidance initially met clinical demands in registration accuracy, tracking speed, and subjective user experience, which can be used as the basis for continual software improvements and clinical application.

A scoping review of deep learning in cancer nursing combined with augmented reality: The era of intelligent nursing is coming

Artificial intelligence has been developing greatly in the field of medicine. As a new research hotspot of artificial intelligence, deep learning (DL) has been widely applied in the fields of cancer risk assessment, symptom recognition, and cancer detection. Therefore, nursing care issues in terms of consuming time and energy, lower accuracy, and lower efficiency can be solved with applying DL in caring cancer patients. In addition, augmented reality (AR) has great navigation potential through combining computer-generated virtual elements with the real world. Thus, DL ​+ ​AR may facilitate patients with cancer to possess a brand-new model of nursing care that is more intelligent, mobile, and adapted to the information age, compared to traditional nursing. With the advent of the era of intelligent nursing, future nursing models can not only learn from the DL ​+ ​AR model to meet the needs of patients with cancer but also reduce nursing workload, save healthcare resources, and improve work efficiency, the quality of nursing care, as well as the quality of life for cancer patients.