Comparative analysis of respiratory motion tracking using Microsoft Kinect v2 sensor

Respiratory motion tracking of the thoracoabdominal surface based on defect-aware point cloud registration

The performance of conventional lung puncture surgery is a complex undertaking due to the surgeon's reliance on visual assessment of respiratory conditions and the manual execution of the technique while the patient maintains breath-holding. However, the failure to correctly perform a puncture technique can lead to negative outcomes, such as the development of sores and pneumothorax. In this work, we proposed a novel approach for monitoring respiratory motion by utilizing defect-aware point cloud registration and descriptor computation. Through a thorough examination of the attributes of the inputs, we suggest the incorporation of a defect detection branch into the registration network. Additionally, we developed two modules with the aim of augmenting the quality of the extracted features. A coarse-to-fine respiratory phase recognition approach based on descriptor computation is devised for the respiratory motion tracking. The efficacy of the suggested registration method is demonstrated through experimental findings conducted on both publicly accessible datasets and thoracoabdominal point cloud datasets. We obtained state-of-the-art registration results on ModelNet40 datasets, with 1.584∘ on rotation mean absolute error and 0.016 mm on translation mean absolute error, respectively. The experimental findings conducted on a thoracoabdominal point cloud dataset indicate that our method exhibits efficacy and efficiency, achieving a frame matching rate of 2 frames per second and a phase recognition accuracy of 96.3%. This allows identifying matching frames from template point clouds that display different parts of a patient's thoracoabdominal surface while breathing regularly to distinguish breathing stages and track breathing.

Data driven surrogate signal extraction for dynamic PET using selective PCA: time windows versus the combination of components

Objective.Respiratory motion correction is beneficial in positron emission tomography (PET), as it can reduce artefacts caused by motion and improve quantitative accuracy. Methods of motion correction are commonly based on a respiratory trace obtained through an external device (like the real time position management system) or a data driven method, such as those based on dimensionality reduction techniques (for instance principal component analysis (PCA)). PCA itself being a linear transformation to the axis of greatest variation. Data driven methods have the advantage of being non-invasive, and can be performed post-acquisition. However, their main downside being that they are adversely affected by the tracer kinetics of the dynamic PET acquisition. Therefore, they are mostly limited to static PET acquisitions. This work seeks to extend on existing PCA-based data-driven motion correction methods, to allow for their applicability to dynamic PET imaging.Approach.The methods explored in this work include; a moving window approach (similar to the Kinetic Respiratory Gating method from Schleyeret al(2014)), extrapolation of the principal component from later time points to earlier time points, and a method to score, select, and combine multiple respiratory components. The resulting respiratory traces were evaluated on 22 data sets from a dynamic [18F]-FDG study on patients with idiopathic pulmonary fibrosis. This was achieved by calculating their correlation with a surrogate signal acquired using a real time position management system.Main results.The results indicate that all methods produce better surrogate signals than when applying conventional PCA to dynamic data (for instance, a higher correlation with a gold standard respiratory trace). Extrapolating a late time point principal component produced more promising results than using a moving window. Scoring, selecting, and combining components held benefits over all other methods.Significance.This work allows for the extraction of a surrogate signal from dynamic PET data earlier in the acquisition and with a greater accuracy than previous work. This potentially allows for numerous other methods (for instance, respiratory motion correction) to be applied to this data (when they otherwise could not be previously used).

Respiratory signal estimation for cardiac perfusion SPECT using deep learning

BACKGROUND

Respiratory motion induces artifacts in reconstructed cardiac perfusion SPECT images. Correction for respiratory motion often relies on a respiratory signal describing the heart displacements during breathing. However, using external tracking devices to estimate respiratory signals can add cost and operational complications in a clinical setting.

PURPOSE

We aim to develop a deep learning (DL) approach that uses only SPECT projection data for respiratory signal estimation.

METHODS

A modified U-Net was implemented that takes temporally finely sampled SPECT sub-projection data (100 ms) as input. These sub-projections are obtained by reframing the 20-s list-mode data, resulting in 200 sub-projections, at each projection angle for each SPECT camera head. The network outputs a 200-time-point motion signal for each projection angle, which was later aggregated over all angles to give a full respiratory signal. The target signal for DL model training was from an external stereo-camera visual tracking system (VTS). In addition to comparing DL and VTS, we also included a data-driven approach based on the center-of-mass (CoM) strategy. This CoM method estimates respiratory signals by monitoring the axial changes of CoM for counts in the heart region of the sub-projections. We utilized 900 subjects with stress cardiac perfusion SPECT studies, with 302 subjects for testing and the remaining 598 subjects for training and validation.

RESULTS

The Pearson's correlation coefficient between the DL respiratory signal and the reference VTS signal was 0.90, compared to 0.70 between the CoM signal and the reference. For respiratory motion correction on SPECT images, all VTS, DL, and CoM approaches partially de-blured the heart wall, resulting in a thinner wall thickness and increased recovered maximal image intensity within the wall, with VTS reducing blurring the most followed by the DL approach. Uptake quantification for the combined anterior and inferior segments of polar maps showed a mean absolute difference from the reference VTS of 1.7% for the DL method for patients with motion >12 mm, compared to 2.6% for the CoM method and 8.5% for no correction.

CONCLUSION

We demonstrate the capability of a DL approach to estimate respiratory signal from SPECT projection data for cardiac perfusion imaging. Our results show that the DL based respiratory motion correction reduces artefacts and achieves similar regional quantification to that obtained using the stereo-camera VTS signals. This may enable fully automatic data-driven respiratory motion correction without relying on external motion tracking devices.

A Respiratory Motion Prediction Method Based on LSTM-AE with Attention Mechanism for Spine Surgery

Respiratory motion-induced vertebral movements can adversely impact intraoperative spine surgery, resulting in inaccurate positional information of the target region and unexpected damage during the operation. In this paper, we propose a novel deep learning architecture for respiratory motion prediction, which can adapt to different patients. The proposed method utilizes an LSTM-AE with attention mechanism network that can be trained using few-shot datasets during operation. To ensure real-time performance, a dimension reduction method based on the respiration-induced physical movement of spine vertebral bodies is introduced. The experiment collected data from prone-positioned patients under general anaesthesia to validate the prediction accuracy and time efficiency of the LSTM-AE-based motion prediction method. The experimental results demonstrate that the presented method (RMSE: 4.39%) outperforms other methods in terms of accuracy within a learning time of 2 min. The maximum predictive errors under the latency of 333 ms with respect to the x, y, and z axes of the optical camera system were 0.13, 0.07, and 0.10 mm, respectively, within a motion range of 2 mm.

High-speed low-noise optical respiratory monitoring for spot scanning proton therapy

PURPOSE

To investigate a novel optical markerless respiratory sensor for surface guided spot scanning proton therapy and to measure its main technical characteristics.

METHODS

The main characteristics of the respiratory sensor including sensitivity, linearity, noise, signal-to-noise, and time delay were measured using a dynamic phantom and electrical measuring equipment on a laboratory stand. The respiratory signals of free breathing and deep-inspiration breath-hold patterns were acquired for various distances with a volunteer. A comparative analysis of this sensor with existing commercially available and experimental respiratory monitoring systems was carried out based on several criteria including principle of operation, patient contact, application to proton therapy, distance range, accuracy (noise, signal-to-noise ratio), and time delay (sampling rate).

RESULTS

The sensor provides optical respiratory monitoring of the chest surface over a distance range of 0.4-1.2 m with the RMS noise of 0.03-0.60 mm, SNR of 40-15 dB (for motion with peak-to-peak of 10 mm), and time delay of 1.2 ± 0.2 ms.

CONCLUSIONS

The investigated optical respiratory sensor was found to be appropriate to use in surface guided spot scanning proton therapy. This sensor combined with a fast respiratory signal processing algorithm may provide accurate beam control and a fast response in patients' irregular breathing movements. A careful study of correlation between the respiratory signal and 4DCT data of tumor position will be required before clinical implementation.

Accuracy of a time-of-flight (ToF) imaging system for monitoring deep-inspiration breath-hold radiotherapy (DIBH-RT) for left breast cancer patients

Deep inspiration breath-hold radiotherapy (DIBH-RT) reduces cardiac dose by over 50%. However, poor breath-hold reproducibility could result in target miss which compromises the treatment success. This study aimed to benchmark the accuracy of a Time-of-Flight (ToF) imaging system for monitoring breath-hold during DIBH-RT. The accuracy of an Argos P330 3D ToF camera (Bluetechnix, Austria) was evaluated for patient setup verification and intra-fraction monitoring among 13 DIBH-RT left breast cancer patients. The ToF imaging was performed simultaneously with in-room cone beam computed tomography (CBCT) and electronic portal imaging device (EPID) imaging systems during patient setup and treatment delivery, respectively. Patient surface depths (PSD) during setup were extracted from the ToF and the CBCT images during free breathing and DIBH using MATLAB (MathWorks, Natick, MA) and the chest surface displacement were compared. The mean difference ± standard deviation, correlation coefficient, and limit of agreement between the CBCT and ToF were 2.88 ± 5.89 mm, 0.92, and - 7.36, 1.60 mm, respectively. The breath-hold stability and reproducibility were estimated using the central lung depth extracted from the EPID images during treatment and compared with the PSD from the ToF. The average correlation between ToF and EPID was - 0.84. The average intra-field reproducibility for all the fields was within 2.70 mm. The average intra-fraction reproducibility and stability were 3.74 mm, and 0.80 mm, respectively. The study demonstrated the feasibility of using ToF camera for monitoring breath-hold during DIBH-RT and shows good breath-hold reproducibility and stability during the treatment delivery.

The use of time-of-flight camera to assess respiratory rates and thoracoabdominal depths in patients with chronic respiratory disease

INTRODUCTION

Over the last 5 years, the analysis of respiratory patterns presents a growing usage in clinical and research purposes, but there is still currently a lack of easy-to-use and affordable devices to perform such kind of evaluation.

OBJECTIVES

The aim of this study is to validate a new specifically developed method, based on Kinect sensor, to assess respiratory patterns against spirometry under various conditions.

METHODS

One hundred and one participants took parts in one of the three validations studies. Twenty-five chronic respiratory disease patients (14 with chronic obstructive pulmonary disease (COPD) [65 ± 10 years old, FEV₁  = 37 (15% predicted value), VC = 62 (20% predicted value)], and 11 with lung fibrosis (LF) [64 ± 14 years old, FEV₁  = 55 (19% predicted value), VC = 62 (20% predicted value)]) and 76 healthy controls (HC) were recruited. The correlations between the signal of the Kinect (depth and respiratory rate) and the spirometer (tidal volume and respiratory rate) were computed in part 1. We then included 66 HC to test the ability of the system to detect modifications of respiratory patterns induced by various conditions known to modify respiratory pattern (cognitive load, inspiratory load and combination) in parts 2 and 3.

RESULTS

There is a strong correlation between the depth recorded by the Kinect and the tidal volume recorded by the spirometer: r = 0.973 for COPD patients, r = 0.989 for LF patients and r = 0.984 for HC. The Kinect is able to detect changes in breathing patterns induced by different respiratory disturbance conditions, gender and oral task.

CONCLUSIONS

Measurements performed with the Kinect sensors are highly correlated with the spirometer in HC and patients with COPD and LF. Kinect is also able to assess respiratory patterns under various loads and disturbances. This method is affordable, easy to use, fully automated and could be used in the current clinical context. Respiratory patterns are important to assess in daily clinics. However, there is currently no affordable and easy-to-use tool to evaluate these parameters in clinics. We validated a new system to assess respiratory patterns using the Kinect sensor in patients with chronic respiratory diseases.

3D Kinect Camera Scheme with Time-Series Deep-Learning Algorithms for Classification and Prediction of Lung Tumor Motility

This paper proposes a time-series deep-learning 3D Kinect camera scheme to classify the respiratory phases with a lung tumor and predict the lung tumor displacement. Specifically, the proposed scheme is driven by two time-series deep-learning algorithmic models: the respiratory-phase classification model and the regression-based prediction model. To assess the performance of the proposed scheme, the classification and prediction models were tested with four categories of datasets: patient-based datasets with regular and irregular breathing patterns; and pseudopatient-based datasets with regular and irregular breathing patterns. In this study, 'pseudopatients' refer to a dynamic thorax phantom with a lung tumor programmed with varying breathing patterns and breaths per minute. The total accuracy of the respiratory-phase classification model was 100%, 100%, 100%, and 92.44% for the four dataset categories, with a corresponding mean squared error (MSE), mean absolute error (MAE), and coefficient of determination (R²) of 1.2-1.6%, 0.65-0.8%, and 0.97-0.98, respectively. The results demonstrate that the time-series deep-learning classification and regression-based prediction models can classify the respiratory phases and predict the lung tumor displacement with high accuracy. Essentially, the novelty of this research lies in the use of a low-cost 3D Kinect camera with time-series deep-learning algorithms in the medical field to efficiently classify the respiratory phase and predict the lung tumor displacement.

AAPM task group report 302: Surface guided radiotherapy

The clinical use of surface imaging has increased dramatically with demonstrated utility for initial patient positioning, real-time motion monitoring, and beam gating in a variety of anatomical sites. The Therapy Physics Subcommittee and the Imaging for Treatment Verification Working Group of the American Association of Physicists in Medicine commissioned Task Group 302 to review the current clinical uses of surface imaging and emerging clinical applications. The specific charge of this task group was to provide technical guidelines for clinical indications of use for general positioning, breast deep-inspiration breath-hold (DIBH) treatment, and frameless stereotactic radiosurgery (SRS). Additionally, the task group was charged with providing commissioning and on-going quality assurance (QA) requirements for surface guided radiation therapy (SGRT) as part of a comprehensive QA program including risk assessment. Workflow considerations for other anatomic sites and for computed tomography (CT) simulation, including motion management are also discussed. Finally, developing clinical applications such as stereotactic body radiotherapy (SBRT) or proton radiotherapy are presented. The recommendations made in this report, which are summarized at the end of the report, are applicable to all video-based SGRT systems available at the time of writing. Review current use of non-ionizing surface imaging functionality and commercially available systems. Summarize commissioning and on-going quality assurance (QA) requirements of surface image-guided systems, including implementation of risk or hazard assessment of surface guided radiotherapy as a part of a total quality management program (e.g., TG-100). Provide clinically relevant technical guidelines that include recommendations for the use of SGRT for general patient positioning, breast DIBH, and frameless brain SRS, including potential pitfalls to avoid when implementing this technology. Discuss emerging clinical applications of SGRT and associated QA implications based on evaluation of technology and risk assessment. This article is protected by copyright. All rights reserved.

Contactless Monitoring of Breathing Pattern and Thoracoabdominal Asynchronies in Preterm Infants Using Depth Cameras: A Feasibility Study

Objective: Monitoring infants’ breathing activity is crucial in research and clinical applications but remains a challenge. This study aims to develop a contactless method to monitor breathing patterns and thoracoabdominal asynchronies in infants inside the incubator, using depth cameras. Methods: We proposed an algorithm to extract the 3D displacements of the ribcage and abdomen from the analysis of depth images. We evaluated the accuracy of the system in-vitro vs. a reference motion capture analyzer. We also conducted a feasibility study on 12 patients receiving non-invasive respiratory support to estimate the mean and the variability of the chest wall displacements in preterm infants and evaluate the suitability of the proposed system in the clinical setting. Results: In-vitro, the mean (95% CI) error in the measurement of amplitude, frequency and phase shift between compartmental displacements was −0.14 (−0.57, 0.28) mm, 0.02 (−0.99, 1.03) bpm, and −0.40 (−1.76, 0.95)°, respectively. In-vivo, the mean (95% CI) amplitude of the ribcage and abdomen displacements were 0.99 (0.34, 2.67) mm and 1.20 (0.40, 2.15) mm, respectively. Conclusions: The developed system proved accurate in-vitro and was suitable for the clinical environment. Clinical Impact: The proposed method has value for evaluating infants’ breathing patterns in research applications and, after further development, may represent a simple monitoring tool for infants’ respiratory activity inside the incubator.

Motion estimation and correction in SPECT, PET and CT

Patient motion impacts single photon emission computed tomography (SPECT), positron emission tomography (PET) and X-ray computed tomography (CT) by giving rise to projection data inconsistencies that can manifest as reconstruction artifacts, thereby degrading image quality and compromising accurate image interpretation and quantification. Methods to estimate and correct for patient motion in SPECT, PET and CT have attracted considerable research effort over several decades. The aims of this effort have been two-fold: to estimate relevant motion fields characterizing the various forms of voluntary and involuntary motion; and to apply these motion fields within a modified reconstruction framework to obtain motion-corrected images. The aims of this review are to outline the motion problem in medical imaging and to critically review published methods for estimating and correcting for the relevant motion fields in clinical and preclinical SPECT, PET and CT. Despite many similarities in how motion is handled between these modalities, utility and applications vary based on differences in temporal and spatial resolution. Technical feasibility has been demonstrated in each modality for both rigid and non-rigid motion, but clinical feasibility remains an important target. There is considerable scope for further developments in motion estimation and correction, and particularly in data-driven methods that will aid clinical utility. State-of-the-art machine learning methods may have a unique role to play in this context.

The Importance of Respiratory Rate Monitoring: From Healthcare to Sport and Exercise

Respiratory rate is a fundamental vital sign that is sensitive to different pathological conditions (e.g., adverse cardiac events, pneumonia, and clinical deterioration) and stressors, including emotional stress, cognitive load, heat, cold, physical effort, and exercise-induced fatigue. The sensitivity of respiratory rate to these conditions is superior compared to that of most of the other vital signs, and the abundance of suitable technological solutions measuring respiratory rate has important implications for healthcare, occupational settings, and sport. However, respiratory rate is still too often not routinely monitored in these fields of use. This review presents a multidisciplinary approach to respiratory monitoring, with the aim to improve the development and efficacy of respiratory monitoring services. We have identified thirteen monitoring goals where the use of the respiratory rate is invaluable, and for each of them we have described suitable sensors and techniques to monitor respiratory rate in specific measurement scenarios. We have also provided a physiological rationale corroborating the importance of respiratory rate monitoring and an original multidisciplinary framework for the development of respiratory monitoring services. This review is expected to advance the field of respiratory monitoring and favor synergies between different disciplines to accomplish this goal.

Clinical paradigms and challenges in surface guided radiation therapy: Where do we go from here?

Surface guided radiotherapy (SGRT) is becoming a routine tool for patient positioning for specific clinical sites in many clinics. However, it has not yet gained its full potential in terms of widespread adoption. This vision paper first examines some of the difficulties in transitioning to SGRT before exploring the current and future role of SGRT alongside and in concert with other imaging techniques. Finally, future horizons and innovative ideas that may shape and impact the direction of SGRT going forward are reviewed.

A Non-contact Spirometer with Time-of-Flight Sensor for Assessment of Pulmonary Function

Assessment of pulmonary function is vital for early detection of chronic diseases such as chronic obstructive pulmonary disease (COPD) in home healthcare. However, monitoring of pulmonary function is often omitted owing to the heavy burden that the use of specific medical devices places on the patients. In this study, we developed a non-contact spirometer using a time-of-flight sensor that measures very small displacements caused by chest wall motion during breathing. However, this sensor occasionally failed when estimating the values from breathing waveforms because their shape depends on the subject test experience. As a result, further measurements were required to address motion artifacts. To accomplish high accuracy estimation in the face of these factors, we developed methods to estimate parameters from a part of the waveform and remove outliers from multiple-region measurements. According to laboratory experiments, the proposed system achieved an absolute error of 5.26 % and a correlation coefficient of 0.88. This study also addressed the limitations of depth sensor measurements, thereby contributing to the implementation of high-accuracy COPD screening.

SVM-based waist circumference estimation using Kinect

BACKGROUND AND OBJECTIVE

Conventional anthropometric studies using Kinect depth sensors have concentrated on estimating the distances between two points such as height. This paper deals with a novel waist measurement method using SVM regression, further widening spectrum of Kinect's potential applications. Waist circumference is a key index for the diagnosis of abdominal obesity, which has been linked to metabolic syndromes and other related diseases. Yet, the existing measuring method, tape measure, requires a trained personnel and is therefore costly and time-consuming.

METHODS

A dataset was constructed by recording both 30 frames of Kinect depth image and careful tape measurement of 19 volunteers by a clinical investigator. This paper proposes a new SVM regressor-based approach for estimating waist circumference. A waist curve vector is extracted from a raw depth image using joint information provided by Kinect SDK. To avoid overfitting, a data augmentation technique is devised. The 30 frontal vectors and 30 backside vectors, each sampled for 1 s per person, are combined to form 900 waist curve vectors and a total of 17,100 samples were collected from 19 individuals. On an individual basis, we performed leave-one-out validation using the SVM regressor with the tape measurement-gold standard of waist circumference measurement-values labeled as ground-truth. On an individual basis, we performed leave-one-out validation using the SVM regressor with the tape measurement-gold standard of waist circumference measurement-values labeled as ground-truth.

RESULTS

The mean error of the SVM regressor was 4.62 cm, which was smaller than that of the geometric estimation method. Potential uses are discussed.

CONCLUSIONS

A possible method for measuring waist circumference using a depth sensor is demonstrated through experimentation. Methods for improving accuracy in the future are presented. Combined with other potential applications of Kinect in healthcare setting, the proposed method will pave the way for patient-centric approach of delivering care without laying burdens on patients.

Design and Evaluation of a MEMS Magnetic Field Sensor-Based Respiratory Monitoring and Training System for Radiotherapy

The patient’s respiratory pattern and reproducibility are important factors affecting the accuracy of radiotherapy for lung cancer or liver cancer cases. Therefore, respiration training is required to induce respiration regularity before radiotherapy. However, the need for specialized personnel, space, and time-consuming training represent limitations. To solve these problems, we have developed a respiratory monitoring and training system based on a micro-electro-mechanical-system (MEMS) magnetic sensor. This system consists of a small attaching magnet, a sensor, and a breathing pattern output device. In this study, we evaluated the performance of the signal measurement in the developed system based on the various respiratory cycles, the amplitudes, and the position angles of the magnet and the sensor. The system can provide a more accurate breathing signal graph with lower measurement error and higher spatial resolution than conventional sensor methods by using additional magnet. In addition, it is possible the patient to monitor and train breathing himself by making it easy to carry and use without restriction of time and space.

[Development of Monitoring Method of Respiratory Waveform in Thoracicoabdominal Part Using Web Camera]

Countermeasures against respiratory movement are important for tumors of thorax and abdomen in stereotactic body radiation therapy. In the present paper, a web-camera-based-respiratory monitoring method without contact with patient's body was proposed for respiratory study. Thoracic and abdominal motion images were taken by a web camera, and were analyzed using simple image-processing techniques for obtaining respiratory waveforms. Four motion images with different respiration rate were obtained from resusci anne simulator. Respiration waveforms were estimated from the moving images by the proposed method, and were compared with respiration waveforms obtained by the conventional respiratory monitoring device. That was found to have a strong correlation. In addition, the two waveforms were similar in Bland-Altman method comparison. The proposed method can provide non-contact, non-invasive, simple, and realistic respiratory monitoring system for radiotherapy.