Healthcare applications of single camera markerless motion capture: a scoping review

Background

Single camera markerless motion capture has the potential to facilitate at home movement assessment due to the ease of setup, portability, and affordable cost of the technology. However, it is not clear what the current healthcare applications of single camera markerless motion capture are and what information is being collected that may be used to inform clinical decision making. This review aims to map the available literature to highlight potential use cases and identify the limitations of the technology for clinicians and researchers interested in the collection of movement data.

Survey Methodology

Studies were collected up to 14 January 2022 using Pubmed, CINAHL and SPORTDiscus using a systematic search. Data recorded included the description of the markerless system, clinical outcome measures, and biomechanical data mapped to the International Classification of Functioning, Disability and Health Framework (ICF). Studies were grouped by patient population.

Results

A total of 50 studies were included for data collection. Use cases for single camera markerless motion capture technology were identified for Neurological Injury in Children and Adults; Hereditary/Genetic Neuromuscular Disorders; Frailty; and Orthopaedic or Musculoskeletal groups. Single camera markerless systems were found to perform well in studies involving single plane measurements, such as in the analysis of infant general movements or spatiotemporal parameters of gait, when evaluated against 3D marker-based systems and a variety of clinical outcome measures. However, they were less capable than marker-based systems in studies requiring the tracking of detailed 3D kinematics or fine movements such as finger tracking.

Conclusions

Single camera markerless motion capture offers great potential for extending the scope of movement analysis outside of laboratory settings in a practical way, but currently suffers from a lack of accuracy where detailed 3D kinematics are required for clinical decision making. Future work should therefore focus on improving tracking accuracy of movements that are out of plane relative to the camera orientation or affected by occlusion, such as supination and pronation of the forearm.

Detection of changes in the behaviour of the elderly person

In this paper, we propose a solution for detecting changes in the behaviour of the elderly person based on the monitoring of activities of daily living (ADL). The elderly person's daily routine is characterized by the following five indexes: 1) percentage of time lying down, 2) percentage of time sitting, 3) percentage of time standing, 4) percentage of time absent from home, and 5) number of falls during the day. In our framework, these indexes are computed using characteristics extracted from depth and thermal data. We hypothesize that elderly persons have a well-defined, regular life routine, organized around their environment, habits, and social relations. Then, given the indexes values, a day is defined as routine or non-routine day. Thus, looking for changes of day type allows to detect changes in a person's routine. The method has been tested on a database of depth and thermal images recorded in a nursing home over an 85 days period. These tests proved the reliability of the proposed method.

Interest of the minimum edit distance to detect behaviour change of the elderly person

In this article, a solution to detect the change of behaviour of the elderly person based on the person's activities of daily living is proposed. This work is based on the hypothesis that the person attaches importance to a rhythmic sequence of days and activities per day. The day of the elderly person is described by a succession of activities, and each activity is associated to a posture (lying down, sitting, standing, absent). Postures are estimated from image analysis measured by thermal or depth cameras in order to preserve the anonymity of the person. The change in posture succession is calculated using the minimum edit distance with respect to the routine day. The number of permutations/inversions reflects the change in the person's behaviour. The method was tested on two elderly persons recorded by thermal and depth cameras during 85 days in a retirement home. It is shown that for a person with a life change behaviour, the average number of permutations and interquartile range, before and after changes, are 41 [28], [48] and 57 [55-62] respectively compared to the learned routine day. The Wilcoxon test confirmed the significant difference between these two periods.Clinical Relevance- Monitoring the daily routine provides indicators for detecting changes in the behaviour of an elderly person.

NT-FDS-A Noise Tolerant Fall Detection System Using Deep Learning on Wearable Devices

Given the high prevalence and detrimental effects of unintentional falls in the elderly, fall detection has become a pertinent public concern. A Fall Detection System (FDS) gathers information from sensors to distinguish falls from routine activities in order to provide immediate medical assistance. Hence, the integrity of collected data becomes imperative. Presence of missing values in data, caused by unreliable data delivery, lossy sensors, local interference and synchronization disturbances and so forth, greatly hamper the credibility and usefulness of data making it unfit for reliable fall detection. This paper presents a noise tolerant FDS performing in presence of missing values in data. The work focuses on Deep Learning (DL) particularly Recurrent Neural Networks (RNNs) with an underlying Bidirectional Long Short-Term Memory (BiLSTM) stack to implement FDS based on wearable sensors. The proposed technique is evaluated on two publicly available datasets—SisFall and UP-Fall Detection. Our system produces an accuracy of 97.21% and 97.41%, sensitivity of 96.97% and 99.77% and specificity of 93.18% and 91.45% on SisFall and UP-Fall Detection respectively, thus outperforming the existing state of the art on these benchmark datasets. The resultant outcomes suggest that the ability of BiLSTM to retain long term dependencies from past and future make it an appropriate model choice to handle missing values for wearable fall detection systems.

Detecting falls and estimation of daily habits with depth images using machine learning algorithms

Different approaches have been proposed in the literature to detect the fall of an elderly person. In this paper, we propose a fall detection method based on the classification of parameters extracted from depth images. Three supervised learning methods are compared: decision tree, K-Nearest Neighbors (K-NN) and Random Forests (RF). The methods have been tested on a database of depth images recorded in a nursing home over a period of 43 days. The Random Forests based method yields the best results, achieving 93% sensitivity and 100% specificity when we restrict our study around the bed. Furthermore, this paper also proposes a 37 days follow-up of the person, to try and estimate his or her daily habits.

Walk Identification using a smart carpet and Mel-Frequency Cepstral Coefficient (MFCC) features

We have developed a real-time system for inhome activity monitoring which could be used to assist the independent living of elders. Our system is a context-aware, and unobtrusive floor-based sensor, which recognizes persons walking or falling, monitors their moving activities and stores the data for regular functional assessment. Here we report an in-depth analysis of the waveform generated by the sensors. We studied the analog characteristics of the signals such as power spectrum, pulse width, number of peeks, and signal shape. Then, we used the Mel-Frequency Cepstral Coefficient to extract features which later were utilized in the classification process. We have evaluated the performance of our technique using the dataset collected from 10 subjects who performed walks under different environmental conditions. We were able to use computational features of the generated waveform, by extracting the Mel Frequency Cepstral Coefficients and using computation intelligence to distinguish different people with an average accuracy of 82%.

An Open Architecture to Support Social and Health Services in a Smart TV Environment

OBJECTIVE

To design, implement, and test a solution to provide social and health services for the elderly at home based on smart TV technologies and access to all services.

METHODS

The architecture proposed is based on an open software platform and standard personal computing hardware. This provides great flexibility to develop new applications over the underlying infrastructure or to integrate new devices, for instance to monitor a broad range of vital signs in those cases where home monitoring is required.

RESULTS

An actual system as a proof-of-concept was designed, implemented, and deployed. Applications range from social network clients to vital signs monitoring; from interactive TV contests to conventional online care applications such as medication reminders or telemedicine.

CONCLUSION

In both cases, the results have been very positive, confirming the initial perception of the TV as a convenient, easy-to-use technology to provide social and health care. The TV set is a much more familiar computing interface for most senior users, and as a consequence, smart TVs become a most convenient solution for the design and implementation of applications and services targeted to this user group.

SIGNIFICANCE

This proposal has been tested in real setting with 62 senior people at their homes. Users included both individuals with experience using computers and others reluctant to them.

Triaxial accelerometer-based fall detection method using a self-constructing cascade-AdaBoost-SVM classifier

In this paper, we propose a cascade-AdaBoost-support vector machine (SVM) classifier to complete the triaxial accelerometer-based fall detection method. The method uses the acceleration signals of daily activities of volunteers from a database and calculates feature values. By taking the feature values of a sliding window as an input vector, the cascade-AdaBoost-SVM algorithm can self-construct based on training vectors, and the AdaBoost algorithm of each layer can automatically select several optimal weak classifiers to form a strong classifier, which accelerates effectively the processing speed in the testing phase, requiring only selected features rather than all features. In addition, the algorithm can automatically determine whether to replace the AdaBoost classifier by support vector machine. We used the UCI database for the experiment, in which the triaxial accelerometers are, respectively, worn around the left and right ankles, and on the chest as well as the waist. The results are compared to those of the neural network, support vector machine, and the cascade-AdaBoost classifier. The experimental results show that the triaxial accelerometers around the chest and waist produce optimal results, and our proposed method has the highest accuracy rate and detection rate as well as the lowest false alarm rate.

Fall detection in homes of older adults using the Microsoft Kinect

A method for detecting falls in the homes of older adults using the Microsoft Kinect and a two-stage fall detection system is presented. The first stage of the detection system characterizes a person's vertical state in individual depth image frames, and then segments on ground events from the vertical state time series obtained by tracking the person over time. The second stage uses an ensemble of decision trees to compute a confidence that a fall preceded on a ground event. Evaluation was conducted in the actual homes of older adults, using a combined nine years of continuous data collected in 13 apartments. The dataset includes 454 falls, 445 falls performed by trained stunt actors and nine naturally occurring resident falls. The extensive data collection allows for characterization of system performance under real-world conditions to a degree that has not been shown in other studies. Cross validation results are included for standing, sitting, and lying down positions, near (within 4 m) versus far fall locations, and occluded versus not occluded fallers. The method is compared against five state-of-the-art fall detection algorithms and significantly better results are achieved.

Efficient source separation algorithms for acoustic fall detection using a microsoft kinect

Falls have become a common health problem among older adults. In previous study, we proposed an acoustic fall detection system (acoustic FADE) that employed a microphone array and beamforming to provide automatic fall detection. However, the previous acoustic FADE had difficulties in detecting the fall signal in environments where interference comes from the fall direction, the number of interferences exceeds FADE's ability to handle or a fall is occluded. To address these issues, in this paper, we propose two blind source separation (BSS) methods for extracting the fall signal out of the interferences to improve the fall classification task. We first propose the single-channel BSS by using nonnegative matrix factorization (NMF) to automatically decompose the mixture into a linear combination of several basis components. Based on the distinct patterns of the bases of falls, we identify them efficiently and then construct the interference free fall signal. Next, we extend the single-channel BSS to the multichannel case through a joint NMF over all channels followed by a delay-and-sum beamformer for additional ambient noise reduction. In our experiments, we used the Microsoft Kinect to collect the acoustic data in real-home environments. The results show that in environments with high interference and background noise levels, the fall detection performance is significantly improved using the proposed BSS approaches.

Doppler radar sensor positioning in a fall detection system

Falling is a common health problem for more than a third of the United States population over 65. We are currently developing a Doppler radar based fall detection system that already has showed promising results. In this paper, we study the sensor positioning in the environment with respect to the subject. We investigate three sensor positions, floor, wall and ceiling of the room, in two experimental configurations. Within each system configuration, subjects performed falls towards or across the radar sensors. We collected 90 falls and 341 non falls for the first configuration and 126 falls and 817 non falls for the second one. Radar signature classification was performed using a SVM classifier. Fall detection performance was evaluated using the area under the ROC curves (AUCs) for each sensor deployment. We found that a fall is more likely to be detected if the subject is falling toward or away from the sensor and a ceiling Doppler radar is more reliable for fall detection than a wall mounted one.

Improving automatic sound-based fall detection using iVAT clustering and GA-based feature selection

Falls represent an important health problem for older adults. This issue continues to generate interest in the research and development of fall detection systems. In previous work we proposed an acoustic fall detection system (acoustic-FADE) that employs an 8-microphone circular array to automatically detect falls. Acoustic-FADE has achieved encouraging results: 100% detection at 3% false alarm rate in laboratory tests. In this paper, we use a dataset from previous work to investigate how to further improve AFADE performance. To analyze the relationship between fall and non-fall signatures we used the improved visual assessment of tendency (iVAT) clustering algorithm in conjunction with a nearest neighbor based distance to find the most challenging false alarms. Then, we employed a genetic algorithm (GA) framework to perform feature selection and find the mel-frequency cepstral coefficients (MFCC) that improve the classification performance. We found that using only three MFCC coefficients (1, 28, 29) instead of our previous choice (1,2,3,4,5,6) improves the classification performance.

A microphone array system for automatic fall detection

More than a third of elderly fall each year in the United States. It has been shown that the longer the lie on the floor, the poorer is the outcome of the medical intervention. To reduce delay of the medical intervention, we have developed an acoustic fall detection system (acoustic-FADE) that automatically detects a fall and reports it promptly to the caregiver. Acoustic-FADE consists of a circular microphone array that captures the sounds in a room. When a sound is detected, acoustic-FADE locates the source, enhances the signal, and classifies it as "fall" or "nonfall." The sound source is located using the steered response power with phase transform technique, which has been shown to be robust under noisy environments and resilient to reverberation effects. Signal enhancement is performed by the beamforming technique based on the estimated sound source location. Height information is used to increase the specificity. The mel-frequency cepstral coefficient features computed from the enhanced signal are utilized in the classification process. We have evaluated the performance of acoustic-FADE using simulated fall and nonfall sounds performed by three stunt actors trained to behave like elderly under different environmental conditions. Using a dataset consisting of 120 falls and 120 nonfalls, the acoustic-FADE achieves 100% sensitivity at a specificity of 97%.

Simulated unobtrusive falls detection with multiple persons

One serious issue related to falls among the elderly living at home or in a residential care facility is the "long lie" scenario, which involves being unable to get up from the floor after a fall for 60 min or more. This research uses a simulated environment to investigate the potential effectiveness of using wireless ambient sensors (dual-technology (microwave/infrared) motion detectors and pressure mats) to track the movement of multiple persons and to unobtrusively detect falls when they occur, therefore reducing the rate of occurrence of "long lie" scenarios. A path-finding algorithm (A*) is used to simulate the movement of one or more persons through the residential area. For analysis, the sensor network is represented as an undirected graph, where nodes in the graph represent sensors, and edges between nodes in the graph imply that these sensors share an overlapping physical region in their area of sensitivity. A second undirected graph is used to represent the physical adjacency of the sensors (even where they do not overlap in their monitored regions). These graphical representations enable the tracking of multiple subjects/groups within the environment, by analyzing the sensor activation and adjacency profiles, hence allowing individuals/groups to be isolated when multiple persons are present, and subsequently monitoring falls events. A falls algorithm, based on a heuristic decision tree classifier model, was tested on 15 scenarios, each including one or more persons; three scenarios of activity of daily living, and 12 different types of falls (four types of fall, each with three postfall scenarios). The sensitivity, specificity, and accuracy of the falls algorithm are 100.00%, 77.14%, and 89.33%, respectively.

Acoustic fall detection using a circular microphone array

Falling is a common health problem for elderly. It is reported that more than one third of adults 65 and older fall each year in the United States. To address the problem, we are currently developing an acoustic fall detection system, FADE, which automatically detects a fall and reports it to the caregiver. In a previous version, FADE used a 3-microphone linear array to eliminate the false alarms produced by sounds produced well above the floor level. To improve the fall detection in noisy and reverberant environments, we replaced the linear array by an 8-microphone circular array that can provide a better 3-D estimation of the sound location. Preliminary experiments show that the sound location estimation performed by the circular array is reliable and robust to interference. We obtained encouraging classification results on a pilot dataset with 55 falls and 120 non-fall sounds.

An acoustic fall detector system that uses sound height information to reduce the false alarm rate

More than one third of about 38 million adults 65 and older fall each year in the United States. To address the above problem we propose to develop an acoustic fall detection system (FADE) that will automatically signal a fall to the monitoring caregiver. As opposed to many existent fall detection systems that require the monitored person to wear devices such as accelerometers or gyroscopes at all times, our system is completely unobtrusive by not requiring any wearable devices. To reduce the false alarm rate we employ an array of acoustic sensors to obtain sound source height information. The sound is considered a false alarm if it comes from a source located at a height higher than 2 feet. We tested our system in a pilot study that consisted of a set of 23 falls performed by a stunt actor during six sessions of about 15 minutes each (1.3 hours in total). The actor was previously trained by our nursing collaborators to fall like an elderly person. The use of height information reduced the false alarm hourly rate from 32 to 5 at a 100% fall detection rate.