SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos

A novel soft tissue-integrated kinematic solver for skeletal motion: Validation and applications

BACKGROUND AND OBJECTIVE

Kinematic solvers for human motion analysis, relying on oversimplified joint definitions, face inherent limitations in capturing the true spectrum of skeletal motion. Recent advancements incorporated soft tissue constraints to derive more realistic joint kinematics. However, these methods require marker data input and are computationally expensive, limiting their application to specific joints. This paper proposes a novel kinematic solver that addresses this gap by explicitly accounting for soft tissues, while allowing for accurate and computational efficient modeling across diverse movements and joints.

METHODS

The proposed soft tissue-integrated kinematic solver determines the kinematics by relying on the principle of force balance. In a cascaded iterative way, the position and orientation of each individual segment is updated by minimizing the force residual acting on the segment The latter is solved through a unique way by defining and aligning two point clouds. Accuracy was assessed with three datasets: in-vivo MRI squats (N = 9), in-vitro cadaver CT squat (N = 1), and in-vitro cadaver arm flexion/extension/pro-supination (N = 1). The accuracy was assessed by computing the absolute error on the joint angles and translations and benchmarked against traditional inverse kinematics with a revolute joint as well as two computer vision techniques (OSSO and SKEL).

RESULTS

All experiments showed that with sufficient input data (over 5 rigid bone markers, or skin zones), the primary motion error was almost without exception under 1.5° This outperformed the inverse kinematics with revolute joint (7.29° flexion-extension), OSSO (9.59° flexion-extension) and SKEL (3.19° flexion-extension) methods. The median error on the secondary kinematics for the humeroulnar and ulnoradial joints were below 3.78° and 2.50 mm when driving the motion with skin zones. For the tibiofemoral joints, errors were under 5.39° and 3.5 mm. Computation time was below 30 s per frame.

CONCLUSIONS

The kinematic solver enables exploring all degrees of freedom accurately without compromising computational efficiency. Unlike biomechanical methods which are limited to marker data, the kinematic solver can analyze both marker and skin data.

Motion Capture Technology in Sports Scenarios: A Survey

Motion capture technology plays a crucial role in optimizing athletes' skills, techniques, and strategies by providing detailed feedback on motion data. This article presents a comprehensive survey aimed at guiding researchers in selecting the most suitable motion capture technology for sports science investigations. By comparing and analyzing the characters and applications of different motion capture technologies in sports scenarios, it is observed that cinematography motion capture technology remains the gold standard in biomechanical analysis and continues to dominate sports research applications. Wearable sensor-based motion capture technology has gained significant traction in specialized areas such as winter sports, owing to its reliable system performance. Computer vision-based motion capture technology has made significant advancements in recognition accuracy and system reliability, enabling its application in various sports scenarios, from single-person technique analysis to multi-person tactical analysis. Moreover, the emerging field of multimodal motion capture technology, which harmonizes data from various sources with the integration of artificial intelligence, has proven to be a robust research method for complex scenarios. A comprehensive review of the literature from the past 10 years underscores the increasing significance of motion capture technology in sports, with a notable shift from laboratory research to practical training applications on sports fields. Future developments in this field should prioritize research and technological advancements that cater to practical sports scenarios, addressing challenges such as occlusion, outdoor capture, and real-time feedback.

Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation

We consider a new problem of adapting a human mesh reconstruction model to out-of-domain streaming videos, where the performance of existing SMPL-based models is significantly affected by the distribution shift represented by different camera parameters, bone lengths, backgrounds, and occlusions. We tackle this problem through online adaptation, gradually correcting the model bias during testing. There are two main challenges: First, the lack of 3D annotations increases the training difficulty and results in 3D ambiguities. Second, non-stationary data distribution makes it difficult to strike a balance between fitting regular frames and hard samples with severe occlusions or dramatic changes. To this end, we propose the Dynamic Bilevel Online Adaptation algorithm (DynaBOA). It first introduces the temporal constraints to compensate for the unavailable 3D annotations and leverages a bilevel optimization procedure to address the conflicts between multi-objectives. DynaBOA provides additional 3D guidance by co-training with similar source examples retrieved efficiently despite the distribution shift. Furthermore, it can adaptively adjust the number of optimization steps on individual frames to fully fit hard samples and avoid overfitting regular frames. DynaBOA achieves state-of-the-art results on three out-of-domain human mesh reconstruction benchmarks.

Temporal Pattern Attention for Multivariate Time Series of Tennis Strokes Classification

Human Action Recognition is a challenging task used in many applications. It interacts with many aspects of Computer Vision, Machine Learning, Deep Learning and Image Processing in order to understand human behaviours as well as identify them. It makes a significant contribution to sport analysis, by indicating players' performance level and training evaluation. The main purpose of this study is to investigate how the content of three-dimensional data influences on classification accuracy of four basic tennis strokes: forehand, backhand, volley forehand, and volley backhand. An entire player's silhouette and its combination with a tennis racket were taken into consideration as input to the classifier. Three-dimensional data were recorded using the motion capture system (Vicon Oxford, UK). The Plug-in Gait model consisting of 39 retro-reflective markers was used for the player's body acquisition. A seven-marker model was created for tennis racket capturing. The racket is represented in the form of a rigid body; therefore, all points associated with it changed their coordinates simultaneously. The Attention Temporal Graph Convolutional Network was applied for these sophisticated data. The highest accuracy, up to 93%, was achieved for the data of the whole player's silhouette together with a tennis racket. The obtained results indicated that for dynamic movements, such as tennis strokes, it is necessary to analyze the position of the whole body of the player as well as the racket position.

SPORT KINESIOLOGY BASED ON THE CONCEPT OF HEALTH AND FITNESS

ABSTRACT Introduction: Exercise is the most effective way to improve physical fitness. One can achieve the effect of wellness and fitness through scientific exercise. Running is a relatively common method of physical exercise. It plays a significant role in improving physical fitness. Objective: This study aimed to investigate the characteristics of lower extremity movements during running. The results of this study may provide better exercise planning for runners. Methods: This paper selects several runners as the research subject. The subjects started running after attaching a motion detector sensor patch to their body. Then, this paper collected kinematic data. The kinematic data includes the joint angles and range of motion (ROM) of the hip, knee, and ankle joints. Results: There was no significant difference in the distribution of peak tibial acceleration, plantar pressure, and maximum pressure of athletes under different track materials (P>0.05). There was a significant age difference between the hip and knee joints of the athletes in the overhead stage (P<0.05). Conclusion: There may not be a necessary connection between ground and lower limb impact in running athletes. Through its adjustment, the human body can dampen the load effect of the foot contact surface. Level of evidence II; Therapeutic studies - investigation of treatment outcomes.

Multiscale Human Activity Recognition and Anticipation Network

Deep convolutional neural networks have been leveraged to achieve huge improvements in video understanding and human activity recognition performance in the past decade. However, most existing methods focus on activities that have similar time scales, leaving the task of action recognition on multiscale human behaviors less explored. In this study, a two-stream multiscale human activity recognition and anticipation (MS-HARA) network is proposed, which is jointly optimized using a multitask learning method. The MS-HARA network fuses the two streams of the network using an efficient temporal-channel attention (TCA)-based fusion approach to improve the model's representational ability for both temporal and spatial features. We investigate the multiscale human activities from two basic categories, namely, midterm activities and long-term activities. The network is designed to function as part of a real-time processing framework to support interaction and mutual understanding between humans and intelligent machines. It achieves state-of-the-art results on several datasets for different tasks and different application domains. The midterm and long-term action recognition and anticipation performance, as well as the network fusion, are extensively tested to show the efficiency of the proposed network. The results show that the MS-HARA network can easily be extended to different application domains.

Toward Modeling Psychomotor Performance in Karate Combats Using Computer Vision Pose Estimation

Technological advances enable the design of systems that interact more closely with humans in a multitude of previously unsuspected fields. Martial arts are not outside the application of these techniques. From the point of view of the modeling of human movement in relation to the learning of complex motor skills, martial arts are of interest because they are articulated around a system of movements that are predefined, or at least, bounded, and governed by the laws of Physics. Their execution must be learned after continuous practice over time. Literature suggests that artificial intelligence algorithms, such as those used for computer vision, can model the movements performed. Thus, they can be compared with a good execution as well as analyze their temporal evolution during learning. We are exploring the application of this approach to model psychomotor performance in Karate combats (called kumites), which are characterized by the explosiveness of their movements. In addition, modeling psychomotor performance in a kumite requires the modeling of the joint interaction of two participants, while most current research efforts in human movement computing focus on the modeling of movements performed individually. Thus, in this work, we explore how to apply a pose estimation algorithm to extract the features of some predefined movements of Ippon Kihon kumite (a one-step conventional assault) and compare classification metrics with four data mining algorithms, obtaining high values with them.