Region Aware Video Object Segmentation With Deep Motion Modeling

Current semi-supervised video object segmentation (VOS) methods often employ the entire features of one frame to predict object masks and update memory. This introduces significant redundant computations. To reduce redundancy, we introduce a Region Aware Video Object Segmentation (RAVOS) approach, which predicts regions of interest (ROIs) for efficient object segmentation and memory storage. RAVOS includes a fast object motion tracker to predict object ROIs in the next frame. For efficient segmentation, object features are extracted based on the ROIs, and an object decoder is designed for object-level segmentation. For efficient memory storage, we propose motion path memory to filter out redundant context by memorizing the features within the motion path of objects. In addition to RAVOS, we also propose a large-scale occluded VOS dataset, dubbed OVOS, to benchmark the performance of VOS models under occlusions. Evaluation on DAVIS and YouTube-VOS benchmarks and our new OVOS dataset show that our method achieves state-of-the-art performance with significantly faster inference time, e.g., 86.1 J & F at 42 FPS on DAVIS and 84.4 J & F at 23 FPS on YouTube-VOS. Project page: ravos.netlify.app.

MH6D: Multi-Hypothesis Consistency Learning for Category-Level 6-D Object Pose Estimation

Six-degree-of-freedom (6DoF) object pose estimation is a crucial task for virtual reality and accurate robotic manipulation. Category-level 6DoF pose estimation has recently become popular as it improves generalization to a complete category of objects. However, current methods focus on data-driven differential learning, which makes them highly dependent on the quality of the real-world labeled data and limits their ability to generalize to unseen objects. To address this problem, we propose multi-hypothesis (MH) consistency learning (MH6D) for category-level 6-D object pose estimation without using real-world training data. MH6D uses a parallel consistency learning structure, alleviating the uncertainty problem of single-shot feature extraction and promoting self-adaptation of domain to reduce the synthetic-to-real domain gap. Specifically, three randomly sampled pose transformations are first performed in parallel on the input point cloud. An attention-guided category-level 6-D pose estimation network with channel attention (CA) and global feature cross-attention (GFCA) modules is then proposed to estimate the three hypothesized 6-D object poses by extracting and fusing the global and local features effectively. Finally, we propose a novel loss function that considers both the process and the final result information allowing MH6D to perform robust consistency learning. We conduct experiments under two different training data settings (i.e., only synthetic data and synthetic and real-world data) to verify the generalization ability of MH6D. Extensive experiments on benchmark datasets demonstrate that MH6D achieves state-of-the-art (SOTA) performance, outperforming most data-driven methods even without using any real-world data. The code is available at https://github.com/CNJianLiu/MH6D.

DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks

Convolutional neural network (CNN) architectures are generally heavy on memory and computational requirements which make them infeasible for embedded systems with limited hardware resources. We propose dual convolutional kernels (DualConv) for constructing lightweight deep neural networks. DualConv combines 3×3 and 1×1 convolutional kernels to process the same input feature map channels simultaneously and exploits the group convolution technique to efficiently arrange convolutional filters. DualConv can be employed in any CNN model such as VGG-16 and ResNet-50 for image classification, you only look once (YOLO) and R-CNN for object detection, or fully convolutional network (FCN) for semantic segmentation. In this work, we extensively test DualConv for classification since these network architectures form the backbone for many other tasks. We also test DualConv for image detection on YOLO-V3. Experimental results show that, combined with our structural innovations, DualConv significantly reduces the computational cost and number of parameters of deep neural networks while surprisingly achieving slightly higher accuracy than the original models in some cases. We use DualConv to further reduce the number of parameters of the lightweight MobileNetV2 by 54% with only 0.68% drop in accuracy on CIFAR-100 dataset. When the number of parameters is not an issue, DualConv increases the accuracy of MobileNetV1 by 4.11% on the same dataset. Furthermore, DualConv significantly improves the YOLO-V3 object detection speed and improves its accuracy by 4.4% on PASCAL visual object classes (VOC) dataset.

The expert eye? An inter-rater comparison of elite tennis serve kinematics and performance

This study examined the reliability of expert tennis coaches/biomechanists to qualitatively assess selected features of the serve with the aid of two-dimensional (2D) video replays. Two expert high-performance coaches rated the serves of 150 male and 150 female players across three different age groups from two different camera viewing angles. Serve performance was rated across 13 variables that represented commonly investigated and coached (serve) mechanics using a 1-7 Likert rating scale. A total of 7800 ratings were performed. The reliability of the experts' ratings was assessed using a Krippendorffs alpha. Strong agreement was shown across all age groups and genders when the experts rated the overall serve score (0.727-0.924), power or speed of the serve (0.720-0.907), rhythm (0.744-0.944), quality of the trunk action (0.775-1.000), leg drive (0.731-0.959) and the likelihood of back injury (0.703-0.934). They encountered greater difficulty in consistently rating shoulder internal rotation speed (0.688-0.717). In high-performance settings, the desire for highly precise measurement and large data sets powered by new technologies, is commonplace but this study revealed that tennis experts, through the use of 2D video, can reliably rate important mechanical features of the game's most important shot, the serve.

Machine learning to predict myocardial injury and death after non-cardiac surgery

Myocardial injury due to ischaemia within 30 days of non-cardiac surgery is prognostically relevant. We aimed to determine the discrimination, calibration, accuracy, sensitivity and specificity of single-layer and multiple-layer neural networks for myocardial injury and death within 30 postoperative days. We analysed data from 24,589 participants in the Vascular Events in Non-cardiac Surgery Patients Cohort Evaluation study. Validation was performed on a randomly selected subset of the study population. Discrimination for myocardial injury by single-layer vs. multiple-layer models generated areas (95%CI) under the receiver operating characteristic curve of: 0.70 (0.69-0.72) vs. 0.71 (0.70-0.73) with variables available before surgical referral, p < 0.001; 0.73 (0.72-0.75) vs. 0.75 (0.74-0.76) with additional variables available on admission, but before surgery, p < 0.001; and 0.76 (0.75-0.77) vs. 0.77 (0.76-0.78) with the addition of subsequent variables, p < 0.001. Discrimination for death by single-layer vs. multiple-layer models generated areas (95%CI) under the receiver operating characteristic curve of: 0.71 (0.66-0.76) vs. 0.74 (0.71-0.77) with variables available before surgical referral, p = 0.04; 0.78 (0.73-0.82) vs. 0.83 (0.79-0.86) with additional variables available on admission but before surgery, p = 0.01; and 0.87 (0.83-0.89) vs. 0.87 (0.85-0.90) with the addition of subsequent variables, p = 0.52. The accuracy of the multiple-layer model for myocardial injury and death with all variables was 70% and 89%, respectively.

Mesh Convolution With Continuous Filters for 3-D Surface Parsing

Geometric feature learning for 3-D surfaces is critical for many applications in computer graphics and 3-D vision. However, deep learning currently lags in hierarchical modeling of 3-D surfaces due to the lack of required operations and/or their efficient implementations. In this article, we propose a series of modular operations for effective geometric feature learning from 3-D triangle meshes. These operations include novel mesh convolutions, efficient mesh decimation, and associated mesh (un)poolings. Our mesh convolutions exploit spherical harmonics as orthonormal bases to create continuous convolutional filters. The mesh decimation module is graphics processing unit (GPU)-accelerated and able to process batched meshes on-the-fly, while the (un)pooling operations compute features for upsampled/downsampled meshes. We provide an open-source implementation of these operations, collectively termed Picasso. Picasso supports heterogeneous mesh batching and processing. Leveraging its modular operations, we further contribute a novel hierarchical neural network for perceptual parsing of 3-D surfaces, named PicassoNet ++ . It achieves highly competitive performance for shape analysis and scene segmentation on prominent 3-D benchmarks. The code, data, and trained models are available at https://github.com/EnyaHermite/Picasso.

A Privacy-Preserving Framework Using Homomorphic Encryption for Smart Metering Systems

Smart metering systems (SMSs) have been widely used by industrial users and residential customers for purposes such as real-time tracking, outage notification, quality monitoring, load forecasting, etc. However, the consumption data it generates can violate customers' privacy through absence detection or behavior recognition. Homomorphic encryption (HE) has emerged as one of the most promising methods to protect data privacy based on its security guarantees and computability over encrypted data. However, SMSs have various application scenarios in practice. Consequently, we used the concept of trust boundaries to help design HE solutions for privacy protection under these different scenarios of SMSs. This paper proposes a privacy-preserving framework as a systematic privacy protection solution for SMSs by implementing HE with trust boundaries for various SMS scenarios. To show the feasibility of the proposed HE framework, we evaluated its performance on two computation metrics, summation and variance, which are often used for billing, usage predictions, and other related tasks. The security parameter set was chosen to provide a security level of 128 bits. In terms of performance, the aforementioned metrics could be computed in 58,235 ms for summation and 127,423 ms for variance, given a sample size of 100 households. These results indicate that the proposed HE framework can protect customer privacy under varying trust boundary scenarios in SMS. The computational overhead is acceptable from a cost-benefit perspective while ensuring data privacy.

Attack to Fool and Explain Deep Networks

Deep visual models are susceptible to adversarial perturbations to inputs. Although these signals are carefully crafted, they still appear noise-like patterns to humans. This observation has led to the argument that deep visual representation is misaligned with human perception. We counter-argue by providing evidence of human-meaningful patterns in adversarial perturbations. We first propose an attack that fools a network to confuse a whole category of objects (source class) with a target label. Our attack also limits the unintended fooling by samples from non-sources classes, thereby circumscribing human-defined semantic notions for network fooling. We show that the proposed attack not only leads to the emergence of regular geometric patterns in the perturbations, but also reveals insightful information about the decision boundaries of deep models. Exploring this phenomenon further, we alter the 'adversarial' objective of our attack to use it as a tool to 'explain' deep visual representation. We show that by careful channeling and projection of the perturbations computed by our method, we can visualize a model's understanding of human-defined semantic notions. Finally, we exploit the explanability properties of our perturbations to perform image generation, inpainting and interactive image manipulation by attacking adversarialy robust 'classifiers'. In all, our major contribution is a novel pragmatic adversarial attack that is subsequently transformed into a tool to interpret the visual models. The article also makes secondary contributions in terms of establishing the utility of our attack beyond the adversarial objective with multiple interesting applications.

979 Emphysematous Pyelonephritis in a Diabetic Patient with Horseshoe Kidney: Case Study and Literature Review

We present to you the case of a 54-year-old female with a background of horseshoe kidney presenting with a fever, right-sided flank pain, and raised inflammatory markers. This presentation was on a background of recurrent urinary tract infections which were managed conservatively with antibiotics. CT-KUB revealed pockets of gas in both kidneys and within the pelvis, with wall thickening of the left renal pelvis. She subsequently received broad spectrum antibiotics and a suprapubic catheter. Bilateral JJ stents were inserted, and ultrasound was performed in the days following to rule out hydronephrosis. Subsequently, a nephrostomy tube was inserted. The patient clinically improved over time. Horseshoe kidney is a rare anomaly, presenting multiple challenges when managing emphysematous pyelonephritis in these patients.

Emphysematous pyelonephritis (EPN) is a necrotising renal infection that leads to the formation of gas in the renal parenchyma, collecting ducts, and surrounding tissue. In the past, multiple eponyms have been used to describe these conditions: ‘renal emphysema’, ‘pneumonephritis’. The pathophysiology of this benign condition is multifactorial. The gas formation that occurs can be either focal or diffuse and has the capability of tracking into perinephric and paranephric spaces.

Approximately 90% of these patients have an underlying diagnosis of uncontrolled diabetes mellitus with pathogens that can produce gas, whilst 10% have urinary tract obstructions. EPN is rare, and the occurrence in a diabetic patient with horseshoe kidney has rarely been reported. The presence of a horseshoe kidney makes management more difficult. Few cases have been reported in the literature.

Fully Convolutional Network-Based Self-Supervised Learning for Semantic Segmentation

Although deep learning has achieved great success in many computer vision tasks, its performance relies on the availability of large datasets with densely annotated samples. Such datasets are difficult and expensive to obtain. In this article, we focus on the problem of learning representation from unlabeled data for semantic segmentation. Inspired by two patch-based methods, we develop a novel self-supervised learning framework by formulating the jigsaw puzzle problem as a patch-wise classification problem and solving it with a fully convolutional network. By learning to solve a jigsaw puzzle comprising 25 patches and transferring the learned features to semantic segmentation task, we achieve a 5.8% point improvement on the Cityscapes dataset over the baseline model initialized from random values. It is noted that we use only about 1/6 training images of Cityscapes in our experiment, which is designed to imitate the real cases where fully annotated images are usually limited to a small number. We also show that our self-supervised learning method can be applied to different datasets and models. In particular, we achieved competitive performance with the state-of-the-art methods on the PASCAL VOC2012 dataset using significantly fewer time costs on pretraining.

Adversarial Attack on Skeleton-Based Human Action Recognition

Deep learning models achieve impressive performance for skeleton-based human action recognition. Graph convolutional networks (GCNs) are particularly suitable for this task due to the graph-structured nature of skeleton data. However, the robustness of these models to adversarial attacks remains largely unexplored due to their complex spatiotemporal nature that must represent sparse and discrete skeleton joints. This work presents the first adversarial attack on skeleton-based action recognition with GCNs. The proposed targeted attack, termed constrained iterative attack for skeleton actions (CIASA), perturbs joint locations in an action sequence such that the resulting adversarial sequence preserves the temporal coherence, spatial integrity, and the anthropomorphic plausibility of the skeletons. CIASA achieves this feat by satisfying multiple physical constraints and employing spatial skeleton realignments for the perturbed skeletons along with regularization of the adversarial skeletons with generative networks. We also explore the possibility of semantically imperceptible localized attacks with CIASA and succeed in fooling the state-of-the-art skeleton action recognition models with high confidence. CIASA perturbations show high transferability in black-box settings. We also show that the perturbed skeleton sequences are able to induce adversarial behavior in the RGB videos created with computer graphics. A comprehensive evaluation with NTU and Kinetics data sets ascertains the effectiveness of CIASA for graph-based skeleton action recognition and reveals the imminent threat to the spatiotemporal deep learning tasks in general.

Autoencoded deep features for semi-automatic, weakly supervised physiological signal labelling

BACKGROUND AND AIMS

Machine Learning is transforming data processing in medical research and clinical practice. Missing data labels are a common limitation to training Machine Learning models. To overcome missing labels in a large dataset of microneurography recordings, a novel autoencoder based semi-supervised, iterative group-labelling methodology was developed.

METHODS

Autoencoders were systematically optimised to extract features from a dataset of 478621 signal excerpts from human microneurography recordings. Selected features were clusters with k-means and randomly selected representations of the corresponding original signals labelled as valid or non-valid muscle sympathetic nerve activity (MSNA) bursts in an iterative, purifying procedure by an expert rater. A deep neural network was trained based on the fully labelled dataset.

RESULTS

Three autoencoders, two based on fully connected neural networks and one based on convolutional neural network, were chosen for feature learning. Iterative clustering followed by labelling of complete clusters resulted in all 478621 signal peak excerpts being labelled as valid or non-valid within 13 iterations. Neural networks trained with the labelled dataset achieved, in a cross validation step with a testing dataset not included in training, on average 93.13% accuracy and 91% area under the receiver operating curve (AUC ROC).

DISCUSSION

The described labelling procedure enabled efficient labelling of a large dataset of physiological signal based on expert ratings. The procedure based on autoencoders may be broadly applicable to a wide range of datasets without labels that require expert input and may be utilised for Machine Learning applications if weak-labels were available.

An investigation of a novel broad autism phenotype: increased facial masculinity among parents of children on the autism spectrum

The broad autism phenotype commonly refers to sub-clinical levels of autistic-like behaviour and cognition presented in biological relatives of autistic people. In a recent study, we reported findings suggesting that the broad autism phenotype may also be expressed in facial morphology, specifically increased facial masculinity. Increased facial masculinity has been reported among autistic children, as well as their non-autistic siblings. The present study builds on our previous findings by investigating the presence of increased facial masculinity among non-autistic parents of autistic children. Using a previously established method, a 'facial masculinity score' and several facial distances were calculated for each three-dimensional facial image of 192 parents of autistic children (58 males, 134 females) and 163 age-matched parents of non-autistic children (50 males, 113 females). While controlling for facial area and age, significantly higher masculinity scores and larger (more masculine) facial distances were observed in parents of autistic children relative to the comparison group, with effect sizes ranging from small to medium (0.16 ≤ d ≤ .41), regardless of sex. These findings add to an accumulating evidence base that the broad autism phenotype is expressed in physical characteristics and suggest that both maternal and paternal pathways are implicated in masculinized facial morphology.

Fast ORB-SLAM Without Keypoint Descriptors

Indirect methods for visual SLAM are gaining popularity due to their robustness to environmental variations. ORB-SLAM2 (Mur-Artal and Tardós, 2017) is a benchmark method in this domain, however, it consumes significant time for computing descriptors that never get reused unless a frame is selected as a keyframe. To overcome these problems, we present FastORB-SLAM which is light-weight and efficient as it tracks keypoints between adjacent frames without computing descriptors. To achieve this, a two stage descriptor-independent keypoint matching method is proposed based on sparse optical flow. In the first stage, we predict initial keypoint correspondences via a simple but effective motion model and then robustly establish the correspondences via pyramid-based sparse optical flow tracking. In the second stage, we leverage the constraints of the motion smoothness and epipolar geometry to refine the correspondences. In particular, our method computes descriptors only for keyframes. We test FastORB-SLAM on TUM and ICL-NUIM RGB-D datasets and compare its accuracy and efficiency to nine existing RGB-D SLAM methods. Qualitative and quantitative results show that our method achieves state-of-the-art accuracy and is about twice as fast as the ORB-SLAM2.

Automatic data extraction from 24 hour blood pressure measurement reports of a large multicenter clinical trial

BACKGROUND AND OBJECTIVES

Ambulatory blood pressure monitoring (ABPM) is usually reported in descriptive values such as circadian averages and standard deviations. Making use of the original, individual blood pressure measurements may be advantageous, particularly for research purposes, as this increases the flexibility of the analytical process, enables alternative statistical analyses and provide novel insights. Here we describe the development of a new multistep, hierarchical data extraction algorithm to collect raw data from .pdf reports and text files as part of a large multi-center clinical study.

METHODS

Original reports were saved in a nested file system, from which they were automatically extracted, read and saved into databases with custom made programs written in Python 3. Data were further processed, cleaned and relevant descriptive statistics such as averages and standard deviations calculated according to a variety of definitions of day- and night-time. Additionally, data control mechanisms for manual review of the data and programmatic auto-detection of extraction errors was implemented as part of the project.

RESULTS

The developed algorithm extracted 97% of the data automatically, the missing data consisted mostly of reports that were saved incorrectly or not formatted in the specified way. Manual checks comparing samples of the extracted data to original reports indicated a high level of accuracy of the extracted data, no errors introduced due to flaws in the extraction software were detected in the extracted dataset.

CONCLUSIONS

The developed multistep, hierarchical data extraction algorithm facilitated collection from different file formats and paired with database cleaning and data processing steps led to an effective and accurate assembly of raw ABPM data for further and adjustable analyses. Manual work was minimized while data quality was ensured with standardized, reproducible procedures.

K-means panning - Developing a new standard in automated MSNA signal recognition with a weakly supervised learning approach

BACKGROUND

Accessibility of labelled datasets is often a key limitation for the application of Machine Learning in clinical research. A novel semi-automated weak-labelling approach based on unsupervised clustering was developed to classify a large dataset of microneurography signals and subsequently used to train a Neural Network to reproduce the labelling process.

METHODS

Clusters of microneurography signals were created with k-means and then labelled in terms of the validity of the signals contained in each cluster. Only purely positive or negative clusters were labelled, whereas clusters with mixed content were passed on to the next iteration of the algorithm to undergo another cycle of unsupervised clustering and labelling of the clusters. After several iterations of this process, only pure labelled clusters remained which were used to train a Deep Neural Network.

RESULTS

Overall, 334,548 individual signal peaks form the integrated data were extracted and more than 99.99% of the data was labelled in six iterations of this novel application of weak labelling with the help of a domain expert. A Deep Neural Network trained based on this dataset achieved consistent accuracies above 95%.

DISCUSSION

Data extraction and the novel iterative approach of labelling unsupervised clusters enabled creation of a large, labelled dataset combining unsupervised learning and expert ratings of signal-peaks on cluster basis in a time effective manner. Further research is needed to validate the methodology and employ it on other types of physiologic data for which it may enable efficient generation of large labelled datasets.

Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds

We propose a spherical kernel for efficient graph convolution of 3D point clouds. Our metric-based kernels systematically quantize the local 3D space to identify distinctive geometric relationships in the data. Similar to the regular grid CNN kernels, the spherical kernel maintains translation-invariance and asymmetry properties, where the former guarantees weight sharing among similar local structures in the data and the latter facilitates fine geometric learning. The proposed kernel is applied to graph neural networks without edge-dependent filter generation, making it computationally attractive for large point clouds. In our graph networks, each vertex is associated with a single point location and edges connect the neighborhood points within a defined range. The graph gets coarsened in the network with farthest point sampling. Analogous to the standard CNNs, we define pooling and unpooling operations for our network. We demonstrate the effectiveness of the proposed spherical kernel with graph neural networks for point cloud classification and semantic segmentation using ModelNet, ShapeNet, RueMonge2014, ScanNet and S3DIS datasets. The source code and the trained models can be downloaded from https://github.com/hlei-ziyan/SPH3D-GCN.

Facial asymmetry in parents of children on the autism spectrum

Greater facial asymmetry has been consistently found in children with autism spectrum disorder (ASD) relative to children without ASD. There is substantial evidence that both facial structure and the recurrence of ASD diagnosis are highly heritable within a nuclear family. Furthermore, sub-clinical levels of autistic-like behavioural characteristics have also been reported in first-degree relatives of individuals with ASD, commonly known as the 'broad autism phenotype'. Therefore, the aim of the current study was to examine whether a broad autism phenotype expresses as facial asymmetry among 192 biological parents of autistic individuals (134 mothers) compared to those of 163 age-matched adults without a family history of ASD (113 females). Using dense surface-modelling techniques on three dimensional facial images, we found evidence for greater facial asymmetry in parents of autistic individuals compared to age-matched adults in the comparison group (p = 0.046, d = 0.21 [0.002, 0.42]). Considering previous findings and the current results, we conclude that facial asymmetry expressed in the facial morphology of autistic children may be related to heritability factors. LAY ABSTRACT: In a previous study, we showed that autistic children presented with greater facial asymmetry than non-autistic children. In the current study, we examined the amount of facial asymmetry shown on three-dimensional facial images of 192 parents of autistic children compared to a control group consisting of 163 similarly aged adults with no known history of autism. Although parents did show greater levels of facial asymmetry than those in the control group, this effect is statistically small. We concluded that the facial asymmetry previously found in autistic children may be related to genetic factors.

Machine learning powered tools for automated analysis of muscle sympathetic nerve activity recordings

Automated analysis and quantification of physiological signals in clinical practice and medical research can reduce manual labor, increase efficiency, and provide more objective, reproducible results. To build a novel platform for the analysis of muscle sympathetic nerve activity (MSNA), we employed state-of-the-art data processing and machine learning applications. Data processing methods for integrated MSNA recordings were developed to evaluate signals regarding the overall quality of the signal, the validity of individual signal peaks regarding the potential to be MSNA bursts and the timing of their occurrence. An overall probability score was derived from this flexible platform to evaluate each individual signal peak automatically. Overall, three deep neural networks were designed and trained to validate individual signal peaks randomly sampled from recordings representing only electrical noise and valid microneurography recordings. A novel data processing method for the whole signal was developed to differentiate between periods of valid MSNA signal recordings and periods in which the signal was not available or lost due to involuntary movement of the recording electrode. A probabilistic model for timing of the signal bursts was implemented as part of the system. Machine Learning algorithms and data processing tools were implemented to replicate the complex decision-making process of manual MSNA analysis. Validation of manual MSNA analysis including intra- and inter-rater validity and a comparison with automated MSNA tools is required. The developed toolbox for automated MSNA analysis can be extended in a flexible way to include algorithms based on other datasets.

Neuroaxial Infantile Hemangiomas: Imaging Manifestations and Association with Hemangioma Syndromes

BACKGROUND AND PURPOSE

Infantile hemangiomas are common lesions in the pediatric population; in rare cases, an infantile hemangioma can be detected along the neural axis. The purposes of our study included determination of the incidence, location, and imaging appearance of neuroaxial infantile hemangiomas and their syndromic association. We also assessed additional features of cerebral and cardiovascular anomalies that may be associated with neuroaxial lesions.

MATERIALS AND METHODS

A retrospective cohort study was performed, searching the radiology database for patients with segmental infantile hemangiomas referred for assessment of possible hemangioma syndromes. We retrospectively reviewed brain and spine MR imaging studies, with particular attention paid to neuroaxial vascular lesions, as well as the relevant clinical data. Neuroaxial hemangioma imaging findings were described, and comparison of segmental cutaneous infantile hemangioma location with the imaging findings was performed in patients with confirmed hemangioma syndromes and in patients with isolated skin infantile hemangioma.

RESULTS

Ninety-five patients with segmental infantile hemangioma were included in the study, 42 of whom had a hemangioma syndrome; of those, 41 had posterior fossa brain malformations, hemangioma, arterial lesions, cardiac abnormalities, and eye abnormalities (PHACE) syndrome and 1 had diffuse neonatal hemangiomatosis. Neuroaxial involvement was detected in 20/42 patients (48%) with hemangioma syndromes and in no subjects with isolated segmental infantile hemangioma (P < .001). The most common intracranial hemangioma location was within the ipsilateral internal auditory canal (83%).

CONCLUSIONS

Many pediatric patients with segmental infantile hemangioma in the setting of hemangioma syndromes, especially those with PHACE, had neuroaxial hemangiomas. This finding may potentially lead to requiring additional clinical evaluation and management of these patients.

Brief Report: Facial Asymmetry and Autistic-Like Traits in the General Population

Atypical facial morphology, particularly increased facial asymmetry, has been identified in some individuals with Autism Spectrum Conditions (ASC). Many cognitive, behavioural and biological features associated with ASC also occur on a continuum in the general population. The aim of the present study was to examine subthreshold levels of autistic traits and facial morphology in non-autistic individuals. Facial asymmetry was measured using three-dimensional facial photogrammetry, and the Autism-spectrum Quotient was used to measure autistic-like traits in a community-ascertained sample of young adults (n = 289). After accounting for covariates, there were no significant associations observed between autistic-like traits and facial asymmetry, suggesting that any potential facial morphology differences linked to ASC may be limited to the clinical condition.

A Comparison of Three Neural Network Approaches for Estimating Joint Angles and Moments from Inertial Measurement Units

The application of artificial intelligence techniques to wearable sensor data may facilitate accurate analysis outside of controlled laboratory settings—the holy grail for gait clinicians and sports scientists looking to bridge the lab to field divide. Using these techniques, parameters that are difficult to directly measure in-the-wild, may be predicted using surrogate lower resolution inputs. One example is the prediction of joint kinematics and kinetics based on inputs from inertial measurement unit (IMU) sensors. Despite increased research, there is a paucity of information examining the most suitable artificial neural network (ANN) for predicting gait kinematics and kinetics from IMUs. This paper compares the performance of three commonly employed ANNs used to predict gait kinematics and kinetics: multilayer perceptron (MLP); long short-term memory (LSTM); and convolutional neural networks (CNN). Overall high correlations between ground truth and predicted kinematic and kinetic data were found across all investigated ANNs. However, the optimal ANN should be based on the prediction task and the intended use-case application. For the prediction of joint angles, CNNs appear favourable, however these ANNs do not show an advantage over an MLP network for the prediction of joint moments. If real-time joint angle and joint moment prediction is desirable an LSTM network should be utilised.

Predicting sleep apnea from three-dimensional face photography

STUDY OBJECTIVES

Craniofacial anatomy is recognized as an important predisposing factor in the pathogenesis of obstructive sleep apnea (OSA). This study used three-dimensional (3D) facial surface analysis of linear and geodesic (shortest line between points over a curved surface) distances to determine the combination of measurements that best predicts presence and severity of OSA.

METHODS

3D face photographs were obtained in 100 adults without OSA (apnea-hypopnea index [AHI] < 5 events/h), 100 with mild OSA (AHI 5 to < 15 events/h), 100 with moderate OSA (AHI 15 to < 30 events/h), and 100 with severe OSA (AHI ≥ 30 events/h). Measurements of linear distances and angles, and geodesic distances were obtained between 24 anatomical landmarks from the 3D photographs. The accuracy with which different combinations of measurements could classify an individual as having OSA or not was assessed using linear discriminant analyses and receiver operating characteristic analyses. These analyses were repeated using different AHI thresholds to define presence of OSA.

RESULTS

Relative to linear measurements, geodesic measurements of craniofacial anatomy improved the ability to identify individuals with and without OSA (classification accuracy 86% and 89% respectively, P < .01). A maximum classification accuracy of 91% was achieved when linear and geodesic measurements were combined into a single predictive algorithm. Accuracy decreased when using AHI thresholds ≥ 10 events/h and ≥ 15 events/h to define OSA although greatest accuracy was always achieved using a combination of linear and geodesic distances.

CONCLUSIONS

This study suggests that 3D photographs of the face have predictive value for OSA and that geodesic measurements enhance this capacity.

Assessing the Capability and Potential of LiDAR for Weed Detection

Conventional methods of uniformly spraying fields to combat weeds, requires large herbicide inputs at significant cost with impacts on the environment. More focused weed control methods such as site-specific weed management (SSWM) have become popular but require methods to identify weed locations. Advances in technology allows the potential for automated methods such as drone, but also ground-based sensors for detecting and mapping weeds. In this study, the capability of Light Detection and Ranging (LiDAR) sensors were assessed to detect and locate weeds. For this purpose, two trials were performed using artificial targets (representing weeds) at different heights and diameter to understand the detection limits of a LiDAR. The results showed the detectability of the target at different scanning distances from the LiDAR was directly influenced by the size of the target and its orientation toward the LiDAR. A third trial was performed in a wheat plot where the LiDAR was used to scan different weed species at various heights above the crop canopy, to verify the capacity of the stationary LiDAR to detect weeds in a field situation. The results showed that 100% of weeds in the wheat plot were detected by the LiDAR, based on their height differences with the crop canopy.

Deep Affinity Network for Multiple Object Tracking

Multiple Object Tracking (MOT) plays an important role in solving many fundamental problems in video analysis and computer vision. Most MOT methods employ two steps: Object Detection and Data Association. The first step detects objects of interest in every frame of a video, and the second establishes correspondence between the detected objects in different frames to obtain their tracks. Object detection has made tremendous progress in the last few years due to deep learning. However, data association for tracking still relies on hand crafted constraints such as appearance, motion, spatial proximity, grouping etc. to compute affinities between the objects in different frames. In this paper, we harness the power of deep learning for data association in tracking by jointly modeling object appearances and their affinities between different frames in an end-to-end fashion. The proposed Deep Affinity Network (DAN) learns compact, yet comprehensive features of pre-detected objects at several levels of abstraction, and performs exhaustive pairing permutations of those features in any two frames to infer object affinities. DAN also accounts for multiple objects appearing and disappearing between video frames. We exploit the resulting efficient affinity computations to associate objects in the current frame deep into the previous frames for reliable on-line tracking. Our technique is evaluated on popular multiple object tracking challenges MOT15, MOT17 and UA-DETRAC. Comprehensive benchmarking under twelve evaluation metrics demonstrates that our approach is among the best performing techniques on the leader board for these challenges. The open source implementation of our work is available at https://github.com/shijieS/SST.git.

Self-Supervised Learning to Detect Key Frames in Videos

Detecting key frames in videos is a common problem in many applications such as video classification, action recognition and video summarization. These tasks can be performed more efficiently using only a handful of key frames rather than the full video. Existing key frame detection approaches are mostly designed for supervised learning and require manual labelling of key frames in a large corpus of training data to train the models. Labelling requires human annotators from different backgrounds to annotate key frames in videos which is not only expensive and time consuming but also prone to subjective errors and inconsistencies between the labelers. To overcome these problems, we propose an automatic self-supervised method for detecting key frames in a video. Our method comprises a two-stream ConvNet and a novel automatic annotation architecture able to reliably annotate key frames in a video for self-supervised learning of the ConvNet. The proposed ConvNet learns deep appearance and motion features to detect frames that are unique. The trained network is then able to detect key frames in test videos. Extensive experiments on UCF101 human action and video summarization VSUMM datasets demonstrates the effectiveness of our proposed method.

Structural Similarity Loss for Learning to Fuse Multi-Focus Images

Convolutional neural networks have recently been used for multi-focus image fusion. However, some existing methods have resorted to adding Gaussian blur to focused images, to simulate defocus, thereby generating data (with ground-truth) for supervised learning. Moreover, they classify pixels as 'focused' or 'defocused', and use the classified results to construct the fusion weight maps. This then necessitates a series of post-processing steps. In this paper, we present an end-to-end learning approach for directly predicting the fully focused output image from multi-focus input image pairs. The suggested approach uses a CNN architecture trained to perform fusion, without the need for ground truth fused images. The CNN exploits the image structural similarity (SSIM) to calculate the loss, a metric that is widely accepted for fused image quality evaluation. What is more, we also use the standard deviation of a local window of the image to automatically estimate the importance of the source images in the final fused image when designing the loss function. Our network can accept images of variable sizes and hence, we are able to utilize real benchmark datasets, instead of simulated ones, to train our network. The model is a feed-forward, fully convolutional neural network that can process images of variable sizes during test time. Extensive evaluation on benchmark datasets show that our method outperforms, or is comparable with, existing state-of-the-art techniques on both objective and subjective benchmarks.

Relation Graph Network for 3D Object Detection in Point Clouds

Convolutional Neural Networks (CNNs) have emerged as a powerful tool for object detection in 2D images. However, their power has not been fully realised for detecting 3D objects directly in point clouds without conversion to regular grids. Moreover, existing state-of-the-art 3D object detection methods aim to recognize objects individually without exploiting their relationships during learning or inference. In this article, we first propose a strategy that associates the predictions of direction vectors with pseudo geometric centers, leading to a win-win solution for 3D bounding box candidates regression. Secondly, we propose point attention pooling to extract uniform appearance features for each 3D object proposal, benefiting from the learned direction features, semantic features and spatial coordinates of the object points. Finally, the appearance features are used together with the position features to build 3D object-object relationship graphs for all proposals to model their co-existence. We explore the effect of relation graphs on proposals' appearance feature enhancement under supervised and unsupervised settings. The proposed relation graph network comprises a 3D object proposal generation module and a 3D relation module, making it an end-to-end trainable network for detecting 3D objects in point clouds. Experiments on challenging benchmark point cloud datasets (SunRGB-D, ScanNet and KITTI) show that our algorithm performs better than existing state-of-the-art.

Small Object Augmentation of Urban Scenes for Real-Time Semantic Segmentation

Semantic segmentation is a key step in scene understanding for autonomous driving. Although deep learning has significantly improved the segmentation accuracy, current highquality models such as PSPNet and DeepLabV3 are inefficient given their complex architectures and reliance on multi-scale inputs. Thus, it is difficult to apply them to real-time or practical applications. On the other hand, existing real-time methods cannot yet produce satisfactory results on small objects such as traffic lights, which are imperative to safe autonomous driving. In this paper, we improve the performance of real-time semantic segmentation from two perspectives, methodology and data. Specifically, we propose a real-time segmentation model coined Narrow Deep Network (NDNet) and build a synthetic dataset by inserting additional small objects into the training images. The proposed method achieves 65.7% mean intersection over union (mIoU) on the Cityscapes test set with only 8.4G floatingpoint operations (FLOPs) on 1024×2048 inputs. Furthermore, by re-training the existing PSPNet and DeepLabV3 models on our synthetic dataset, we obtained an average 2% mIoU improvement on small objects.

Assessing physical activity in people with mental illness: 23-country reliability and validity of the simple physical activity questionnaire (SIMPAQ)

BACKGROUND

Physical inactivity is a key contributor to the global burden of disease and disproportionately impacts the wellbeing of people experiencing mental illness. Increases in physical activity are associated with improvements in symptoms of mental illness and reduction in cardiometabolic risk. Reliable and valid clinical tools that assess physical activity would improve evaluation of intervention studies that aim to increase physical activity and reduce sedentary behaviour in people living with mental illness.

METHODS

The five-item Simple Physical Activity Questionnaire (SIMPAQ) was developed by a multidisciplinary, international working group as a clinical tool to assess physical activity and sedentary behaviour in people living with mental illness. Patients with a DSM or ICD mental illness diagnoses were recruited and completed the SIMPAQ on two occasions, one week apart. Participants wore an Actigraph accelerometer and completed brief cognitive and clinical assessments.

RESULTS

Evidence of SIMPAQ validity was assessed against accelerometer-derived measures of physical activity. Data were obtained from 1010 participants. The SIMPAQ had good test-retest reliability. Correlations for moderate-vigorous physical activity was comparable to studies conducted in general population samples. Evidence of validity for the sedentary behaviour item was poor. An alternative method to calculate sedentary behaviour had stronger evidence of validity. This alternative method is recommended for use in future studies employing the SIMPAQ.

CONCLUSIONS

The SIMPAQ is a brief measure of physical activity and sedentary behaviour that can be reliably and validly administered by health professionals.

Hyperspectral Recovery from RGB Images using Gaussian Processes

We propose to recover spectral details from RGB images of known spectral quantization by modeling natural spectra under Gaussian Processes and combining them with the RGB images. Our technique exploits Process Kernels to model the relative smoothness of reflectance spectra, and encourages non-negativity in the resulting signals for better estimation of the reflectance values. The Gaussian Processes are inferred in sets using clusters of spatio-spectrally correlated hyperspectral training patches. Each set is transformed to match the spectral quantization of the test RGB image. We extract overlapping patches from the RGB image and match them to the hyperspectral training patches by spectrally transforming the latter. The RGB patches are encoded over the transformed Gaussian Processes related to those hyperspectral patches and the resulting image is constructed by combining the codes with the original processes. Our approach infers the desired Gaussian Processes under a fully Bayesian model inspired by Beta-Bernoulli Process, for which we also present the inference procedure. A thorough evaluation using three hyperspectral datasets demonstrates the effective extraction of spectral details from RGB images by the proposed technique.

Design of a Pilot SOFC System for the Combined Production of Hydrogen and Electricity under Refueling Station Requirements

The objective of the current work is to support the design of a pilot hydrogen and electricity producing plant that uses natural gas (or biomethane) as raw material, as a transition option towards a 100% renewable transportation system. The plant, with a solid oxide fuel cell (SOFC) as principal technology, is intended to be the main unit of an electric vehicle station. The refueling station has to work at different operation periods characterized by the hydrogen demand and the electricity needed for supply and self-consumption. The same set of heat exchangers has to satisfy the heating and cooling needs of the different operation periods. In order to optimize the operating variables of the pilot plant and to provide the best heat exchanger network, the applied methodology follows a systematic procedure for multi-objective, i.e. maximum plant efficiency and minimum number of heat exchanger matches, and multi-period optimization. The solving strategy combines process flow modeling in steady state, superstructure-based mathematical programming and the use of an evolutionary-based algorithm for optimization. The results show that the plant can reach a daily weighted efficiency exceeding 60%, up to 80% when considering heat utilization.

Converting a Common Low-Cost Document Scanner into a Multispectral Scanner

Forged documents and counterfeit currency can be better detected with multispectral imaging in multiple color channels instead of the usual red, green and blue. However, multispectral cameras/scanners are expensive. We propose the construction of a low cost scanner designed to capture multispectral images of documents. A standard sheet-feed scanner was modified by disconnecting its internal light source and connecting an external multispectral light source comprising of narrow band light emitting diodes (LED). A document was scanned by illuminating the scanner light guide successively with different LEDs and capturing a scan of the document. The system costs less than a hundred dollars and is portable. It can potentially be used for applications in verification of questioned documents, checks, receipts and bank notes.

On-field player workload exposure and knee injury risk monitoring via deep learning

In sports analytics, an understanding of accurate on-field 3D knee joint moments (KJM) could provide an early warning system for athlete workload exposure and knee injury risk. Traditionally, this analysis has relied on captive laboratory force plates and associated downstream biomechanical modeling, and many researchers have approached the problem of portability by extrapolating models built on linear statistics. An alternative approach would be to capitalize on recent advances in deep learning. In this study, using the pre-trained CaffeNet convolutional neural network (CNN) model, multivariate regression of marker-based motion capture to 3D KJM for three sports-related movement types were compared. The strongest overall mean correlation to source modeling of 0.8895 was achieved over the initial 33% of stance phase for sidestepping. The accuracy of these mean predictions of the three critical KJM associated with anterior cruciate ligament (ACL) injury demonstrate the feasibility of on-field knee injury assessment using deep learning in lieu of laboratory embedded force plates. This multidisciplinary research approach significantly advances machine representation of real-world physical models with practical application for both community and professional level athletes.

Increased facial asymmetry in autism spectrum conditions is associated with symptom presentation

A key research priority in the study of autism spectrum conditions (ASC) is the discovery of biological markers that may help to identify and elucidate etiologically distinct subgroups. One physical marker that has received increasing research attention is facial structure. Although there remains little consensus in the field, findings relating to greater facial asymmetry (FA) in ASC exhibit some consistency. As there is growing recognition of the importance of replicatory studies in ASC research, the aim of this study was to investigate the replicability of increased FA in autistic children compared to nonautistic peers. Using three-dimensional photogrammetry, this study examined FA in 84 autistic children, 110 typically developing children with no family history of the condition, and 49 full siblings of autistic children. In support of previous literature, significantly greater depth-wise FA was identified in autistic children relative to the two comparison groups. As a further investigation, increased lateral FA in autistic children was found to be associated with greater severity of ASC symptoms on the Autism Diagnostic Observation Schedule, second edition, specifically related to repetitive and restrictive behaviors. These outcomes provide an important and independent replication of increased FA in ASC, as well as a novel contribution to the field. Having confirmed the direction and areas of increased FA in ASC, these findings could motivate a search for potential underlying brain dysmorphogenesis. Autism Res 2019, 12: 1774-1783. © 2019 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: This study looked at the amount of facial asymmetry (FA) in autistic children compared to typically developing children and children who have siblings with autism. The study found that autistic children, compared to the other two groups, had greater FA, and that increased FA was related to greater severity of autistic symptoms. The face and brain grow together during the earliest stages of development, and so findings of facial differences in autism might inform future studies of early brain differences associated with the condition.

Critical care usage after major gastrointestinal and liver surgery: a prospective, multicentre observational study

BACKGROUND

Patient selection for critical care admission must balance patient safety with optimal resource allocation. This study aimed to determine the relationship between critical care admission, and postoperative mortality after abdominal surgery.

METHODS

This prespecified secondary analysis of a multicentre, prospective, observational study included consecutive patients enrolled in the DISCOVER study from UK and Republic of Ireland undergoing major gastrointestinal and liver surgery between October and December 2014. The primary outcome was 30-day mortality. Multivariate logistic regression was used to explore associations between critical care admission (planned and unplanned) and mortality, and inter-centre variation in critical care admission after emergency laparotomy.

RESULTS

Of 4529 patients included, 37.8% (n=1713) underwent planned critical care admissions from theatre. Some 3.1% (n=86/2816) admitted to ward-level care subsequently underwent unplanned critical care admission. Overall 30-day mortality was 2.9% (n=133/4519), and the risk-adjusted association between 30-day mortality and critical care admission was higher in unplanned [odds ratio (OR): 8.65, 95% confidence interval (CI): 3.51-19.97) than planned admissions (OR: 2.32, 95% CI: 1.43-3.85). Some 26.7% of patients (n=1210/4529) underwent emergency laparotomies. After adjustment, 49.3% (95% CI: 46.8-51.9%, P<0.001) were predicted to have planned critical care admissions, with 7% (n=10/145) of centres outside the 95% CI.

CONCLUSIONS

After risk adjustment, no 30-day survival benefit was identified for either planned or unplanned postoperative admissions to critical care within this cohort. This likely represents appropriate admission of the highest-risk patients. Planned admissions in selected, intermediate-risk patients may present a strategy to mitigate the risk of unplanned admission. Substantial inter-centre variation exists in planned critical care admissions after emergency laparotomies.

Nonparametric Coupled Bayesian Dictionary and Classifier Learning for Hyperspectral Classification

We present a principled approach to learn a discriminative dictionary along a linear classifier for hyperspectral classification. Our approach places Gaussian Process priors over the dictionary to account for the relative smoothness of the natural spectra, whereas the classifier parameters are sampled from multivariate Gaussians. We employ two Beta-Bernoulli processes to jointly infer the dictionary and the classifier. These processes are coupled under the same sets of Bernoulli distributions. In our approach, these distributions signify the frequency of the dictionary atom usage in representing class-specific training spectra, which also makes the dictionary discriminative. Due to the coupling between the dictionary and the classifier, the popularity of the atoms for representing different classes gets encoded into the classifier. This helps in predicting the class labels of test spectra that are first represented over the dictionary by solving a simultaneous sparse optimization problem. The labels of the spectra are predicted by feeding the resulting representations to the classifier. Our approach exploits the nonparametric Bayesian framework to automatically infer the dictionary size-the key parameter in discriminative dictionary learning. Moreover, it also has the desirable property of adaptively learning the association between the dictionary atoms and the class labels by itself. We use Gibbs sampling to infer the posterior probability distributions over the dictionary and the classifier under the proposed model, for which, we derive analytical expressions. To establish the effectiveness of our approach, we test it on benchmark hyperspectral images. The classification performance is compared with the state-of-the-art dictionary learning-based classification methods.

Predicting Athlete Ground Reaction Forces and Moments From Spatio-Temporal Driven CNN Models

The accurate prediction of three-dimensional (3-D) ground reaction forces and moments (GRF/Ms) outside the laboratory setting would represent a watershed for on-field biomechanical analysis. To extricate the biomechanist's reliance on ground embedded force plates, this study sought to improve on an earlier partial least squares (PLS) approach by using deep learning to predict 3-D GRF/Ms from legacy marker based motion capture sidestepping trials, ranking multivariate regression of GRF/Ms from five convolutional neural network (CNN) models. In a possible first for biomechanics, tactical feature engineering techniques were used to compress space-time and facilitate fine-tuning from three pretrained CNNs, from which a model derivative of ImageNet called "CaffeNet" achieved the strongest average correlation to ground truth GRF/Ms [Formula: see text] 0.9881 and [Formula: see text] 0.9715 ([Formula: see text] 4.31 and 7.04%). These results demonstrate the power of CNN models to facilitate real-world multivariate regression with practical application for spatio-temporal sports analytics.

Benchmark Data Set and Method for Depth Estimation from Light Field Images

Convolutional Neural Networks (CNN) have performed extremely well for many image analysis tasks. However, supervised training of deep CNN architectures requires huge amounts of labelled data which is unavailable for light field images. In this paper, we leverage on synthetic light field images and propose a two stream CNN network that learns to estimate the disparities of multiple correlated neighbourhood pixels from their Epipolar Plane Images (EPI). Since the EPIs are unrelated except at their intersection, a two stream network is proposed to learn convolution weights individually for the EPIs and then combine the outputs of the two streams for disparity estimation. The CNN estimated disparity map is then refined using the central RGB light field image as a prior in a variational technique. We also propose a new real world dataset comprising light field images of 19 objects captured with the Lytro Illum camera in outdoor scenes and their corresponding 3D pointclouds, as ground truth, captured with the 3dMD scanner. This dataset will be made public to allow more precise 3D pointcloud level comparison of algorithms in the future which is currently not possible. Experiments on the synthetic and real world datasets show that our algorithm outperforms existing state-of-the-art for depth estimation from light field images.