CL3: Generalization of Contrastive Loss for Lifelong Learning

Lifelong learning portrays learning gradually in nonstationary environments and emulates the process of human learning, which is efficient, robust, and able to learn new concepts incrementally from sequential experience. To equip neural networks with such a capability, one needs to overcome the problem of catastrophic forgetting, the phenomenon of forgetting past knowledge while learning new concepts. In this work, we propose a novel knowledge distillation algorithm that makes use of contrastive learning to help a neural network to preserve its past knowledge while learning from a series of tasks. Our proposed generalized form of contrastive distillation strategy tackles catastrophic forgetting of old knowledge, and minimizes semantic drift by maintaining a similar embedding space, as well as ensures compactness in feature distribution to accommodate novel tasks in a current model. Our comprehensive study shows that our method achieves improved performances in the challenging class-incremental, task-incremental, and domain-incremental learning for supervised scenarios.

Exploiting Field Dependencies for Learning on Categorical Data

Traditional approaches for learning on categorical data underexploit the dependencies between columns (a.k.a. fields) in a dataset because they rely on the embedding of data points driven alone by the classification/regression loss. In contrast, we propose a novel method for learning on categorical data with the goal of exploiting dependencies between fields. Instead of modelling statistics of features globally (i.e., by the covariance matrix of features), we learn a global field dependency matrix that captures dependencies between fields and then we refine the global field dependency matrix at the instance-wise level with different weights (so-called local dependency modelling) w.r.t. each field to improve the modelling of the field dependencies. Our algorithm exploits the meta-learning paradigm, i.e., the dependency matrices are refined in the inner loop of the meta-learning algorithm without the use of labels, whereas the outer loop intertwines the updates of the embedding matrix (the matrix performing projection) and global dependency matrix in a supervised fashion (with the use of labels). Our method is simple yet it outperforms several state-of-the-art methods on six popular dataset benchmarks. Detailed ablation studies provide additional insights into our method.

The Prevalence of Hypertension and Obesity in Iranian Professional Drivers

Background

Professional driving is associated with overworking, lack of physical activity, and high stress, which are susceptible to cardiovascular diseases (CVDs). We aimed to determine the prevalence of hypertension and obesity in Iranian professional drivers.

Methods

Overall, 132,452 drivers were included by census sampling methods and those who did not pass periodic examinations were excluded. Demographics and anthropometric data, including height and weight and the driver's blood pressure, were recorded. The criteria for hypertension assumed as the systolic blood pressure ≥ 130 mm and/or diastolic blood pressure ≥ 80 mm, and the criteria for prehypertension assumed as 120-129 systolic and < 80 mm Hg. In addition, body mass index (BMI) ≥ 25 is assumed as overweight, and BMI ≥ 30 is assumed as obesity.

Results

Overall, 113,856 male drivers were included in the final analysis. The prevalence of HTN, pre-HTN, and abnormal blood pressure (HTN + pre-HTN) was calculated to be 14.2%, 57.4%, and 71.6%, respectively. Khuzestan, West Azerbaijan, and Yazd had the most prevalence of abnormal blood pressure. The prevalence of overweight, obesity, and abnormal weight (overweight + obesity) was calculated to be 50.9%, 22.6%, and 73.5%, respectively, and the northwest provinces had the highest prevalence of abnormal weight.

Conclusion

Professional Iranian drivers have a high prevalence of abnormal blood pressure and weight associated with job-related risk factors. Preventive measures should be taken to confront a possible outbreak of CVDs in this population.

Subspace distillation for continual learning

An ultimate objective in continual learning is to preserve knowledge learned in preceding tasks while learning new tasks. To mitigate forgetting prior knowledge, we propose a novel knowledge distillation technique that takes into the account the manifold structure of the latent/output space of a neural network in learning novel tasks. To achieve this, we propose to approximate the data manifold up-to its first order, hence benefiting from linear subspaces to model the structure and maintain the knowledge of a neural network while learning novel concepts. We demonstrate that the modeling with subspaces provides several intriguing properties, including robustness to noise and therefore effective for mitigating Catastrophic Forgetting in continual learning. We also discuss and show how our proposed method can be adopted to address both classification and segmentation problems. Empirically, we observe that our proposed method outperforms various continual learning methods on several challenging datasets including Pascal VOC, and Tiny-Imagenet. Furthermore, we show how the proposed method can be seamlessly combined with existing learning approaches to improve their performances. The codes of this article will be available at https://github.com/csiro-robotics/SDCL.

How Reproducible are Surface Areas Calculated from the BET Equation?

Porosity and surface area analysis play a prominent role in modern materials science. At the heart of this sits the Brunauer-Emmett-Teller (BET) theory, which has been a remarkably successful contribution to the field of materials science. The BET method was developed in the 1930s for open surfaces but is now the most widely used metric for the estimation of surface areas of micro- and mesoporous materials. Despite its widespread use, the calculation of BET surface areas causes a spread in reported areas, resulting in reproducibility problems in both academia and industry. To prove this, for this analysis, 18 already-measured raw adsorption isotherms were provided to sixty-one labs, who were asked to calculate the corresponding BET areas. This round-robin exercise resulted in a wide range of values. Here, the reproducibility of BET area determination from identical isotherms is demonstrated to be a largely ignored issue, raising critical concerns over the reliability of reported BET areas. To solve this major issue, a new computational approach to accurately and systematically determine the BET area of nanoporous materials is developed. The software, called "BET surface identification" (BETSI), expands on the well-known Rouquerol criteria and makes an unambiguous BET area assignment possible.

Thermal Infrared Imaging Can Differentiate Skin Temperature Changes Associated With Intense Single Leg Exercise, But Not With Delayed Onset of Muscle Soreness

Muscle damage and soreness associated with increased exercise training loads or unaccustomed activity can be debilitating and impact the quality of subsequent activity/performance. Current techniques to assess muscle soreness are either time consuming, invasive or subjective. Infrared thermography has been identified as a quick, non-invasive, portable and athlete friendly method of assessing skin temperature. This study assessed the capability of thermal infrared imaging to detect skin temperature changes that may accompany the inflammatory response associated with delayed onset muscular soreness (DOMS). Eight recreationally trained participants (age 25 ± 3 years, mass 74.9 ± 13.6 kg, training minutes 296 ± 175 min·wk^-1) completed 6 sets of 25 maximal concentric/eccentric contractions of the right knee flexors/extensors on a dynamometer to induce muscle damage and DOMS. The left knee extensors acted as a non-exercise control. Neuromuscular performance, subjective pain assessment and infrared thermography were undertaken at baseline, 24 and 48 hr post the DOMS-inducing exercise protocol. Data were analysed using Bayesian hierarchical regression and Cohen's d was also calculated. Maximal voluntary contraction torque was statistically lower at 24 hr (d = -0.70) and 48 hr (d = -0.52) compared to baseline, after the DOMS-inducing exercise protocol. These neuromuscular impairments coincided with statistically higher ratings of muscle soreness at 24 hr (d = 0.96) and 48 hr (d = 0.48). After adjusting for ambient temperature, anterior thigh skin temperature was statistically elevated at 24 hr, but not 48 hr, compared with baseline, in both the exercised and non-exercised leg. Thigh temperature was not different statistically between legs at these time points. Infrared imaging was able to detect elevations in skin temperature, at 24 hrs after the DOMS inducing exercise protocol, in both the exercised and non-exercised thigh. Elevations in the skin temperature of both thighs, potentially identifies a systemic inflammatory response occurring at 24 hr after the DOMS-inducing exercise protocol.

Fast vanishing-point detection in unstructured environments

Vision-based road detection in unstructured environments is a challenging problem as there are hardly any discernible and invariant features that can characterize the road or its boundaries in such environments. However, a salient and consistent feature of most roads or tracks regardless of type of the environments is that their edges, boundaries, and even ruts and tire tracks left by previous vehicles on the path appear to converge into a single point known as the vanishing point. Hence, estimating this vanishing point plays a pivotal role in the determination of the direction of the road. In this paper, we propose a novel methodology based on image texture analysis for the fast estimation of the vanishing point in challenging and unstructured roads. The key attributes of the methodology consist of the optimal local dominant orientation method that uses joint activities of only four Gabor filters to precisely estimate the local dominant orientation at each pixel location in the image plane, the weighting of each pixel based on its dominant orientation, and an adaptive distance-based voting scheme for the estimation of the vanishing point. A series of quantitative and qualitative analyses are presented using natural data sets from the Defense Advanced Research Projects Agency Grand Challenge projects to demonstrate the effectiveness and the accuracy of the proposed methodology.