BlinkLinMulT: Transformer-Based Eye Blink Detection

This work presents BlinkLinMulT, a transformer-based framework for eye blink detection. While most existing approaches rely on frame-wise eye state classification, recent advancements in transformer-based sequence models have not been explored in the blink detection literature. Our approach effectively combines low- and high-level feature sequences with linear complexity cross-modal attention mechanisms and addresses challenges such as lighting changes and a wide range of head poses. Our work is the first to leverage the transformer architecture for blink presence detection and eye state recognition while successfully implementing an efficient fusion of input features. In our experiments, we utilized several publicly available benchmark datasets (CEW, ZJU, MRL Eye, RT-BENE, EyeBlink8, Researcher's Night, and TalkingFace) to extensively show the state-of-the-art performance and generalization capability of our trained model. We hope the proposed method can serve as a new baseline for further research.

Cross-Viewpoint Semantic Mapping: Integrating Human and Robot Perspectives for Improved 3D Semantic Reconstruction

Allocentric semantic 3D maps are highly useful for a variety of human-machine interaction related tasks since egocentric viewpoints can be derived by the machine for the human partner. Class labels and map interpretations, however, may differ or could be missing for the participants due to the different perspectives. Particularly, when considering the viewpoint of a small robot, which significantly differs from the viewpoint of a human. In order to overcome this issue, and to establish common ground, we extend an existing real-time 3D semantic reconstruction pipeline with semantic matching across human and robot viewpoints. We use deep recognition networks, which usually perform well from higher (i.e., human) viewpoints but are inferior from lower viewpoints, such as that of a small robot. We propose several approaches for acquiring semantic labels for images taken from unusual perspectives. We start with a partial 3D semantic reconstruction from the human perspective that we transfer and adapt to the small robot's perspective using superpixel segmentation and the geometry of the surroundings. The quality of the reconstruction is evaluated in the Habitat simulator and a real environment using a robot car with an RGBD camera. We show that the proposed approach provides high-quality semantic segmentation from the robot's perspective, with accuracy comparable to the original one. In addition, we exploit the gained information and improve the recognition performance of the deep network for the lower viewpoints and show that the small robot alone is capable of generating high-quality semantic maps for the human partner. The computations are close to real-time, so the approach enables interactive applications.

Design of a Machine Learning System to Predict the Thickness of a Melanoma Lesion in a Non-Invasive Way from Dermoscopic Images

OBJECTIVES

Melanoma is the deadliest form of skin cancer, but it can be fully cured through early detection and treatment in 99% of cases. Our aim was to develop a non-invasive machine learning system that can predict the thickness of a melanoma lesion, which is a proxy for tumor progression, through dermoscopic images. This method can serve as a valuable tool in identifying urgent cases for treatment.

METHODS

A modern convolutional neural network architecture (EfficientNet) was used to construct a model capable of classifying dermoscopic images of melanoma lesions into three distinct categories based on thickness. We incorporated techniques to reduce the impact of an imbalanced training dataset, enhanced the generalization capacity of the model through image augmentation, and utilized five-fold cross-validation to produce more reliable metrics.

RESULTS

Our method achieved 71% balanced accuracy for three-way classification when trained on a small public dataset of 247 melanoma images. We also presented performance projections for larger training datasets.

CONCLUSIONS

Our model represents a new state-of-the-art method for classifying melanoma thicknesses. Performance can be further optimized by expanding training datasets and utilizing model ensembles. We have shown that earlier claims of higher performance were mistaken due to data leakage during the evaluation process.

Peripheral gene interactions define interpretable clusters of core ASD genes in a network-based investigation of the omnigenic theory

According to the recently proposed omnigenic theory, all expressed genes in a relevant tissue are contributing directly or indirectly to the manifestation of complex disorders such as autism. Thus, holistic approaches can be complementary in studying genetics of these complex disorders to focusing on a limited number of candidate genes. Gene interaction networks can be used for holistic studies of the omnigenic nature of autism. We used Louvain clustering on tissue-specific gene interaction networks and their subgraphs exclusively containing autism-related genes to study the effects of peripheral gene interactions. We observed that the autism gene clusters are significantly weaker connected to each other and the peripheral genes in non-neuronal tissues than in brain-related tissues. The biological functions of the brain clusters correlated well with previous findings on autism, such as synaptic signaling, regulation of DNA methylation, or regulation of lymphocyte activation, however, on the other tissues they did not enrich as significantly. Furthermore, ASD subjects with disruptive mutations in specific gene clusters show phenotypical differences compared to other disruptive variants carrying ASD individuals. Our results strengthen the omnigenic theory and can advance our understanding of the genetic background of autism.

Combination of multifunctional ursolic acid with kinase inhibitors for anti-cancer drug carrier vesicles

A sterically stabilized unilamellar nanocarrier vesicle (SSV) system containing dipalmitoylphosphatidylcholine, cholesterol, ursolic acid and PEGylated phospholipid has been developed by exploiting the structural advantages of ursolic acid: by spontaneously attaching to the lipid head groups, it induces curvature at the outer side of the bilayers, allowing the preparation of size-limited vesicles without extrusion. Ursolic acid (UA) also interacts with the PEG chains, supporting steric stabilization even when the amount of PEGylated phospholipid is reduced. Using fluorescence immunohistochemistry, vesicles containing ursolic acid (UA-SSVs) were found to accumulate in the tumor in 3 h on xenografted mouse, suggesting the potential use of these vesicles for passive tumor targeting. Further on, mono- and combination therapy with UA and six different kinase inhibitors (crizotinib, erlotinib, foretinib, gefitinib, refametinib, trametinib) was tested on seven cancer cell-lines. In most combinations synergism was observed, in the case of trametinib even at very low concentration (0.001 μM), which targets the MAPK pathway most often activated in human cancers. The coupled intercalation of UA and trametinib (2:1 molar ratio) into vesicles causes further structural advantageous molecular interactions, promoting the formation of small vesicles. The high drug:lipid molar ratio (~0.5) in the novel type of co-delivery vesicles enables their direct medical application, possibly also overcoming the multidrug resistance effect.

The Autism Palette: Combinations of Impairments Explain the Heterogeneity in ASD

Autism spectrum disorder (ASD) is a heterogeneous neuropsychiatric condition traditionally defined by core symptoms in social behavior, speech/communication, repetitive behavior, and restricted interests. Beyond the core symptoms, autism has strong association with other disorders such as intellectual disability (ID), epilepsy, schizophrenia among many others. This paper outlines a theory of ASD with capacity to connect heterogeneous "core" symptoms, medical and psychiatric comorbidities as well as other etiological theories of autism in a unifying cognitive framework rooted in neuroscience and genetics. Cognition is embedded into an ever-developing structure modified by experiences, including the outcomes of environment influencing behaviors. The key constraint of cognition is that the brain can handle only 7±2 relevant variables at a time, whereas sensory variables, i.e., the number of sensory neurons is orders of magnitude larger. As a result, (a) the extraction, (b) the encoding, and (c) the capability for the efficient cognitive manipulation of the relevant variables, and (d) the compensatory mechanisms that counteract computational delays of the distributed components are critical. We outline our theoretical model to describe a Cartesian Factor (CF) forming, autoencoder-like cognitive mechanism which breaks combinatorial explosion and is accelerated by internal reinforcing machineries and discuss the neural processes that support CF formation. Impairments in any of these aspects may disrupt learning, cognitive manipulation, decisions on interactions, and execution of decisions. We suggest that social interactions are the most susceptible to combinations of diverse small impairments and can be spoiled in many ways that pile up. Comorbidity is experienced, if any of the many potential impairments is relatively strong. We consider component spoiling impairments as the basic colors of autism, whereas the combinations of individual impairments make the palette of autism. We put forth arguments on the possibility of dissociating the different main elements of the impairments that can appear together. For example, impairments of generalization (domain general learning) and impairments of dealing with many variable problems, such as social situations may appear independently and may mutually enhance their impacts. We also consider mechanisms that may lead to protection.

Semi-Supervised Learning of Cartesian Factors: A Top-Down Model of the Entorhinal Hippocampal Complex

The existence of place cells (PCs), grid cells (GCs), border cells (BCs), and head direction cells (HCs) as well as the dependencies between them have been enigmatic. We make an effort to explain their nature by introducing the concept of Cartesian Factors. These factors have specific properties: (i) they assume and complement each other, like direction and position and (ii) they have localized discrete representations with predictive attractors enabling implicit metric-like computations. In our model, HCs make the distributed and local representation of direction. Predictive attractor dynamics on that network forms the Cartesian Factor "direction." We embed these HCs and idiothetic visual information into a semi-supervised sparse autoencoding comparator structure that compresses its inputs and learns PCs, the distributed local and direction independent (allothetic) representation of the Cartesian Factor of global space. We use a supervised, information compressing predictive algorithm and form direction sensitive (oriented) GCs from the learned PCs by means of an attractor-like algorithm. Since the algorithm can continue the grid structure beyond the region of the PCs, i.e., beyond its learning domain, thus the GCs and the PCs together form our metric-like Cartesian Factors of space. We also stipulate that the same algorithm can produce BCs. Our algorithm applies (a) a bag representation that models the "what system" and (b) magnitude ordered place cell activities that model either the integrate-and-fire mechanism, or theta phase precession, or both. We relate the components of the algorithm to the entorhinal-hippocampal complex and to its working. The algorithm requires both spatial and lifetime sparsification that may gain support from the two-stage memory formation of this complex.

MDR-1 and MRP-1 activity in peripheral blood leukocytes of rheumatoid arthritis patients

Background

Rheumatoid Arthritis is a chronic disease leading to decreased quality of life with a rather variable response rate to Disease Modifying Anti Rheumatic Drugs. Methotrexate (MTX) is the gold standard therapy in Rheumatoid Arthritis. The Multidrug resistance Related Protein and Multi Drug Resistance protein 1, also called P-glycoprotein-170 transporters can alter the intracellular concentration of different drugs. Methotrexate is an MRP1 substrate and thus the functional activity of MRP1 might have a clinical impact on the efficiency of the Methotrexate-therapy in Rheumatoid Arthritis.

Methods

We have compared the functional Multidrug Activity Factors (MAF) of the MDR1 and MRP1 transporters of Peripheral Blood Leukocytes of 59 Rheumatoid Arthritis patients with various response rate to MTX-therapy (MTX-responder, MTX-resistant and MTX-intolerant RA-groups) and 47 non-RA controls in six different leukocyte subpopulations (neutrophil leukocytes, monocytes, lymphocytes, CD4+, CD8+ and CD19+ cells). There was a decreased MAF of RA patients compared to non- Rheumatoid Arthritis patients and healthy controls in the leukocyte subpopulations. There was a significant difference between the MAF values of the MTX-responder and MTX intolerant groups. But we have not found significant differences between the MAF values of the MTX-responder and MTX-resistant Rheumatoid Arthritis -groups.

Results

Our results suggest that MDR1 and MRP1 functional activity does not seem to affect the response rate to MTX-therapy of Rheumatoid Arthritis-patients, but it might be useful in predicting MTX-side effects. We have demonstrated the decreased functional MDR-activity on almost 60 Rheumatoid Arthritis patients, which can be interpreted as a sign of the immune-suppressive effect of the MTX-treatment.

Effects of ursolic acid on the structural and morphological behaviours of dipalmitoyl lecithin vesicles

Effects of ursolic acid on the structural and morphological characteristics of dipalmitoyl lecithin(DPPC)-water system was studied by using differential scanning calorimetry (DSC), small- and wide-angle X-ray scattering (SWAXS), freeze-fracture method combined with transmission electron-microscopy (FF-TEM) and infrared spectroscopy (FT-IR). The surface of the uncorrelated lipid system is rippled or grained and a huge number of small, presumably unilamellar vesicles are present if the UA/DPPC molar ratio is 0.1 mol/mol or higher. Besides the destroyed layer packing of regular multilamellar vesicles, non-bilayer (e.g. cubic or hexagonal) local structures are evidenced by SAXS and FF-TEM methods. The ability of UA to induce non-bilayer structures in hydrated DPPC system originates from the actual geometry form of associated lipid and UA molecules as concluded from the FT-IR measurements and theoretical calculations. Beside numerous beneficial e.g. chemopreventive and chemotherapeutic effect of ursolic acid against cancer, their impact to modify the lipid bilayers can be utilized in liposomal formulations.

Decreased functional activity of multidrug resistance protein in primary colorectal cancer

Background

The ATP-Binding Cassette (ABC)-transporter MultiDrug Resistance Protein 1 (MDR1) and Multidrug Resistance Related Protein 1 (MRP1) are expressed on the surface of enterocytes, which has led to the belief that these high capacity transporters are responsible for modulating chemosensitvity of colorectal cancer. Several immunohistochemistry and reverse transcription polymerase chain reaction (RT-PCR) studies have provided controversial results in regards to the expression levels of these two ABC-transporters in colorectal cancer. Our study was designed to determine the yet uninvestigated functional activity of MDR1 and MRP1 transporters in normal human enterocytes compared to colorectal cancer cells from surgical biopsies.

Methods

100 colorectal cancer and 28 adjacent healthy mucosa samples were obtained by intraoperative surgical sampling. Activity of MDR1 and MRP1 of viable epithelial and cancer cells were determined separately with the modified calcein-assay for multidrug resistance activity and sufficient data of 73 cancer and 11 healthy mucosa was analyzed statistically.

Results

Significantly decreased mean MDR1 activity was found in primary colorectal cancer samples compared to normal mucosa, while mean MRP1 activity showed no significant change. Functional activity was not affected by gender, age, stage or grade and localization of the tumor.

Conclusion

We found lower MDR activity in cancer cells versus adjacent, apparently, healthy control tissue, thus, contrary to general belief, MDR activity seems not to play a major role in primary drug resistance, but might rather explain preferential/selective activity of Irinotecan and/or Oxaliplatin. Still, this picture might be more complex since chemotherapy by itself might alter MDR activity, and furthermore, today limited data is available about MDR activity of cancer stem cells in colorectal cancers.

Virtual slides

The virtual slide(s) for this article can be found here: http://www.diagnosticpathology.diagnomx.eu/vs/1675739129145824

Spatio-temporal Event Classification using Time-series Kernel based Structured Sparsity

In many behavioral domains, such as facial expression and gesture, sparse structure is prevalent. This sparsity would be well suited for event detection but for one problem. Features typically are confounded by alignment error in space and time. As a consequence, high-dimensional representations such as SIFT and Gabor features have been favored despite their much greater computational cost and potential loss of information. We propose a Kernel Structured Sparsity (KSS) method that can handle both the temporal alignment problem and the structured sparse reconstruction within a common framework, and it can rely on simple features. We characterize spatio-temporal events as time-series of motion patterns and by utilizing time-series kernels we apply standard structured-sparse coding techniques to tackle this important problem. We evaluated the KSS method using both gesture and facial expression datasets that include spontaneous behavior and differ in degree of difficulty and type of ground truth coding. KSS outperformed both sparse and non-sparse methods that utilize complex image features and their temporal extensions. In the case of early facial event classification KSS had 10% higher accuracy as measured by F₁ score over kernel SVM methods.

Topology Learning Solved by Extended Objects: A Neural Network Model

It is shown that local, extended objects of a metrical topological space shape the receptive fields of competitive neurons to local filters. Self-organized topology learning is then solved with the help of Hebbian learning together with extended objects that provide unique information about neighborhood relations. A topographical map is deduced and is used to speed up further adaptation in a changing environment with the help of Kohonen-type learning that teaches the neighbors of winning neurons as well.