Single-cell dissection of the multicellular ecosystem and molecular features underlying microvascular invasion in HCC

BACKGROUND AND AIMS

Microvascular invasion (MVI) is a crucial pathological hallmark of HCC that is closely associated with poor outcomes, early recurrence, and intrahepatic metastasis following surgical resection and transplantation. However, the intricate tumor microenvironment and transcriptional programs underlying MVI in HCC remain poorly understood.

APPROACH AND RESULTS

We performed single-cell RNA sequencing of 46,789 individual cells from 10 samples of MVI+ (MVI present) and MVI- (MVI absent) patients with HCC. We conducted comprehensive and comparative analyses to characterize cellular and molecular features associated with MVI and validated key findings using external bulk, single-cell, and spatial transcriptomic datasets coupled with multiplex immunofluorescence assays. The comparison identified specific subtypes of immune and stromal cells critical to the formation of the immunosuppressive and pro-metastatic microenvironment in MVI+ tumors, including cycling T cells, lysosomal associated membrane protein 3+ dendritic cells, triggering receptor expressed on myeloid cells 2+ macrophages, myofibroblasts, and arterial i endothelial cells. MVI+ malignant cells are characterized by high proliferation rates, whereas MVI- malignant cells exhibit an inflammatory milieu. Additionally, we identified the midkine-dominated interaction between triggering receptor expressed on myeloid cells 2+ macrophages and malignant cells as a contributor to MVI formation and tumor progression. Notably, we unveiled a spatially co-located multicellular community exerting a dominant role in shaping the immunosuppressive microenvironment of MVI and correlating with unfavorable prognosis.

CONCLUSIONS

This study provides a comprehensive single-cell atlas of MVI in HCC, shedding light on the complex multicellular ecosystem and molecular features associated with MVI. These findings deepen our understanding of the underlying mechanisms driving MVI and provide valuable insights for improving clinical diagnosis and developing more effective treatment strategies.

A Unified Particle-Based Solver for Non-Newtonian Behaviors Simulation

In this article, we present a unified framework to simulate non-Newtonian behaviors. We combine viscous and elasto-plastic stress into a unified particle solver to achieve various non-Newtonian behaviors ranging from fluid-like to solid-like. Our constitutive model is based on a Generalized Maxwell model, which incorporates viscosity, elasticity and plasticity in one non-linear framework by a unified way. On the one hand, taking advantage of the viscous term, we construct a series of strain-rate dependent models for classical non-Newtonian behaviors such as shear-thickening, shear-thinning, Bingham plastic, etc. On the other hand, benefiting from the elasto-plastic model, we empower our framework with the ability to simulate solid-like non-Newtonian behaviors, i.e., visco-elasticity/plasticity. In addition, we enrich our method with a heat diffusion model to make our method flexible in simulating phase change. Through sufficient experiments, we demonstrate a wide range of non-Newtonian behaviors ranging from viscous fluid to deformable objects. We believe this non-Newtonian model will enhance the realism of physically-based animation, which has great potential for computer graphics.

Fluid Inverse Volumetric Modeling and Applications from Surface Motion

In this study, we devise a framework for volumetrically reconstructing fluid from observable, measurable free surface motion. Our innovative method amalgamates the benefits of deep learning and conventional simulation to preserve the guiding motion and temporal coherence of the reproduced fluid. We infer surface velocities by encoding and decoding spatiotemporal features of surface sequences, and a 3D CNN is used to generate the volumetric velocity field, which is then combined with 3D labels of obstacles and boundaries. Concurrently, we employ a network to estimate the fluid's physical properties. To progressively evolve the flow field over time, we input the reconstructed velocity field and estimated parameters into the physical simulator as the initial state. Our approach yields promising results for both synthetic fluid generated by different fluid solvers and captured real fluid. The developed framework naturally lends itself to a variety of graphics applications, such as 1) effective reproductions of fluid behaviors visually congruent with the observed surface motion, and 2) physics-guided re-editing of fluid scenes. Extensive experiments affirm that our novel method surpasses state-of-the-art approaches for 3D fluid inverse modeling and animation in graphics.

FineStyle: Semantic-Aware Fine-Grained Motion Style Transfer with Dual Interactive-Flow Fusion

We present FineStyle, a novel framework for motion style transfer that generates expressive human animations with specific styles for virtual reality and vision fields. It incorporates semantic awareness, which improves motion representation and allows for precise and stylish animation generation. Existing methods for motion style transfer have all failed to consider the semantic meaning behind the motion, resulting in limited controls over the generated human animations. To improve, FineStyle introduces a new cross-modality fusion module called Dual Interactive-Flow Fusion (DIFF). As the first attempt, DIFF integrates motion style features and semantic flows, producing semantic-aware style codes for fine-grained motion style transfer. FineStyle uses an innovative two-stage semantic guidance approach that leverages semantic clues to enhance the discriminative power of both semantic and style features. At an early stage, a semantic-guided encoder introduces distinct semantic clues into the style flow. Then, at a fine stage, both flows are further fused interactively, selecting the matched and critical clues from both flows. Extensive experiments demonstrate that FineStyle outperforms state-of-the-art methods in visual quality and controllability. By considering the semantic meaning behind motion style patterns, FineStyle allows for more precise control over motion styles. Source code and model are available on https://github.com/XingliangJin/Fine-Style.git.

ZetaDesign: an end-to-end deep learning method for protein sequence design and side-chain packing

Computational protein design has been demonstrated to be the most powerful tool in the last few years among protein designing and repacking tasks. In practice, these two tasks are strongly related but often treated separately. Besides, state-of-the-art deep-learning-based methods cannot provide interpretability from an energy perspective, affecting the accuracy of the design. Here we propose a new systematic approach, including both a posterior probability and a joint probability parts, to solve the two essential questions once for all. This approach takes the physicochemical property of amino acids into consideration and uses the joint probability model to ensure the convergence between structure and amino acid type. Our results demonstrated that this method could generate feasible, high-confidence sequences with low-energy side conformations. The designed sequences can fold into target structures with high confidence and maintain relatively stable biochemical properties. The side chain conformation has a significantly lower energy landscape without delegating to a rotamer library or performing the expensive conformational searches. Overall, we propose an end-to-end method that combines the advantages of both deep learning and energy-based methods. The design results of this model demonstrate high efficiency, and precision, as well as a low energy state and good interpretability.

Interactive Virtual Ankle Movement Controlled by Wrist sEMG Improves Motor Imagery: An Exploratory Study

Virtual reality (VR) techniques can significantly enhance motor imagery training by creating a strong illusion of action for central sensory stimulation. In this study, we establish a precedent by using surface electromyography (sEMG) of contralateral wrist movement to trigger virtual ankle movement through an improved data-driven approach with a continuous sEMG signal for fast and accurate intention recognition. Our developed VR interactive system can provide feedback training for stroke patients in the early stages, even if there is no active ankle movement. Our objectives are to evaluate: 1) the effects of VR immersion mode on body illusion, kinesthetic illusion, and motor imagery performance in stroke patients; 2) the effects of motivation and attention when utilizing wrist sEMG as a trigger signal for virtual ankle motion; 3) the acute effects on motor function in stroke patients. Through a series of well-designed experiments, we have found that, compared to the 2D condition, VR significantly increases the degree of kinesthetic illusion and body ownership of the patients, and improves their motor imagery performance and motor memory. When compared to conditions without feedback, using contralateral wrist sEMG signals as trigger signals for virtual ankle movement enhances patients' sustained attention and motivation during repetitive tasks. Furthermore, the combination of VR and feedback has an acute impact on motor function. Our exploratory study suggests that the sEMG-based immersive virtual interactive feedback provides an effective option for active rehabilitation training for severe hemiplegia patients in the early stages, with great potential for clinical application.

Graph Diffusion Convolutional Network for Skeleton Based Semantic Recognition of Two-Person Actions

Graph Convolutional Networks (GCNs) have successfully boosted skeleton-based human action recognition. However, existing GCN-based methods mostly cast the problem as separated person's action recognition while ignoring the interaction between the action initiator and the action responder, especially for the fundamental two-person interactive action recognition. It is still challenging to effectively take into account the intrinsic local-global clues of the two-person activity. Additionally, message passing in GCN depends on adjacency matrix, but skeleton-based human action recognition methods tend to calculate the adjacency matrix with the fixed natural skeleton connectivity. It means that messages can only travel along a fixed path at different layers of the network or in different actions, which greatly reduces the flexibility of the network. To this end, we propose a novel graph diffusion convolutional network for skeleton based semantic recognition of two-person actions by embedding the graph diffusion into GCNs. At technical fronts, we dynamically construct the adjacency matrix based on practical action information, so that we can guide the message propagation in a more meaningful way. Simultaneously, we introduce the frame importance calculation module to conduct dynamic convolution, so that we can avoid the negative effect caused by the traditional convolution, wherein the shared weights may fail to capture key frames or be affected by noisy frames. Besides, we comprehensively leverage the multidimensional features related to joints' local visual appearances, global spatial relationship and temporal coherency, and for different features, different metrics are designed to measure the similarity underlying the corresponding real physical law of the motions. Moreover, extensive experiments and comprehensive evaluations on four public large-scale datasets (NTU-RGB+D 60, NTU-RGB+D 120, Kinetics-Skeleton 400, and SBU-Interaction) demonstrate that our method outperforms the state-of-the-art methods.

Exploring invertebrate indicators of ecosystem health by focusing on the flow transitional zones in a large, shallow eutrophic lake

The river-lake transitional zone provides a unique environment for the biological community and can reduce pollution inputs in lake ecosystems from their catchments. To explore environmental conditions with high purification potential in Lake Taihu and indicator species, we examined the river-to-lake changes in water and sediment quality and benthic invertebrate communities in the transitional zone of four regions. The spatial variations in the environment and invertebrate community observed in this study followed the previously reported patterns in Taihu; the northern and western regions were characterized by higher nutrient concentrations in water, higher heavy metal concentrations in sediment, and higher total invertebrate density and biomass dominated by pollution-tolerant oligochaetes and chironomids. Although nutrient concentrations were low and transparency was high in the eastern region, the taxon richness was the lowest there, which disagreed with the previous findings and might be due to a poor cover of macrophytes in this study. The river-to-lake change was large in the southern region for water quality and the invertebrate community. Water circulation induced by strong wind-wave actions in the lake sites of the southern region is assumed to have promoted photosynthetic and nutrient uptake activities and favored invertebrates that require well-aerated conditions such as polychaetes and burrowing crustaceans. Invertebrates usually adapted to brackish and saline environments are suggested to be indicators of a well-circulated environment with active biogeochemical processes and a less eutrophic state in Taihu, and wind-wave actions are key to maintaining such a community and natural purifying processes.

MPMNet: A Data-Driven MPM Framework for Dynamic Fluid-Solid Interaction

High-accuracy, high-efficiency physics-based fluid-solid interaction is essential for reality modeling and computer animation in online games or real-time Virtual Reality (VR) systems. However, the large-scale simulation of incompressible fluid and its interaction with the surrounding solid environment is either time-consuming or suffering from the reduced time/space resolution due to the complicated iterative nature pertinent to numerical computations of involved Partial Differential Equations (PDEs). In recent years, we have witnessed significant growth in exploring a different, alternative data-driven approach to addressing some of the existing technical challenges in conventional model-centric graphics and animation methods. This paper showcases some of our exploratory efforts in this direction. One technical concern of our research is to address the central key challenge of how to best construct the numerical solver effectively and how to best integrate spatiotemporal/dimensional neural networks with the available MPM's pressure solvers. In particular, we devise the MPMNet, a hybrid data-driven framework supporting the popular and powerful Material Point Method (MPM), to combine the comprehensive properties of MPM in numerically handling physical behaviors ranging from fluid to deformable solids and the high efficiency of data-driven models. At the architectural level, our MPMNet comprises three primary components: A data processing module to describe the physical properties by way of the input fields; A deep neural network group to learn the spatiotemporal features; And an iterative refinement process to continue to reduce possible numerical errors. The goal of these special technical developments is to aim at involved numerical acceleration while preserving physical accuracy, realizing efficient and accurate fluid-solid interactions in a data-driven fashion. The extensive experimental results verify that our MPMNet can tremendously speed up the computation compared with the popular numerical methods as the complexity of interaction scenes increases while better retaining the numerical accuracy.

How does digital village construction influences carbon emission? The case of China

Taking 30 provinces in China from 2011 to 2020 as a research sample, this paper empirically tests the impact of digital village construction on carbon emissions. This study found that there is an "inverted U" curve relationship between digital rural construction and rural carbon emissions. Agricultural planting structure and agricultural technology efficiency are important ways for digital village construction to reduce agricultural carbon emissions. The study also found that the higher the level of economic development, the stronger the carbon emission reduction effect of digital village construction. In addition, there are also significant differences in the carbon emission reduction effect of digital village construction in regions with different environmental regulation intensities. Finally, in terms of the relationship between digital economic activities and carbon emission reduction, the research conclusions of this paper have important implications.

Neurophysiological and Subjective Analysis of VR Emotion Induction Paradigm

The ecological validity of emotion-inducing scenarios is essential for emotion research. In contrast to the classical passive induction paradigm, immersive VR fully engages the psychological and physiological components of the subject, which is considered an ecologically valid paradigm for studying emotion. Several studies investigate the emotional responses to different VR tasks or games using subjective scales. However, little research regards VR as an eliciting material, especially when systematically analyzing emotional processes in VR from a neurophysiological perspective. To fill this gap and scientifically evaluate VR's ability to be used as an active method for emotion elicitation, we investigate the dynamic relationship between explicit information (subjective evaluations) and implicit information (objective neurophysiological data). A total of 28 participants are enlisted to watch eight VR videos while their SAM/IPQ scores and EEG data are recorded simultaneously. In ecologically valid scenarios, the subjective results demonstrate that VR has significant advantages for evoking emotion in arousal-valence. This conclusion is backed by our examination of objective neurophysiological evidence that VR videos effectively induce high-arousal emotions. In addition, we obtain features of critical channels and frequency oscillations associated with emotional valence, thereby validating previous research in more lifelike circumstances. In particular, we discover hemispheric asymmetry in the occipital region under high and low emotional arousal, which adds to our understanding of neural features and the dynamics of emotional arousal. As a result, we successfully integrate EEG and VR to demonstrate that VR is more pragmatic for evoking natural feelings and is beneficial for emotional research. Our research has set a precedent for new methodologies of using VR induction paradigms to acquire a more reliable explanation of affective computing.

Foveated Stochastic Lightcuts

Foveated rendering provides an idea for accelerating rendering algorithms without sacrificing the perceived rendering quality in virtual reality applications. In this paper, we propose a foveated stochastic lightcuts method to render high-quality many-lights illumination effects in high perception-sensitive regions. First, we introduce a spatiotemporal-luminance based lightcuts generation method to generate lightcuts with different accuracy for different visual perception-sensitive regions. Then we propose a multi-resolution light samples selection method to select the light sample for each node in the lightcuts more efficiently. Our method supports full-dynamic scenes containing over 250k dynamic light sources and dynamic diffuse/specular/glossy objects. It provides frame rates up to 110fps for high-quality many-lights illumination effects in high perception-sensitive regions of the HVS in VR HMDs. Compared with the state-of-the-art stochastic lightcuts method using the same rendering time, our method achieves smaller mean squared errors in the fovea and periphery. We also conduct user studies to prove that the perceived quality of our method has a high visual similarity with the results of the ground truth rendered by using the stochastic lightcuts with 2048 light samples per pixel.

Salient Object Detection via Dynamic Scale Routing

Recent research advances in salient object detection (SOD) could largely be attributed to ever-stronger multi-scale feature representation empowered by the deep learning technologies. The existing SOD deep models extract multi-scale features via the off-the-shelf encoders and combine them smartly via various delicate decoders. However, the kernel sizes in this commonly-used thread are usually "fixed". In our new experiments, we have observed that kernels of small size are preferable in scenarios containing tiny salient objects. In contrast, large kernel sizes could perform better for images with large salient objects. Inspired by this observation, we advocate the "dynamic" scale routing (as a brand-new idea) in this paper. It will result in a generic plug-in that could directly fit the existing feature backbone. This paper's key technical innovations are two-fold. First, instead of using the vanilla convolution with fixed kernel sizes for the encoder design, we propose the dynamic pyramid convolution (DPConv), which dynamically selects the best-suited kernel sizes w.r.t. the given input. Second, we provide a self-adaptive bidirectional decoder design to accommodate the DPConv-based encoder best. The most significant highlight is its capability of routing between feature scales and their dynamic collection, making the inference process scale-aware. As a result, this paper continues to enhance the current SOTA performance. Both the code and dataset are publicly available at https://github.com/wuzhenyubuaa/DPNet.

Nonfinite-modality data augmentation for brain image registration

Brain image registration is fundamental for brain medical image analysis. However, the lack of paired images with diverse modalities and corresponding ground truth deformations for training hinder its development. We propose a novel nonfinite-modality data augmentation for brain image registration to combat this. Specifically, some available whole-brain segmentation masks, including complete fine brain anatomical structures, are collected from the actual brain dataset, OASIS-3. One whole-brain segmentation mask can generate many nonfinite-modality brain images by randomly merging some fine anatomical structures and subsequently sampling the intensities for each fine anatomical structure using random Gaussian distribution. Furthermore, to get more realistic deformations as the ground truth, an improved 3D Variational Auto-encoder (VAE) is proposed by introducing the intensity-level reconstruction loss and the structure-level reconstruction loss. Based on the generated images and trained improved 3D VAE, a new Synthetic Nonfinite-Modality Brain Image Dataset (SNMBID) is created. Experiments show that pre-training on SNMBID can improve the accuracy of registration. Notably, SNMBID can be a landmark for evaluating other brain registration methods, and the model trained on the SNMBID can be a baseline for the brain image registration task. Our code is available at https://github.com/MangoWAY/SMIBID_BrainRegistration.

Hierarchical anatomical structure-aware based thoracic CT images registration

Accurate thoracic CT image registration remains challenging due to complex joint deformations and different motion patterns in multiple organs/tissues during breathing. To combat this, we devise a hierarchical anatomical structure-aware based registration framework. It affords a coordination scheme necessary for constraining a general free-form deformation (FFD) during thoracic CT registration. The key is to integrate the deformations of different anatomical structures in a divide-and-conquer way. Specifically, a deformation ability-aware dissimilarity metric is proposed for complex joint deformations containing large-scale flexible deformation of the lung region, rigid displacement of the bone region, and small-scale flexible deformation of the rest region. Furthermore, a motion pattern-aware regularization is devised to handle different motion patterns, which contain sliding motion along the lung surface, almost no displacement of the spine and smooth deformation of other regions. Moreover, to accommodate large-scale deformation, a novel hierarchical strategy, wherein different anatomical structures are fused on the same control lattice, registers images from coarse to fine via elaborate Gaussian pyramids. Extensive experiments and comprehensive evaluations have been executed on the 4D-CT DIR and 3D DIR COPD datasets. It confirms that this newly proposed method is locally comparable to state-of-the-art registration methods specializing in local deformations, while guaranteeing overall accuracy. Additionally, in contrast to the current popular learning-based methods that typically require dozens of hours or more pre-training with powerful graphics cards, our method only takes an average of 63 s to register a case with an ordinary graphics card of RTX2080 SUPER, making our method still worth promoting. Our code is available at https://github.com/heluxixue/Structure_Aware_Registration/tree/master.

Effect of remediation reagents on bacterial composition and ecological function in black-odorous water sediments

Black-odorous urban water bodies and sediments pose a serious environmental problem. In this study, we conducted microcosm batch experiments to investigate the effect of remediation reagents (magnesium hydroxide and calcium nitrate) on native bacterial communities and their ecological functions in the black-odorous sediment of urban water. The dominant phyla (Proteobacteria, Actinobacteria, Chloroflexi, and Planctomycetes) and classes (Alphaproteobacteria, Betaproteobacteria, and Gammaproteobacteria, Actinobacteria, Anaerolineae, and Planctomycetia) were determined under calcium nitrate and magnesium hydroxide treatments. Functional groups related to aerobic metabolism, including aerobic chemoheterotrophy, dark sulfide oxidation, and correlated dominant genera (Thiobacillus, Lysobacter, Gp16, and Gaiella) became more abundant under calcium nitrate treatment, whereas functional genes potentially involved in dissimilatory sulfate reduction became less abundant. The relative abundance of chloroplasts, fermentation, and correlated genera (Desulfomonile and unclassified Cyanobacteria) decreased under magnesium hydroxide treatment. Overall, these results indicated that calcium nitrate addition improved hypoxia-related reducing conditions in the sediment and promoted aerobic chemoheterotrophy.

Blue light irradiation increases the relative abundance of the diatom Nitzschia palea in co-culture with cyanobacterium Microcystis aeruginosa

Lake eutrophication is associated with cyanobacterial blooms. The pennate diatom Nitzschia palea (N. palea) inhibits the growth of the cyanobacterium Microcystis aeruginosa (M. aeruginosa); therefore, increasing the relative abundance of N. palea may contribute to the inhibition of Microcystis blooms. Several studies have demonstrated that blue light irradiation promotes diatom growth and inhibits cyanobacterial growth. In this study, we evaluated the effects of blue light irradiation on N. palea and M. aeruginosa abundance. Monocultures and co-cultures of N. palea and M. aeruginosa were exposed to blue light and fluorescent light at 32 μmol photons m-2  s-1 . The relative abundance of N. palea under fluorescent light decreased gradually, whereas the abundance under blue light was relatively higher (approximately 74% and 98% under fluorescent light and blue light, respectively, at the end of the experiment). The inhibition efficiency of blue light on the growth rate of M. aeruginosa was related to the light intensity. The optimal light intensity was considered 20 μmol photons m-2  s-1 based on the inhibition efficiency of 100%. Blue light irradiation can be used to increase the abundance of N. palea to control Microcystis blooms. PRACTITIONER POINTS: The effects of blue light irradiation on N. palea abundance was discussed. Monocultures and co-cultures of N. palea and M. aeruginosa were exposed to blue light and to fluorescent light. The relative abundance of N. palea increased upon irradiation with blue light in co-culture with M. aeruginosa.

Quantitative influence and performance analysis of virtual reality laparoscopic surgical training system

BACKGROUND

Virtual reality (VR) surgery training has become a trend in clinical education. Many research papers validate the effectiveness of VR-based surgical simulators in training medical students. However, most existing articles employ subjective methods to study the residents' surgical skills improvement. Few of them investigate how to improve the surgery skills on specific dimensions substantially.

METHODS

Our paper resorts to physiological approaches to objectively study the quantitative influence and performance analysis of VR laparoscopic surgical training system for medical students. Fifty-one participants were recruited from a pool of medical students. They conducted four pre and post experiments in the training box. They were trained on VR-based laparoscopic surgery simulators (VRLS) in the middle of pre and post experiments. Their operation and physiological data (heart rate and electroencephalogram) are recorded during the pre and post experiments. The physiological data is used to compute cognitive load and flow experience quantitatively. Senior surgeons graded their performance using newly designed hybrid standards for fundamental tasks and Global operative assessment of laparoscopic skills (GOALS) standards for colon resection tasks. Finally, the participants were required to fill the questionnaires about their cognitive load and flow experience.

RESULTS

After training on VRLS, the time of the experimental group to complete the same task could drop sharply (p < 0.01). The performance scores are enhanced significantly (p < 0.01). The performance and cognitive load computed from EEG are negatively correlated (p < 0.05).

CONCLUSION

The results show that the VRLS could highly improve medical students' performance and enable the participants to obtain flow experience with a lower cognitive load. Participants' performance is negatively correlated with cognitive load through quantitative physiological analysis. This might provide a new way of assessing skill acquirement.

Multiple roles of bamboo as a regulator of cyanobacterial bloom in aquatic systems

To understand the potential roles of terrestrial bamboo on controlling cyanobacterial blooms in aquatic systems, growth rates of the cyanobacterium Microcystis aeruginosa and its competitor algae were examined under different concentrations of bamboo extract. In mono-species cultures with unicellular algal strains, 5.0 g L-1 extract treatment suppressed M. aeruginosa growth, while it had little effect on the growth of green alga Scenedesmus obliquus or promoted the growth of diatom Nitzschia palea. In co-species cultures, the extract treatment increased the effect of S. obliquus and N. palea on the growth of M. aeruginosa. Under the extract treatment with a field-collected M. aeruginosa population, its cell density declined and its colony was etiolated and sank, while co-cultured N. palea increased explosively by invading the colony. These results suggest that bamboo forest stands along banks and artificially installed bamboo poles can affect the aquatic environment for phytoplankton community. Enhancing the growth of competitors, especially diatoms that can invade cyanobacterial colonies, by using extracts or by providing substrates for growth, was suggested to be the major way of the bloom control by bamboo.

Automatic Dental Plaque Segmentation based on Local-to-global Features Fused Self-attention Network

The accurate detection of dental plaque at an early stage will definitely prevent periodontal diseases and dental caries. However, it remains difficult for the current dental examination to accurately recognize dental plague without using medical dyeing reagent due to the low contrast between dental plaque and healthy teeth. To combat this problem, this paper proposes a novel network enhanced by a self-attention module for intelligent dental plaque segmentation. The key motivation is to directly utilize oral endoscope images (bypassing the need of dyeing reagent) and get the accurate pixel-level dental plaque segmentation results. The algorithm needs to conduct self-attention at the super-pixel level and fuse the super-pixels' local-to-global features. Our newly-designed network architecture will afford the simultaneous fusion of multiple-scale complementary information guided by the powerful deep learning paradigm. The critical fused information includes the statistical distribution of the plaques color, the heat kernel signature (HKS) based local-to-global structure relationship, and the circle-LBP based local texture pattern in the nearby regions centering around the plaque area. To further refine the fuzed multiple-scale features, we devise an attention module based on CNN, which could focalize the regions of interest in plaque more easily, especially for much challenging cases. Extensive experiments and comprehensive evaluations confirm that, for a small-scale training dataset, our method could outperform the state-of-the-art methods. Meanwhile, the user studies verify the claim that our method is more accurate than conventional dental practice conducted by experienced dentists.

Simulating Multi-Scale, Granular Materials and Their Transitions With a Hybrid Euler-Lagrange Solver

Multi-scale granular materials, such as powdered materials and mudslides, are pretty common in nature. Modeling such materials and their phase transitions remains challenging since this task involves the delicate representations of various ranges of particles with multiple scales that cause their property variations among liquid, granular solid (i.e., particles), and smoke-like materials. To effectively animate the complicated yet intriguing natural phenomena involving multi-scale granular materials and their phase transitions in graphics with high fidelity, this article advocates a hybrid Euler-Lagrange solver to handle the behaviors of involved discontinuous fluid-like materials faithfully. At the algorithmic level, we present a unified framework that tightly couples the affine particle-in-cell (APIC) solver with density field to achieve the transformation spanning across granular particles, dust cloud, powders, and their natural mixtures. For example, a part of the granular particles could be transformed into dust cloud while interacting with air and being represented by density field. Meanwhile, the velocity decrease of the involved materials could also result in the transit from the density-field-driven dust to powder particles. Besides, to further enhance our modeling and simulation power to broaden the range of multi-scale materials, we introduce a moisture property for granular particles to control the transitions between particles and viscous liquid. At the geometric level, we devise an additional surface-tracking procedure to simulate the viscous liquid phase. We can arrive at delicate viscous behaviors by controlling the corresponding yield conditions. Through various experiments with the different scenes design being conducted in our unified framework, we can validate the mixed multi-scale materials' mutual transformation processes. Our unified framework furnished with a hybrid solver can significantly enhance the modeling flexibility and the animation potential of the particle-grid hybrid materials in graphics.

Design and Evaluation of Personalized Percutaneous Coronary Intervention Surgery Simulation System

In recent years, medical simulators have been widely applied to a broad range of surgery training tasks. However, most of the existing surgery simulators can only provide limited immersive environments with a few pre-processed organ models, while ignoring the instant modeling of various personalized clinical cases, which brings substantive differences between training experiences and real surgery situations. To this end, we present a virtual reality (VR) based surgery simulation system for personalized percutaneous coronary intervention (PCI). The simulation system can directly take patient-specific clinical data as input and generate virtual 3D intervention scenarios. Specially, we introduce a fiber-based patient-specific cardiac dynamic model to simulate the nonlinear deformation among the multiple layers of the cardiac structure, which can well respect and correlate the atriums, ventricles and vessels, and thus gives rise to more effective visualization and interaction. Meanwhile, we design a tracking and haptic feedback hardware, which can enable users to manipulate physical intervention instruments and interact with virtual scenarios. We conduct quantitative analysis on deformation precision and modeling efficiency, and evaluate the simulation system based on the user studies from 16 cardiologists and 20 intervention trainees, comparing it to traditional desktop intervention simulators. The results confirm that our simulation system can provide a better user experience, and is a suitable platform for PCI surgery training and rehearsal.

Dynamics induced by environmental stochasticity in a phytoplankton-zooplankton system with toxic phytoplankton

Environmental stochasticity and toxin-producing phytoplankton (TPP) are the key factors that affect the aquatic ecosystems. To investigate the effects of environmental stochasticity and TPP on the dynamics of plankton populations, a stochastic phytoplankton-zooplankton system with two TPP is studied theoretically and numerically in this paper. Theoretically, we first prove that the system possesses a unique and global positive solution with positive initial values, and then derive some sufficient conditions guaranteeing the extinction and persistence in the mean of the system. Significantly, it is shown that the system has a stationary distribution when toxin liberation rate reaches some a critical value. Additionally, numerical analysis shows that the white noise can affect the survival of plankton populations directly. Furthermore, it has been observed that the increasing one toxin liberation rate can increase the survival chance of phytoplankton and reduce the biomass of zooplankton, but the combined effects of two liberation rates on the changes in plankton populations are stronger than that of controlling any one of the two TPP.

Rethinking Image Salient Object Detection: Object-Level Semantic Saliency Reranking First, Pixelwise Saliency Refinement Later

Human attention is an interactive activity between our visual system and our brain, using both low-level visual stimulus and high-level semantic information. Previous image salient object detection (SOD) studies conduct their saliency predictions via a multitask methodology in which pixelwise saliency regression and segmentation-like saliency refinement are conducted simultaneously. However, this multitask methodology has one critical limitation: the semantic information embedded in feature backbones might be degenerated during the training process. Our visual attention is determined mainly by semantic information, which is evidenced by our tendency to pay more attention to semantically salient regions even if these regions are not the most perceptually salient at first glance. This fact clearly contradicts the widely used multitask methodology mentioned above. To address this issue, this paper divides the SOD problem into two sequential steps. First, we devise a lightweight, weakly supervised deep network to coarsely locate the semantically salient regions. Next, as a postprocessing refinement, we selectively fuse multiple off-the-shelf deep models on the semantically salient regions identified by the previous step to formulate a pixelwise saliency map. Compared with the state-of-the-art (SOTA) models that focus on learning the pixelwise saliency in single images using only perceptual clues, our method aims at investigating the object-level semantic ranks between multiple images, of which the methodology is more consistent with the human attention mechanism. Our method is simple yet effective, and it is the first attempt to consider salient object detection as mainly an object-level semantic reranking problem.

A Global-Local Self-Adaptive Network for Drone-View Object Detection

Directly benefiting from the deep learning methods, object detection has witnessed a great performance boost in recent years. However, drone-view object detection remains challenging for two main reasons: (1) Objects of tiny-scale with more blurs w.r.t. ground-view objects offer less valuable information towards accurate and robust detection; (2) The unevenly distributed objects make the detection inefficient, especially for regions occupied by crowded objects. Confronting such challenges, we propose an end-to-end global-local self-adaptive network (GLSAN) in this paper. The key components in our GLSAN include a global-local detection network (GLDN), a simple yet efficient self-adaptive region selecting algorithm (SARSA), and a local super-resolution network (LSRN). We integrate a global-local fusion strategy into a progressive scale-varying network to perform more precise detection, where the local fine detector can adaptively refine the target's bounding boxes detected by the global coarse detector via cropping the original images for higher-resolution detection. The SARSA can dynamically crop the crowded regions in the input images, which is unsupervised and can be easily plugged into the networks. Additionally, we train the LSRN to enlarge the cropped images, providing more detailed information for finer-scale feature extraction, helping the detector distinguish foreground and background more easily. The SARSA and LSRN also contribute to data augmentation towards network training, which makes the detector more robust. Extensive experiments and comprehensive evaluations on the VisDrone2019-DET benchmark dataset and UAVDT dataset demonstrate the effectiveness and adaptivity of our method. Towards an industrial application, our network is also applied to a DroneBolts dataset with proven advantages. Our source codes have been available at https://github.com/dengsutao/glsan.

Vectorized Painting with Temporal Diffusion Curves

This paper presents a vector painting system for digital artworks. We first propose Temporal Diffusion Curve (TDC), a new form of vector graphics, and a novel random-access solver for modeling the evolution of strokes. With the help of a procedural stroke processing function, the TDC strokes can achieve various shapes and effects for multiple art styles. Based on these, we build a painting system of great potential. Thanks to the random-access solver, our method has real-time performance regardless of the rendering resolution, provides straightforward editing possibilities on strokes both at runtime and afterward, and is effective and straightforward for art production. Compared with the previous Diffusion Curve, our method uses strokes as the basic graphics primitives, which are able to intersect each other and much more consistent with the intuition and painting habits of human. We finally demonstrate that professional artists can create multiple genres of artworks with our painting system.

Stage-wise Salient Object Detection in 360° Omnidirectional Image via Object-level Semantical Saliency Ranking

The 2D image based salient object detection (SOD) has been extensively explored, while the 360° omnidirectional image based SOD has received less research attention and there exist three major bottlenecks that are limiting its performance. Firstly, the currently available training data is insufficient for the training of 360° SOD deep model. Secondly, the visual distortions in 360° omnidirectional images usually result in large feature gap between 360° images and 2D images; consequently, the widely used stage-wise training-a widely-used solution to alleviate the training data shortage problem, becomes infeasible when conducing SOD in 360° omnidirectional images. Thirdly, the existing 360° SOD approach has followed a multi-task methodology that performs salient object localization and segmentation-like saliency refinement at the same time, being faced with extremely large problem domain, making the training data shortage dilemma even worse. To tackle all these issues, this paper divides the 360° SOD into a multi-staqe task, the key rationale of which is to decompose the original complex problem domain into sequential easy sub problems that only demand for small-scale training data. Meanwhile, we learn how to rank the "object-level semantical saliency", aiming to locate salient viewpoints and objects accurately. Specifically, to alleviate the training data shortage problem, we have released a novel dataset named 360-SSOD, containing 1,105 360° omnidirectional images with manually annotated object-level saliency ground truth, whose semantical distribution is more balanced than that of the existing dataset. Also, we have compared the proposed method with 13 SOTA methods, and all quantitative results have demonstrated the performance superiority.

Data-Level Recombination and Lightweight Fusion Scheme for RGB-D Salient Object Detection

Existing RGB-D salient object detection methods treat depth information as an independent component to complement RGB and widely follow the bistream parallel network architecture. To selectively fuse the CNN features extracted from both RGB and depth as a final result, the state-of-the-art (SOTA) bistream networks usually consist of two independent subbranches: one subbranch is used for RGB saliency, and the other aims for depth saliency. However, depth saliency is persistently inferior to the RGB saliency because the RGB component is intrinsically more informative than the depth component. The bistream architecture easily biases its subsequent fusion procedure to the RGB subbranch, leading to a performance bottleneck. In this paper, we propose a novel data-level recombination strategy to fuse RGB with D (depth) before deep feature extraction, where we cyclically convert the original 4-dimensional RGB-D into DGB, RDB and RGD. Then, a newly lightweight designed triple-stream network is applied over these novel formulated data to achieve an optimal channel-wise complementary fusion status between the RGB and D, achieving a new SOTA performance.

Effects of substrate and water depth of a eutrophic pond on the physiological status of a submerged plant, Vallisneria natans

Effects of substrate and water depth on the physiological status of a submerged macrophyte, Vallisneria natans (Lour.) H. Hara, were determined by measuring biomarkers in leaves and roots, to understand factors limiting the re-establishment of V. natans in urban eutrophic ponds. Ramets of V. natans were grown in the laboratory using aquaria containing water and bottom mud from a eutrophic pond and maintained under sufficient light in an incubator. The growth and chlorophyll-a (Chl-a) content of leaves were greater in aquaria with mud than in those with sand, which was used as the reference substrate. The contents of a peroxidation product (malondialdehyde (MDA)) and three antioxidant enzymes (superoxide dismutase (SOD), catalase (CAT), and peroxidase (POD)) in leaves and roots, used as stress biomarkers, changed during the experiment, although differences in these contents between mud and sand were not consistent across the experimental days. To control water depth in the field, ramets of V. natans were grown in cages with different substrates (mud and sand) installed at different depths (0.5, 1.2, and 2.0 m) in the pond. The mean light quantum during the experiment decreased with increasing depth, from 79.3 μmol/m² s at 0.5 m to 7.9 μmol/m² s at 2.0 m. The Chl-a content in leaves decreased, whereas the MDA content in both leaves and roots increased with increasing water depth. All enzyme activities increased at the beginning and then decreased to the end of the experiment at 2.0 m depth, suggesting deterioration of enzyme activities due to depth-related stress. The MDA content and CAT activity were higher for sand than for mud, whereas the difference in the growth and the leaf Chl-a content between substrates remained unclear in the pond. On comparing the laboratory and field experiments, the leaf Chl-a content was found to be lower and the MDA content and enzyme activities exhibited sharp increase for ramets grown in the pond, even at 0.5 m depth, when compared with those grown in the aquaria. Our results suggest that the bottom mud of the pond is not the major limiting factor in the re-establishment of V. natans. Because water depth and light attenuation exerted strong stress on V. natans, shallow areas or measures to improve water transparency are required to promote the introduction of V. natans in eutrophic ponds for successful restoration in urban areas.

A Novel Pathological Images and Genomic Data Fusion Framework for Breast Cancer Survival Prediction

Survival analysis is a valid solution for cancer treatments and outcome evaluations. Due to the wide application of medical imaging and genome technology, computer-aided survival analysis has become a popular and promising area, from which we can get relatively satisfactory results. Although there are already some impressive technologies in this field, most of them make some recommendations using single-source medical data and have not combined multi-level and multi-source data efficiently. In this paper, we propose a novel pathological images and gene expression data fusion framework to perform the survival prediction. Different from previous methods, our framework can extract correlated multi-scale deep features from whole slide images (WSIs) and dimensionality reduced gene expression data respectively for jointly survival analysis. The experiment results demonstrate that the integrated multi-level image and genome features can achieve higher prediction accuracy compared with single-source features.

Fluid Simulation with Adaptive Staggered Power Particles on GPUs

This paper extends the recently proposed power-particle-based fluid simulation method with staggered discretization, GPU implementation, and adaptive sampling, largely enhancing the efficiency and usability of the method. In contrast to the original formulation which uses co-located pressures and velocities, in this paper, a staggered scheme is adapted to the Power Particles to benefit visual details and computing efficiency. Meanwhile, we propose a novel facet-based power diagrams construction algorithm suitable for parallelization and explore its GPU implementation, achieving an order of magnitude boost in performance over the existing code library. In addition, to utilize the potential of Power Particles to control individual cell volume, we apply adaptive particle sampling to improve the detail level with varying resolution. The proposed method can be entirely carried out on GPUs, and our extensive experiments validate our method both in terms of efficiency and visual quality.

Deep learning-based dental plaque detection on primary teeth: a comparison with clinical assessments

BACKGROUND

Dental plaque causes many common oral diseases (e.g., caries, gingivitis, and periodontitis). Therefore, plaque detection and control are extremely important for children's oral health. The objectives of this study were to design a deep learning-based artificial intelligence (AI) model to detect plaque on primary teeth and to evaluate the diagnostic accuracy of the model.

METHODS

A conventional neural network (CNN) framework was adopted, and 886 intraoral photos of primary teeth were used for training. To validate clinical feasibility, 98 intraoral photos of primary teeth were assessed by the AI model. Additionally, tooth photos were acquired using a digital camera. One experienced pediatric dentist examined the photos and marked the regions containing plaque. Then, a plaque-disclosing agent was applied, and the areas with plaque were identified. After 1 week, the dentist drew the plaque area on the 98 photos taken by the digital camera again to evaluate the consistency of manual diagnosis. Additionally, 102 intraoral photos of primary teeth were marked to denote the plaque areas obtained by the AI model and the dentist to evaluate the diagnostic capacity of each approach based on lower-resolution photos. The mean intersection-over-union (MIoU) metric was employed to indicate detection accuracy.

RESULTS

The MIoU for detecting plaque on the tested tooth photos was 0.726 ± 0.165. The dentist's MIoU was 0.695 ± 0.269 when first diagnosing the 98 photos taken by the digital camera and 0.689 ± 0.253 after 1 week. Compared to the dentist, the AI model demonstrated a higher MIoU (0.736 ± 0.174), and the results did not change after 1 week. When the dentist and the AI model assessed the 102 intraoral photos, the MIoU was 0.652 ± 0.195 for the dentist and 0.724 ± 0.159 for the model. The results of a paired t-test found no significant difference between the AI model and human specialist (P > .05) in diagnosing dental plaque on primary teeth.

CONCLUSIONS

The AI model showed clinically acceptable performance in detecting dental plaque on primary teeth compared with an experienced pediatric dentist. This finding illustrates the potential of such AI technology to help improve pediatric oral health.

The impact of financial contagion on real economy-An empirical research based on combination of complex network technology and spatial econometrics model

This study presents financial network indicators that can be applied to inspect the financial contagion on real economy, as well as the spatial spillover and industry aggregation effects. We propose to design both a directed and undirected networks of financial sectors of top 20 countries in GDP based on symbolized transfer entropy and Pearson correlation coefficients. We examine the effect and usefulness of the network indicators by newly using them instead of the original Dow Jones financial sector as explanatory variables to construct the higher-order information spatial econometric models. The results demonstrate that the estimated accuracies obtained from both the two networks are improved significantly compared with the spatial econometric model using the original data. It indicates that the network indictors are more effective to capture the dynamic information of financial systems. And meanwhile, the accuracy based on the directed network is a little higher than the undirected network, which indicates the symbolized transfer entropy, i.e. the directed and weighted network, is more suitable and effective to reflect relationships in the financial field. In addition, the results also show that under the global financial crisis, the co-movement between financial sectors of a country/region and the global financial sector as well as between financial sectors and real economy sectors is increased. However, some sectors in particular Utilities and Healthcare are impacted slightly. This study tries to use the financial network indicators in modeling to study contagion channels on the real economy and the industry aggregation effects and suggest how network indicators can be practically used in financial fields.

Poisson Vector Graphics (PVG)

This paper presents Poisson vector graphics (PVG), an extension of the popular diffusion curves (DC), for generating smooth-shaded images. Armed with two new types of primitives, called Poisson curves and Poisson regions, PVG can easily produce photorealistic effects such as specular highlights, core shadows, translucency and halos. Within the PVG framework, the users specify color as the Dirichlet boundary condition of diffusion curves and control tone by offsetting the Laplacian of colors, where both controls are simply done by mouse click and slider dragging. PVG distinguishes itself from other diffusion based vector graphics for 3 unique features: 1) explicit separation of colors and tones, which follows the basic drawing principle and eases editing; 2) native support of seamless cloning in the sense that PCs and PRs can automatically fit into the target background; and 3) allowed intersecting primitives (except for DC-DC intersection) so that users can create layers. Through extensive experiments and a preliminary user study, we demonstrate that PVG is a simple yet powerful authoring tool that can produce photo-realistic vector graphics from scratch.

Context-Interactive CNN for Person Re-Identification

Despite growing progresses in recent years, cross-scenario person re-identification remains challenging, mainly due to the pedestrians commonly surrounded by highly-complex environment contexts. In reality, the human perception mechanism could adaptively find proper contextualized spatial-temporal clues towards pedestrian recognition. However, conventional methods fall short in adaptively leveraging the long-term spatial-temporal information due to ever-increasing computational cost. Moreover, CNN-based deep learning methods are hard to conduct optimization due to the non-differentiable property of the built-in context search operation. To ameliorate, this paper proposes a novel Context-Interactive CNN (CI-CNN) to dynamically find both spatial and temporal contexts by embedding multi-task Reinforcement Learning (MTRL). The CI-CNN streamlines the multi-task reinforcement learning by using an actor-critic agent to capture the temporal-spatial context simultaneously, which comprises a context-policy network and a context-critic network. The former network learns policies to determine the optimal spatial context region and temporal sequence range. Based on the inferred temporal-spatial cues, the latter one focuses on the identification task and provides feedback for the policy network. Thus, CI-CNN can simultaneously zoom in/out the perception field in spatial and temporal domain for the context interaction with the environment. By fostering the collaborative interaction between the person and context, our method could achieve outstanding performance on various public benchmarks, which confirms the rationality of our hypothesis, and verifies the effectiveness of our CI-CNN framework.

Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition

Thyroid ultrasonography is a widely used clinical technique for nodule diagnosis in thyroid regions. However, it remains difficult to detect and recognize the nodules due to low contrast, high noise, and diverse appearance of nodules. In today's clinical practice, senior doctors could pinpoint nodules by analyzing global context features, local geometry structure, and intensity changes, which would require rich clinical experience accumulated from hundreds and thousands of nodule case studies. To alleviate doctors' tremendous labor in the diagnosis procedure, we advocate a machine learning approach to the detection and recognition tasks in this paper. In particular, we develop a multitask cascade convolution neural network (MC-CNN) framework to exploit the context information of thyroid nodules. It may be noted that our framework is built upon a large number of clinically confirmed thyroid ultrasound images with accurate and detailed ground truth labels. Other key advantages of our framework result from a multitask cascade architecture, two stages of carefully designed deep convolution networks in order to detect and recognize thyroid nodules in a pyramidal fashion, and capturing various intrinsic features in a global-to-local way. Within our framework, the potential regions of interest after initial detection are further fed to the spatial pyramid augmented CNNs to embed multiscale discriminative information for fine-grained thyroid recognition. Experimental results on 4309 clinical ultrasound images have indicated that our MC-CNN is accurate and effective for both thyroid nodules detection and recognition. For the correct diagnosis rate of malignant and benign thyroid nodules, its mean Average Precision (mAP) performance can achieve up to [Formula: see text] accuracy, which outperforms the common CNNs by [Formula: see text] on average. In addition, we conduct rigorous user studies to confirm that our MC-CNN outperforms experienced doctors, yet only consuming roughly [Formula: see text] ( 1/48) of doctors' examination time on average. Therefore, the accuracy and efficiency of our new method exhibit its great potential in clinical applications.

Computer-aided system for diagnosing thyroid nodules on ultrasound: A comparison with radiologist-based clinical assessments

BACKGROUND

The purpose of this study was to compare the diagnostic efficiency of a thyroid ultrasound computer-aided diagnosis (CAD) system with that of 1 radiologist.

METHODS

This study retrospectively reviewed 342 surgically resected thyroid nodules from July 2013 to December 2013 at our center. The nodules were assessed on typical ultrasound images using the CAD system and reviewed by 1 experienced radiologist. The radiologist stratified the risk of malignancy using the Thyroid Imaging Reporting and Data Systems (TIRADS) and the American Thyroid Association (ATA) guidelines.

RESULTS

The radiologist, using TI-RADS and ATA guidelines, performed better than the CAD system (P < .01). The sensitivity of the CAD system was similar to that of an experienced radiologist (P > .05; P < .01; and P > .05). However, we found that the CAD system had lower specificity (P < .01).

CONCLUSION

The sensitivity of a thyroid ultrasound CAD system in differentiating nodules was similar to that of an experienced radiologist. However, the CAD system had lower specificity.

Knot Optimization for Biharmonic B-splines on Manifold Triangle Meshes

Biharmonic B-splines, proposed by Feng and Warren, are an elegant generalization of univariate B-splines to planar and curved domains with fully irregular knot configuration. Despite the theoretic breakthrough, certain technical difficulties are imperative, including the necessity of Voronoi tessellation, the lack of analytical formulation of bases on general manifolds, expensive basis re-computation during knot refinement/removal, being applicable for simple domains only (e.g., such as euclidean planes, spherical and cylindrical domains, and tori). To ameliorate, this paper articulates a new biharmonic B-spline computing paradigm with a simple formulation. We prove that biharmonic B-splines have an equivalent representation, which is solely based on a linear combination of Green's functions of the bi-Laplacian operator. Consequently, without explicitly computing their bases, biharmonic B-splines can bypass the Voronoi partitioning and the discretization of bi-Laplacian, enable the computational utilities on any compact 2-manifold. The new representation also facilitates optimization-driven knot selection for constructing biharmonic B-splines on manifold triangle meshes. We develop algorithms for spline evaluation, data interpolation and hierarchical data decomposition. Our results demonstrate that biharmonic B-splines, as a new type of spline functions with theoretic and application appeal, afford progressive update of fully irregular knots, free of singularity, without the need of explicit parameterization, making it ideal for a host of graphics tasks on manifolds.

Video Saliency Detection via Spatial-Temporal Fusion and Low-Rank Coherency Diffusion

This paper advocates a novel video saliency detection method based on the spatial-temporal saliency fusion and low-rank coherency guided saliency diffusion. In sharp contrast to the conventional methods, which conduct saliency detection locally in a frame-by-frame way and could easily give rise to incorrect low-level saliency map, in order to overcome the existing difficulties, this paper proposes to fuse the color saliency based on global motion clues in a batch-wise fashion. And we also propose low-rank coherency guided spatial-temporal saliency diffusion to guarantee the temporal smoothness of saliency maps. Meanwhile, a series of saliency boosting strategies are designed to further improve the saliency accuracy. First, the original long-term video sequence is equally segmented into many short-term frame batches, and the motion clues of the individual video batch are integrated and diffused temporally to facilitate the computation of color saliency. Then, based on the obtained saliency clues, inter-batch saliency priors are modeled to guide the low-level saliency fusion. After that, both the raw color information and the fused low-level saliency are regarded as the low-rank coherency clues, which are employed to guide the spatial-temporal saliency diffusion with the help of an additional permutation matrix serving as the alternative rank selection strategy. Thus, it could guarantee the robustness of the saliency map's temporal consistence, and further boost the accuracy of the computed saliency map. Moreover, we conduct extensive experiments on five public available benchmarks, and make comprehensive, quantitative evaluations between our method and 16 state-of-the-art techniques. All the results demonstrate the superiority of our method in accuracy, reliability, robustness, and versatility.

Unsupervised Multi-Class Co-Segmentation via Joint-Cut Over $L_{1}$ -Manifold Hyper-Graph of Discriminative Image Regions

This paper systematically advocates a robust and efficient unsupervised multi-class co-segmentation approach by leveraging underlying subspace manifold propagation to exploit the cross-image coherency. It can combat certain image co-segmentation difficulties due to viewpoint change, partial occlusion, complex background, transient illumination, and cluttering texture patterns. Our key idea is to construct a powerful hyper-graph joint-cut framework, which incorporates mid-level image regions-based intra-image feature representation and L₁-manifold graph-based inter-image coherency exploration. For local image region generation, we propose a bi-harmonic distance distribution difference metric to govern the super-pixel clustering in a bottom-up way. It not only affords drastic data reduction but also gives rise to discriminative and structure meaningful feature representation. As for the inter-image coherency, we leverage multi-type features involved L₁-graph to detect the underlying local manifold from cross-image regions. As a result, the implicit supervising information could be encoded into the unsupervised hyper-graph joint-cut framework. We conduct extensive experiments and make comprehensive evaluations with other state-of-the-art methods over various benchmarks, including iCoseg, MSRC, and Oxford flower. All the results demonstrate the superiorities of our method in terms of accuracy, robustness, efficiency, and versatility.