26
|
Liu H, Dai C, Yu H, Guo Q, Li J, Hao A, Kikuchi J, Zhao M. Dynamics induced by environmental stochasticity in a phytoplankton-zooplankton system with toxic phytoplankton. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2021; 18:4101-4126. [PMID: 34198428 DOI: 10.3934/mbe.2021206] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Environmental stochasticity and toxin-producing phytoplankton (TPP) are the key factors that affect the aquatic ecosystems. To investigate the effects of environmental stochasticity and TPP on the dynamics of plankton populations, a stochastic phytoplankton-zooplankton system with two TPP is studied theoretically and numerically in this paper. Theoretically, we first prove that the system possesses a unique and global positive solution with positive initial values, and then derive some sufficient conditions guaranteeing the extinction and persistence in the mean of the system. Significantly, it is shown that the system has a stationary distribution when toxin liberation rate reaches some a critical value. Additionally, numerical analysis shows that the white noise can affect the survival of plankton populations directly. Furthermore, it has been observed that the increasing one toxin liberation rate can increase the survival chance of phytoplankton and reduce the biomass of zooplankton, but the combined effects of two liberation rates on the changes in plankton populations are stronger than that of controlling any one of the two TPP.
Collapse
|
27
|
Ma G, Li S, Chen C, Hao A, Qin H. Rethinking Image Salient Object Detection: Object-Level Semantic Saliency Reranking First, Pixelwise Saliency Refinement Later. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2021; 30:4238-4252. [PMID: 33819154 DOI: 10.1109/tip.2021.3068649] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Human attention is an interactive activity between our visual system and our brain, using both low-level visual stimulus and high-level semantic information. Previous image salient object detection (SOD) studies conduct their saliency predictions via a multitask methodology in which pixelwise saliency regression and segmentation-like saliency refinement are conducted simultaneously. However, this multitask methodology has one critical limitation: the semantic information embedded in feature backbones might be degenerated during the training process. Our visual attention is determined mainly by semantic information, which is evidenced by our tendency to pay more attention to semantically salient regions even if these regions are not the most perceptually salient at first glance. This fact clearly contradicts the widely used multitask methodology mentioned above. To address this issue, this paper divides the SOD problem into two sequential steps. First, we devise a lightweight, weakly supervised deep network to coarsely locate the semantically salient regions. Next, as a postprocessing refinement, we selectively fuse multiple off-the-shelf deep models on the semantically salient regions identified by the previous step to formulate a pixelwise saliency map. Compared with the state-of-the-art (SOTA) models that focus on learning the pixelwise saliency in single images using only perceptual clues, our method aims at investigating the object-level semantic ranks between multiple images, of which the methodology is more consistent with the human attention mechanism. Our method is simple yet effective, and it is the first attempt to consider salient object detection as mainly an object-level semantic reranking problem.
Collapse
|
28
|
Wang X, Li S, Chen C, Hao A, Qin H. Depth quality-aware selective saliency fusion for RGB-D image salient object detection. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2020.12.071] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
29
|
Deng S, Li S, Xie K, Song W, Liao X, Hao A, Qin H. A Global-Local Self-Adaptive Network for Drone-View Object Detection. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2021; 30:1556-1569. [PMID: 33360993 DOI: 10.1109/tip.2020.3045636] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Directly benefiting from the deep learning methods, object detection has witnessed a great performance boost in recent years. However, drone-view object detection remains challenging for two main reasons: (1) Objects of tiny-scale with more blurs w.r.t. ground-view objects offer less valuable information towards accurate and robust detection; (2) The unevenly distributed objects make the detection inefficient, especially for regions occupied by crowded objects. Confronting such challenges, we propose an end-to-end global-local self-adaptive network (GLSAN) in this paper. The key components in our GLSAN include a global-local detection network (GLDN), a simple yet efficient self-adaptive region selecting algorithm (SARSA), and a local super-resolution network (LSRN). We integrate a global-local fusion strategy into a progressive scale-varying network to perform more precise detection, where the local fine detector can adaptively refine the target's bounding boxes detected by the global coarse detector via cropping the original images for higher-resolution detection. The SARSA can dynamically crop the crowded regions in the input images, which is unsupervised and can be easily plugged into the networks. Additionally, we train the LSRN to enlarge the cropped images, providing more detailed information for finer-scale feature extraction, helping the detector distinguish foreground and background more easily. The SARSA and LSRN also contribute to data augmentation towards network training, which makes the detector more robust. Extensive experiments and comprehensive evaluations on the VisDrone2019-DET benchmark dataset and UAVDT dataset demonstrate the effectiveness and adaptivity of our method. Towards an industrial application, our network is also applied to a DroneBolts dataset with proven advantages. Our source codes have been available at https://github.com/dengsutao/glsan.
Collapse
|
30
|
Li Y, Zhai X, Hou F, Liu Y, Hao A, Qin H. Vectorized Painting with Temporal Diffusion Curves. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2021; 27:228-240. [PMID: 31329122 DOI: 10.1109/tvcg.2019.2929808] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
This paper presents a vector painting system for digital artworks. We first propose Temporal Diffusion Curve (TDC), a new form of vector graphics, and a novel random-access solver for modeling the evolution of strokes. With the help of a procedural stroke processing function, the TDC strokes can achieve various shapes and effects for multiple art styles. Based on these, we build a painting system of great potential. Thanks to the random-access solver, our method has real-time performance regardless of the rendering resolution, provides straightforward editing possibilities on strokes both at runtime and afterward, and is effective and straightforward for art production. Compared with the previous Diffusion Curve, our method uses strokes as the basic graphics primitives, which are able to intersect each other and much more consistent with the intuition and painting habits of human. We finally demonstrate that professional artists can create multiple genres of artworks with our painting system.
Collapse
|
31
|
Ma G, Li S, Chen C, Hao A, Qin H. Stage-wise Salient Object Detection in 360° Omnidirectional Image via Object-level Semantical Saliency Ranking. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2020; 26:3535-3545. [PMID: 32941153 DOI: 10.1109/tvcg.2020.3023636] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/15/2023]
Abstract
The 2D image based salient object detection (SOD) has been extensively explored, while the 360° omnidirectional image based SOD has received less research attention and there exist three major bottlenecks that are limiting its performance. Firstly, the currently available training data is insufficient for the training of 360° SOD deep model. Secondly, the visual distortions in 360° omnidirectional images usually result in large feature gap between 360° images and 2D images; consequently, the widely used stage-wise training-a widely-used solution to alleviate the training data shortage problem, becomes infeasible when conducing SOD in 360° omnidirectional images. Thirdly, the existing 360° SOD approach has followed a multi-task methodology that performs salient object localization and segmentation-like saliency refinement at the same time, being faced with extremely large problem domain, making the training data shortage dilemma even worse. To tackle all these issues, this paper divides the 360° SOD into a multi-staqe task, the key rationale of which is to decompose the original complex problem domain into sequential easy sub problems that only demand for small-scale training data. Meanwhile, we learn how to rank the "object-level semantical saliency", aiming to locate salient viewpoints and objects accurately. Specifically, to alleviate the training data shortage problem, we have released a novel dataset named 360-SSOD, containing 1,105 360° omnidirectional images with manually annotated object-level saliency ground truth, whose semantical distribution is more balanced than that of the existing dataset. Also, we have compared the proposed method with 13 SOTA methods, and all quantitative results have demonstrated the performance superiority.
Collapse
|
32
|
Wang X, Li S, Chen C, Fang Y, Hao A, Qin H. Data-Level Recombination and Lightweight Fusion Scheme for RGB-D Salient Object Detection. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2020; 30:458-471. [PMID: 33201813 DOI: 10.1109/tip.2020.3037470] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Existing RGB-D salient object detection methods treat depth information as an independent component to complement RGB and widely follow the bistream parallel network architecture. To selectively fuse the CNN features extracted from both RGB and depth as a final result, the state-of-the-art (SOTA) bistream networks usually consist of two independent subbranches: one subbranch is used for RGB saliency, and the other aims for depth saliency. However, depth saliency is persistently inferior to the RGB saliency because the RGB component is intrinsically more informative than the depth component. The bistream architecture easily biases its subsequent fusion procedure to the RGB subbranch, leading to a performance bottleneck. In this paper, we propose a novel data-level recombination strategy to fuse RGB with D (depth) before deep feature extraction, where we cyclically convert the original 4-dimensional RGB-D into DGB, RDB and RGD. Then, a newly lightweight designed triple-stream network is applied over these novel formulated data to achieve an optimal channel-wise complementary fusion status between the RGB and D, achieving a new SOTA performance.
Collapse
|
33
|
Hao A, Kobayashi S, Huang H, Mi Q, Iseri Y. Effects of substrate and water depth of a eutrophic pond on the physiological status of a submerged plant, Vallisneria natans. PeerJ 2020; 8:e10273. [PMID: 33240623 PMCID: PMC7659635 DOI: 10.7717/peerj.10273] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2020] [Accepted: 10/08/2020] [Indexed: 11/20/2022] Open
Abstract
Effects of substrate and water depth on the physiological status of a submerged macrophyte, Vallisneria natans (Lour.) H. Hara, were determined by measuring biomarkers in leaves and roots, to understand factors limiting the re-establishment of V. natans in urban eutrophic ponds. Ramets of V. natans were grown in the laboratory using aquaria containing water and bottom mud from a eutrophic pond and maintained under sufficient light in an incubator. The growth and chlorophyll-a (Chl-a) content of leaves were greater in aquaria with mud than in those with sand, which was used as the reference substrate. The contents of a peroxidation product (malondialdehyde (MDA)) and three antioxidant enzymes (superoxide dismutase (SOD), catalase (CAT), and peroxidase (POD)) in leaves and roots, used as stress biomarkers, changed during the experiment, although differences in these contents between mud and sand were not consistent across the experimental days. To control water depth in the field, ramets of V. natans were grown in cages with different substrates (mud and sand) installed at different depths (0.5, 1.2, and 2.0 m) in the pond. The mean light quantum during the experiment decreased with increasing depth, from 79.3 μmol/m2 s at 0.5 m to 7.9 μmol/m2 s at 2.0 m. The Chl-a content in leaves decreased, whereas the MDA content in both leaves and roots increased with increasing water depth. All enzyme activities increased at the beginning and then decreased to the end of the experiment at 2.0 m depth, suggesting deterioration of enzyme activities due to depth-related stress. The MDA content and CAT activity were higher for sand than for mud, whereas the difference in the growth and the leaf Chl-a content between substrates remained unclear in the pond. On comparing the laboratory and field experiments, the leaf Chl-a content was found to be lower and the MDA content and enzyme activities exhibited sharp increase for ramets grown in the pond, even at 0.5 m depth, when compared with those grown in the aquaria. Our results suggest that the bottom mud of the pond is not the major limiting factor in the re-establishment of V. natans. Because water depth and light attenuation exerted strong stress on V. natans, shallow areas or measures to improve water transparency are required to promote the introduction of V. natans in eutrophic ponds for successful restoration in urban areas.
Collapse
|
34
|
Li S, Shi H, Sui D, Hao A, Qin H. A Novel Pathological Images and Genomic Data Fusion Framework for Breast Cancer Survival Prediction. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:1384-1387. [PMID: 33018247 DOI: 10.1109/embc44109.2020.9176360] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Survival analysis is a valid solution for cancer treatments and outcome evaluations. Due to the wide application of medical imaging and genome technology, computer-aided survival analysis has become a popular and promising area, from which we can get relatively satisfactory results. Although there are already some impressive technologies in this field, most of them make some recommendations using single-source medical data and have not combined multi-level and multi-source data efficiently. In this paper, we propose a novel pathological images and gene expression data fusion framework to perform the survival prediction. Different from previous methods, our framework can extract correlated multi-scale deep features from whole slide images (WSIs) and dimensionality reduced gene expression data respectively for jointly survival analysis. The experiment results demonstrate that the integrated multi-level image and genome features can achieve higher prediction accuracy compared with single-source features.
Collapse
|
35
|
Zhai X, Hou F, Qin H, Hao A. Fluid Simulation with Adaptive Staggered Power Particles on GPUs. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2020; 26:2234-2246. [PMID: 30561345 DOI: 10.1109/tvcg.2018.2886322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
This paper extends the recently proposed power-particle-based fluid simulation method with staggered discretization, GPU implementation, and adaptive sampling, largely enhancing the efficiency and usability of the method. In contrast to the original formulation which uses co-located pressures and velocities, in this paper, a staggered scheme is adapted to the Power Particles to benefit visual details and computing efficiency. Meanwhile, we propose a novel facet-based power diagrams construction algorithm suitable for parallelization and explore its GPU implementation, achieving an order of magnitude boost in performance over the existing code library. In addition, to utilize the potential of Power Particles to control individual cell volume, we apply adaptive particle sampling to improve the detail level with varying resolution. The proposed method can be entirely carried out on GPUs, and our extensive experiments validate our method both in terms of efficiency and visual quality.
Collapse
|
36
|
You W, Hao A, Li S, Wang Y, Xia B. Deep learning-based dental plaque detection on primary teeth: a comparison with clinical assessments. BMC Oral Health 2020; 20:141. [PMID: 32404094 PMCID: PMC7222297 DOI: 10.1186/s12903-020-01114-6] [Citation(s) in RCA: 37] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2020] [Accepted: 04/14/2020] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Dental plaque causes many common oral diseases (e.g., caries, gingivitis, and periodontitis). Therefore, plaque detection and control are extremely important for children's oral health. The objectives of this study were to design a deep learning-based artificial intelligence (AI) model to detect plaque on primary teeth and to evaluate the diagnostic accuracy of the model. METHODS A conventional neural network (CNN) framework was adopted, and 886 intraoral photos of primary teeth were used for training. To validate clinical feasibility, 98 intraoral photos of primary teeth were assessed by the AI model. Additionally, tooth photos were acquired using a digital camera. One experienced pediatric dentist examined the photos and marked the regions containing plaque. Then, a plaque-disclosing agent was applied, and the areas with plaque were identified. After 1 week, the dentist drew the plaque area on the 98 photos taken by the digital camera again to evaluate the consistency of manual diagnosis. Additionally, 102 intraoral photos of primary teeth were marked to denote the plaque areas obtained by the AI model and the dentist to evaluate the diagnostic capacity of each approach based on lower-resolution photos. The mean intersection-over-union (MIoU) metric was employed to indicate detection accuracy. RESULTS The MIoU for detecting plaque on the tested tooth photos was 0.726 ± 0.165. The dentist's MIoU was 0.695 ± 0.269 when first diagnosing the 98 photos taken by the digital camera and 0.689 ± 0.253 after 1 week. Compared to the dentist, the AI model demonstrated a higher MIoU (0.736 ± 0.174), and the results did not change after 1 week. When the dentist and the AI model assessed the 102 intraoral photos, the MIoU was 0.652 ± 0.195 for the dentist and 0.724 ± 0.159 for the model. The results of a paired t-test found no significant difference between the AI model and human specialist (P > .05) in diagnosing dental plaque on primary teeth. CONCLUSIONS The AI model showed clinically acceptable performance in detecting dental plaque on primary teeth compared with an experienced pediatric dentist. This finding illustrates the potential of such AI technology to help improve pediatric oral health.
Collapse
|
37
|
Chen X, Hao A, Li Y. The impact of financial contagion on real economy-An empirical research based on combination of complex network technology and spatial econometrics model. PLoS One 2020; 15:e0229913. [PMID: 32142544 PMCID: PMC7059932 DOI: 10.1371/journal.pone.0229913] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2019] [Accepted: 02/17/2020] [Indexed: 11/18/2022] Open
Abstract
This study presents financial network indicators that can be applied to inspect the financial contagion on real economy, as well as the spatial spillover and industry aggregation effects. We propose to design both a directed and undirected networks of financial sectors of top 20 countries in GDP based on symbolized transfer entropy and Pearson correlation coefficients. We examine the effect and usefulness of the network indicators by newly using them instead of the original Dow Jones financial sector as explanatory variables to construct the higher-order information spatial econometric models. The results demonstrate that the estimated accuracies obtained from both the two networks are improved significantly compared with the spatial econometric model using the original data. It indicates that the network indictors are more effective to capture the dynamic information of financial systems. And meanwhile, the accuracy based on the directed network is a little higher than the undirected network, which indicates the symbolized transfer entropy, i.e. the directed and weighted network, is more suitable and effective to reflect relationships in the financial field. In addition, the results also show that under the global financial crisis, the co-movement between financial sectors of a country/region and the global financial sector as well as between financial sectors and real economy sectors is increased. However, some sectors in particular Utilities and Healthcare are impacted slightly. This study tries to use the financial network indicators in modeling to study contagion channels on the real economy and the industry aggregation effects and suggest how network indicators can be practically used in financial fields.
Collapse
|
38
|
Hou F, Sun Q, Fang Z, Liu YJ, Hu SM, Qin H, Hao A, He Y. Poisson Vector Graphics (PVG). IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2020; 26:1361-1371. [PMID: 30176598 DOI: 10.1109/tvcg.2018.2867478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
This paper presents Poisson vector graphics (PVG), an extension of the popular diffusion curves (DC), for generating smooth-shaded images. Armed with two new types of primitives, called Poisson curves and Poisson regions, PVG can easily produce photorealistic effects such as specular highlights, core shadows, translucency and halos. Within the PVG framework, the users specify color as the Dirichlet boundary condition of diffusion curves and control tone by offsetting the Laplacian of colors, where both controls are simply done by mouse click and slider dragging. PVG distinguishes itself from other diffusion based vector graphics for 3 unique features: 1) explicit separation of colors and tones, which follows the basic drawing principle and eases editing; 2) native support of seamless cloning in the sense that PCs and PRs can automatically fit into the target background; and 3) allowed intersecting primitives (except for DC-DC intersection) so that users can create layers. Through extensive experiments and a preliminary user study, we demonstrate that PVG is a simple yet powerful authoring tool that can produce photo-realistic vector graphics from scratch.
Collapse
|
39
|
Song W, Li S, Chang T, Hao A, Zhao Q, Qin H. Context-Interactive CNN for Person Re-Identification. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2019; 29:2860-2874. [PMID: 31751241 DOI: 10.1109/tip.2019.2953587] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Despite growing progresses in recent years, cross-scenario person re-identification remains challenging, mainly due to the pedestrians commonly surrounded by highly-complex environment contexts. In reality, the human perception mechanism could adaptively find proper contextualized spatial-temporal clues towards pedestrian recognition. However, conventional methods fall short in adaptively leveraging the long-term spatial-temporal information due to ever-increasing computational cost. Moreover, CNN-based deep learning methods are hard to conduct optimization due to the non-differentiable property of the built-in context search operation. To ameliorate, this paper proposes a novel Context-Interactive CNN (CI-CNN) to dynamically find both spatial and temporal contexts by embedding multi-task Reinforcement Learning (MTRL). The CI-CNN streamlines the multi-task reinforcement learning by using an actor-critic agent to capture the temporal-spatial context simultaneously, which comprises a context-policy network and a context-critic network. The former network learns policies to determine the optimal spatial context region and temporal sequence range. Based on the inferred temporal-spatial cues, the latter one focuses on the identification task and provides feedback for the policy network. Thus, CI-CNN can simultaneously zoom in/out the perception field in spatial and temporal domain for the context interaction with the environment. By fostering the collaborative interaction between the person and context, our method could achieve outstanding performance on various public benchmarks, which confirms the rationality of our hypothesis, and verifies the effectiveness of our CI-CNN framework.
Collapse
|
40
|
Wang Q, Gao C, Zhang W, Luo S, Zhou M, Liu Y, Liu R, Zhang Y, Wang Z, Hao A. Biomorphic carbon derived from corn husk as a promising anode materials for potassium ion battery. Electrochim Acta 2019. [DOI: 10.1016/j.electacta.2019.134902] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
|
41
|
Song W, Li S, Liu J, Qin H, Zhang B, Zhang S, Hao A. Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition. IEEE J Biomed Health Inform 2018; 23:1215-1224. [PMID: 29994412 DOI: 10.1109/jbhi.2018.2852718] [Citation(s) in RCA: 55] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Thyroid ultrasonography is a widely used clinical technique for nodule diagnosis in thyroid regions. However, it remains difficult to detect and recognize the nodules due to low contrast, high noise, and diverse appearance of nodules. In today's clinical practice, senior doctors could pinpoint nodules by analyzing global context features, local geometry structure, and intensity changes, which would require rich clinical experience accumulated from hundreds and thousands of nodule case studies. To alleviate doctors' tremendous labor in the diagnosis procedure, we advocate a machine learning approach to the detection and recognition tasks in this paper. In particular, we develop a multitask cascade convolution neural network (MC-CNN) framework to exploit the context information of thyroid nodules. It may be noted that our framework is built upon a large number of clinically confirmed thyroid ultrasound images with accurate and detailed ground truth labels. Other key advantages of our framework result from a multitask cascade architecture, two stages of carefully designed deep convolution networks in order to detect and recognize thyroid nodules in a pyramidal fashion, and capturing various intrinsic features in a global-to-local way. Within our framework, the potential regions of interest after initial detection are further fed to the spatial pyramid augmented CNNs to embed multiscale discriminative information for fine-grained thyroid recognition. Experimental results on 4309 clinical ultrasound images have indicated that our MC-CNN is accurate and effective for both thyroid nodules detection and recognition. For the correct diagnosis rate of malignant and benign thyroid nodules, its mean Average Precision (mAP) performance can achieve up to [Formula: see text] accuracy, which outperforms the common CNNs by [Formula: see text] on average. In addition, we conduct rigorous user studies to confirm that our MC-CNN outperforms experienced doctors, yet only consuming roughly [Formula: see text] ( 1/48) of doctors' examination time on average. Therefore, the accuracy and efficiency of our new method exhibit its great potential in clinical applications.
Collapse
|
42
|
Gao L, Liu R, Jiang Y, Song W, Wang Y, Liu J, Wang J, Wu D, Li S, Hao A, Zhang B. Computer-aided system for diagnosing thyroid nodules on ultrasound: A comparison with radiologist-based clinical assessments. Head Neck 2017; 40:778-783. [PMID: 29286180 DOI: 10.1002/hed.25049] [Citation(s) in RCA: 39] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2017] [Accepted: 11/16/2017] [Indexed: 11/11/2022] Open
Abstract
BACKGROUND The purpose of this study was to compare the diagnostic efficiency of a thyroid ultrasound computer-aided diagnosis (CAD) system with that of 1 radiologist. METHODS This study retrospectively reviewed 342 surgically resected thyroid nodules from July 2013 to December 2013 at our center. The nodules were assessed on typical ultrasound images using the CAD system and reviewed by 1 experienced radiologist. The radiologist stratified the risk of malignancy using the Thyroid Imaging Reporting and Data Systems (TIRADS) and the American Thyroid Association (ATA) guidelines. RESULTS The radiologist, using TI-RADS and ATA guidelines, performed better than the CAD system (P < .01). The sensitivity of the CAD system was similar to that of an experienced radiologist (P > .05; P < .01; and P > .05). However, we found that the CAD system had lower specificity (P < .01). CONCLUSION The sensitivity of a thyroid ultrasound CAD system in differentiating nodules was similar to that of an experienced radiologist. However, the CAD system had lower specificity.
Collapse
|
43
|
Matsuda S, Hao A, Saito M, Yoshizawa T. Clinical features and outcomes of the paraneoplastic neurological syndromes: Our 7-year experience. J Neurol Sci 2017. [DOI: 10.1016/j.jns.2017.08.2101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
44
|
Zielecka-Dębska D, Hao A, Matkowski R, Kornafel J, Szelachowska J. Influence of the MCM7 Protein Expression on Oral Cancer Patient Prognosis, Using Different Methods of the Measurement. Int J Radiat Oncol Biol Phys 2017. [DOI: 10.1016/j.ijrobp.2017.06.1531] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
45
|
Zielecka-Dębska D, Hao A, Matkowski R, Kornafel J, Szelachowska J. The Prognostic Value of E-cadherin Expression in Oral Cancer Patients. Int J Radiat Oncol Biol Phys 2017. [DOI: 10.1016/j.ijrobp.2017.06.1530] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
46
|
Hao A, Saitoh M, Matsuda S, Yoshizawa T. Extrathymic malignancies in patients with myasthenia gravis. J Neurol Sci 2017. [DOI: 10.1016/j.jns.2017.08.3721] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
47
|
Hou F, He Y, Qin H, Hao A. Knot Optimization for Biharmonic B-splines on Manifold Triangle Meshes. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2017; 23:2082-2095. [PMID: 27608469 DOI: 10.1109/tvcg.2016.2605092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Biharmonic B-splines, proposed by Feng and Warren, are an elegant generalization of univariate B-splines to planar and curved domains with fully irregular knot configuration. Despite the theoretic breakthrough, certain technical difficulties are imperative, including the necessity of Voronoi tessellation, the lack of analytical formulation of bases on general manifolds, expensive basis re-computation during knot refinement/removal, being applicable for simple domains only (e.g., such as euclidean planes, spherical and cylindrical domains, and tori). To ameliorate, this paper articulates a new biharmonic B-spline computing paradigm with a simple formulation. We prove that biharmonic B-splines have an equivalent representation, which is solely based on a linear combination of Green's functions of the bi-Laplacian operator. Consequently, without explicitly computing their bases, biharmonic B-splines can bypass the Voronoi partitioning and the discretization of bi-Laplacian, enable the computational utilities on any compact 2-manifold. The new representation also facilitates optimization-driven knot selection for constructing biharmonic B-splines on manifold triangle meshes. We develop algorithms for spline evaluation, data interpolation and hierarchical data decomposition. Our results demonstrate that biharmonic B-splines, as a new type of spline functions with theoretic and application appeal, afford progressive update of fully irregular knots, free of singularity, without the need of explicit parameterization, making it ideal for a host of graphics tasks on manifolds.
Collapse
|
48
|
Chen C, Li S, Wang Y, Qin H, Hao A. Video Saliency Detection via Spatial-Temporal Fusion and Low-Rank Coherency Diffusion. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2017; 26:3156-3170. [PMID: 28221994 DOI: 10.1109/tip.2017.2670143] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
This paper advocates a novel video saliency detection method based on the spatial-temporal saliency fusion and low-rank coherency guided saliency diffusion. In sharp contrast to the conventional methods, which conduct saliency detection locally in a frame-by-frame way and could easily give rise to incorrect low-level saliency map, in order to overcome the existing difficulties, this paper proposes to fuse the color saliency based on global motion clues in a batch-wise fashion. And we also propose low-rank coherency guided spatial-temporal saliency diffusion to guarantee the temporal smoothness of saliency maps. Meanwhile, a series of saliency boosting strategies are designed to further improve the saliency accuracy. First, the original long-term video sequence is equally segmented into many short-term frame batches, and the motion clues of the individual video batch are integrated and diffused temporally to facilitate the computation of color saliency. Then, based on the obtained saliency clues, inter-batch saliency priors are modeled to guide the low-level saliency fusion. After that, both the raw color information and the fused low-level saliency are regarded as the low-rank coherency clues, which are employed to guide the spatial-temporal saliency diffusion with the help of an additional permutation matrix serving as the alternative rank selection strategy. Thus, it could guarantee the robustness of the saliency map's temporal consistence, and further boost the accuracy of the computed saliency map. Moreover, we conduct extensive experiments on five public available benchmarks, and make comprehensive, quantitative evaluations between our method and 16 state-of-the-art techniques. All the results demonstrate the superiority of our method in accuracy, reliability, robustness, and versatility.
Collapse
|
49
|
Ma J, Li S, Qin H, Hao A. Unsupervised Multi-Class Co-Segmentation via Joint-Cut Over $L_{1}$ -Manifold Hyper-Graph of Discriminative Image Regions. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2017; 26:1216-1230. [PMID: 28114015 DOI: 10.1109/tip.2016.2631883] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
This paper systematically advocates a robust and efficient unsupervised multi-class co-segmentation approach by leveraging underlying subspace manifold propagation to exploit the cross-image coherency. It can combat certain image co-segmentation difficulties due to viewpoint change, partial occlusion, complex background, transient illumination, and cluttering texture patterns. Our key idea is to construct a powerful hyper-graph joint-cut framework, which incorporates mid-level image regions-based intra-image feature representation and L1-manifold graph-based inter-image coherency exploration. For local image region generation, we propose a bi-harmonic distance distribution difference metric to govern the super-pixel clustering in a bottom-up way. It not only affords drastic data reduction but also gives rise to discriminative and structure meaningful feature representation. As for the inter-image coherency, we leverage multi-type features involved L1-graph to detect the underlying local manifold from cross-image regions. As a result, the implicit supervising information could be encoded into the unsupervised hyper-graph joint-cut framework. We conduct extensive experiments and make comprehensive evaluations with other state-of-the-art methods over various benchmarks, including iCoseg, MSRC, and Oxford flower. All the results demonstrate the superiorities of our method in terms of accuracy, robustness, efficiency, and versatility.
Collapse
|
50
|
Liu X, Hou F, Qin H, Hao A. Robust Optimization-Based Coronary Artery Labeling From X-Ray Angiograms. IEEE J Biomed Health Inform 2016; 20:1608-1620. [DOI: 10.1109/jbhi.2015.2485227] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|