L1/2 regularization: a thresholding representation theory and a fast solver

The joint quantile regression modeling of mixed ordinal and continuous responses with its application to an obesity risk data

In clinical medical health research, individual measurements sometimes appear as a mixture of ordinal and continuous responses. There are some statistical correlations between response indicators. Regarding the joint modeling of mixed responses, the effect of a set of explanatory variables on the conditional mean of mixed responses is usually studied based on a mean regression model. However, mean regression results tend to underperform for data with non-normal errors and outliers. Quantile regression (QR) offers not only robust estimates but also the ability to analyze the impact of explanatory variables on various quantiles of the response variable. In this paper, we propose a joint QR modeling approach for mixed ordinal and continuous responses and apply it to the analysis of a set of obesity risk data. Firstly, we construct the joint QR model for mixed ordinal and continuous responses based on multivariate asymmetric Laplace distribution and a latent variable model. Secondly, we perform parameter estimation of the model using a Markov chain Monte Carlo algorithm. Finally, Monte Carlo simulation and a set of obesity risk data analysis are used to verify the validity of the proposed model and method.

ACTION: Augmentation and computation toolbox for brain network analysis with functional MRI

Functional magnetic resonance imaging (fMRI) has been increasingly employed to investigate functional brain activity. Many fMRI-related software/toolboxes have been developed, providing specialized algorithms for fMRI analysis. However, existing toolboxes seldom consider fMRI data augmentation, which is quite useful, especially in studies with limited or imbalanced data. Moreover, current studies usually focus on analyzing fMRI using conventional machine learning models that rely on human-engineered fMRI features, without investigating deep learning models that can automatically learn data-driven fMRI representations. In this work, we develop an open-source toolbox, called Augmentation and Computation Toolbox for braIn netwOrk aNalysis (ACTION), offering comprehensive functions to streamline fMRI analysis. The ACTION is a Python-based and cross-platform toolbox with graphical user-friendly interfaces. It enables automatic fMRI augmentation, covering blood-oxygen-level-dependent (BOLD) signal augmentation and brain network augmentation. Many popular methods for brain network construction and network feature extraction are included. In particular, it supports constructing deep learning models, which leverage large-scale auxiliary unlabeled data (3,800+ resting-state fMRI scans) for model pretraining to enhance model performance for downstream tasks. To facilitate multi-site fMRI studies, it is also equipped with several popular federated learning strategies. Furthermore, it enables users to design and test custom algorithms through scripting, greatly improving its utility and extensibility. We demonstrate the effectiveness and user-friendliness of ACTION on real fMRI data and present the experimental results. The software, along with its source code and manual, can be accessed online.

Convergence Analysis of Online Gradient Method for High-Order Neural Networks and Their Sparse Optimization

In this article, we investigate the boundedness and convergence of the online gradient method with the smoothing group regularization for the sigma-pi-sigma neural network (SPSNN). This enhances the sparseness of the network and improves its generalization ability. For the original group regularization, the error function is nonconvex and nonsmooth, which can cause oscillation of the error function. To ameliorate this drawback, we propose a simple and effective smoothing technique, which can effectively eliminate the deficiency of the original group regularization. The group regularization effectively optimizes the network structure from two aspects redundant hidden nodes tending to zero and redundant weights of surviving hidden nodes in the network tending to zero. This article shows the strong and weak convergence results for the proposed method and proves the boundedness of weights. Experiment results clearly demonstrate the capability of the proposed method and the effectiveness of redundancy control. The simulation results are observed to support the theoretical results.

Self-Supervised Medical Image Denoising Based on WISTA-Net for Human Healthcare in Metaverse

Medical image processing plays an important role in the interaction of real world and metaverse for healthcare. Self-supervised denoising based on sparse coding methods, without any prerequisite on large-scale training samples, has been attracting extensive attention for medical image processing. Whereas, existing self-supervised methods suffer from poor performance and low efficiency. In this paper, to achieve state-of-the-art denoising performance on the one hand, we present a self-supervised sparse coding method, named the weighted iterative shrinkage thresholding algorithm (WISTA). It does not rely on noisy-clean ground-truth image pairs to learn from only a single noisy image. On the other hand, to further improve denoising efficiency, we unfold the WISTA to construct a deep neural network (DNN) structured WISTA, named WISTA-Net. Specifically, in WISTA, motivated by the merit of the lp-norm, WISTA-Net has better denoising performance than the classical orthogonal matching pursuit (OMP) algorithm and the ISTA. Moreover, leveraging the high-efficiency of DNN structure in parameter updating, WISTA-Net outperforms the compared methods in denoising efficiency. In detail, for a 256 by 256 noisy image, the running time of WISTA-Net is 4.72 s on the CPU, which is much faster than WISTA, OMP, and ISTA by 32.88 s, 13.06 s, and 6.17 s, respectively.

CardioDPi: An explainable deep-learning model for identifying cardiotoxic chemicals targeting hERG, Cav1.2, and Nav1.5 channels

The cardiotoxic effects of various pollutants have been a growing concern in environmental and material science. These effects encompass arrhythmias, myocardial injury, cardiac insufficiency, and pericardial inflammation. Compounds such as organic solvents and air pollutants disrupt the potassium, sodium, and calcium ion channels cardiac cell membranes, leading to the dysregulation of cardiac function. However, current cardiotoxicity models have disadvantages of incomplete data, ion channels, interpretability issues, and inability of toxic structure visualization. Herein, an interpretable deep-learning model known as CardioDPi was developed, which is capable of discriminating cardiotoxicity induced by the human Ether-à-go-go-related gene (hERG) channel, sodium channel (Na_v1.5), and calcium channel (Ca_v1.5) blockade. External validation yielded promising area under the ROC curve (AUC) values of 0.89, 0.89, and 0.94 for the hERG, Na_v1.5, and Ca_v1.5 channels, respectively. The CardioDPi can be freely accessed on the web server CardioDPipredictor (http://cardiodpi.sapredictor.cn/). Furthermore, the structural characteristics of cardiotoxic compounds were analyzed and structural alerts (SAs) can be extracted using the user-friendly CardioDPi-SAdetector web service (http://cardiosa.sapredictor.cn/). CardioDPi is a valuable tool for identifying cardiotoxic chemicals that are environmental and health risks. Moreover, the SA system provides essential insights for mode-of-action studies concerning cardiotoxic compounds.

Accelerated Partially Shared Dictionary Learning With Differentiable Scale-Invariant Sparsity for Multi-View Clustering

Multiview dictionary learning (DL) is attracting attention in multiview clustering due to the efficient feature learning ability. However, most existing multiview DL algorithms are facing problems in fully utilizing consistent and complementary information simultaneously in the multiview data and learning the most precise representation for multiview clustering because of gaps between views. This article proposes an efficient multiview DL algorithm for multiview clustering, which uses the partially shared DL model with a flexible ratio of shared sparse coefficients to excavate both consistency and complementarity in the multiview data. In particular, a differentiable scale-invariant function is used as the sparsity regularizer, which considers the absolute sparsity of coefficients as the l0 norm regularizer but is continuous and differentiable almost everywhere. The corresponding optimization problem is solved by the proximal splitting method with extrapolation technology; moreover, the proximal operator of the differentiable scale-invariant regularizer can be derived. The synthetic experiment results demonstrate that the proposed algorithm can recover the synthetic dictionary well with reasonable convergence time costs. Multiview clustering experiments include six real-world multiview datasets, and the performances show that the proposed algorithm is not sensitive to the regularizer parameter as the other algorithms. Furthermore, an appropriate coefficient sharing ratio can help to exploit consistent information while keeping complementary information from multiview data and thus enhance performances in multiview clustering. In addition, the convergence performances show that the proposed algorithm can obtain the best performances in multiview clustering among compared algorithms and can converge faster than compared multiview algorithms mostly.

Model Compression for Communication Efficient Federated Learning

Despite the many advantages of using deep neural networks over shallow networks in various machine learning tasks, their effectiveness is compromised in a federated learning setting due to large storage sizes and high computational resource requirements for training. A large model size can potentially require infeasible amounts of data to be transmitted between the server and clients for training. To address these issues, we investigate the traditional and novel compression techniques to construct sparse models from dense networks whose storage and bandwidth requirements are significantly lower. We do this by separately considering compression techniques for the server model to address downstream communication and the client models to address upstream communication. Both of these play a crucial role in developing and maintaining sparsity across communication cycles. We empirically demonstrate the efficacy of the proposed schemes by testing their performance on standard datasets and verify that they outperform various state-of-the-art baseline schemes in terms of accuracy and communication volume.

Beyond ℓ1 sparse coding in V1

Growing evidence indicates that only a sparse subset from a pool of sensory neurons is active for the encoding of visual stimuli at any instant in time. Traditionally, to replicate such biological sparsity, generative models have been using the ℓ1 norm as a penalty due to its convexity, which makes it amenable to fast and simple algorithmic solvers. In this work, we use biological vision as a test-bed and show that the soft thresholding operation associated to the use of the ℓ1 norm is highly suboptimal compared to other functions suited to approximating ℓp with 0 ≤ p < 1 (including recently proposed continuous exact relaxations), in terms of performance. We show that ℓ1 sparsity employs a pool with more neurons, i.e. has a higher degree of overcompleteness, in order to maintain the same reconstruction error as the other methods considered. More specifically, at the same sparsity level, the thresholding algorithm using the ℓ1 norm as a penalty requires a dictionary of ten times more units compared to the proposed approach, where a non-convex continuous relaxation of the ℓ0 pseudo-norm is used, to reconstruct the external stimulus equally well. At a fixed sparsity level, both ℓ0- and ℓ1-based regularization develop units with receptive field (RF) shapes similar to biological neurons in V1 (and a subset of neurons in V2), but ℓ0-based regularization shows approximately five times better reconstruction of the stimulus. Our results in conjunction with recent metabolic findings indicate that for V1 to operate efficiently it should follow a coding regime which uses a regularization that is closer to the ℓ0 pseudo-norm rather than the ℓ1 one, and suggests a similar mode of operation for the sensory cortex in general.

An Incremental Broad-Learning-System-Based Approach for Tremor Attenuation for Robot Tele-Operation

The existence of the physiological tremor of the human hand significantly affects the application of tele-operation systems in performing high-precision tasks, such as tele-surgery, and currently, the process of effectively eliminating the physiological tremor has been an important yet challenging research topic in the tele-operation robot field. Some scholars propose using deep learning algorithms to solve this problem, but a large number of hyperparameters lead to a slow training speed. Later, the support-vector-machine-based methods have been applied to solve the problem, thereby effectively canceling tremors. However, these methods may lose the prediction accuracy, because learning energy cannot be accurately assigned. Therefore, in this paper, we propose a broad-learning-system-based tremor filter, which integrates a series of incremental learning algorithms to achieve fast remodeling and reach the desired performance. Note that the broad-learning-system-based filter has a fast learning rate while ensuring the accuracy due to its simple and novel network structure. Unlike other algorithms, it uses incremental learning algorithms to constantly update network parameters during training, and it stops learning when the error converges to zero. By focusing on the control performance of the slave robot, a sliding mode control approach has been used to improve the performance of closed-loop systems. In simulation experiments, the results demonstrated the feasibility of our proposed method.

A unified analysis of convex and non‑convex 𝓛p‑ball projection problems

The task of projecting onto ℓ p norm balls is ubiquitous in statistics and machine learning, yet the availability of actionable algorithms for doing so is largely limited to the special cases of p ∈ { 0 ,   1 ,   2 ,   ∞ } . In this paper, we introduce novel, scalable methods for projecting onto the ℓ p -ball for general p > 0 . For p ≥ 1 , we solve the univariate Lagrangian dual via a dual Newton method. We then carefully design a bisection approach For p < 1 , presenting theoretical and empirical evidence of zero or a small duality gap in the non-convex case. The success of our contributions is thoroughly assessed empirically, and applied to large-scale regularized multi-task learning and compressed sensing. The code implementing our methods is publicly available on Github.

Robust transformed l1 metric for fluorescence molecular tomography

BACKGROUND AND OBJECTIVE

Fluorescence molecular tomography (FMT) is a non-invasive molecular imaging modality that can be used to observe the three-dimensional distribution of fluorescent probes in vivo. FMT is a promising imaging technique in clinical and preclinical research that has attracted significant attention. Numerous regularization based reconstruction algorithms have been proposed. However, traditional algorithms that use the squared l₂-norm distance usually exaggerate the influence of noise and measurement and calculation errors, and their robustness cannot be guaranteed.

METHODS

In this study, we propose a novel robust transformed l₁ (TL1) metric that interpolates l₀ and l₁ norms through a nonnegative parameter α∈(0,+∞). The TL1 metric looks like the l_p-norm with p∈(0,1). These are markedly different because TL1 metric has two properties, boundedness and Lipschitz-continuity, which make the TL1 criterion suitable distance metric, particularly for robustness, owing to its stronger noise suppression. Subsequently, we apply the proposed metric to FMT and build a robust model to reduce the influence of noise. The nonconvexity of the proposed model made direct optimization difficult, and a continuous optimization method was developed to solve the model. The problem was converted into a difference in convex programming problem for the TL1 metric (DCATL1), and the corresponding algorithm converged linearly.

RESULTS

Various numerical simulations and in vivo bead-implanted mouse experiments were conducted to verify the performance of the proposed method. The experimental results show that the DCATL1 algorithm is more robust than the state-of-the-art approaches and achieves better source localization and morphology recovery.

CONCLUSIONS

The in vivo experiments showed that DCATL1 can be used to visualize the distribution of fluorescent probes inside biological tissues and promote preclinical application in small animals, demonstrating the feasibility and effectiveness of the proposed method.

Non-negative matrix factorization for overlapping community detection in directed weighted networks with sparse constraints

Detecting overlapping communities is essential for analyzing the structure and function of complex networks. However, most existing approaches only consider network topology and overlook the benefits of attribute information. In this paper, we propose a novel attribute-information non-negative matrix factorization approach that integrates sparse constraints and optimizes an objective function for detecting communities in directed weighted networks. Our algorithm updates the basic non-negative matrix adaptively, incorporating both network topology and attribute information. We also add a sparsity constraint term of graph regularization to maintain the intrinsic geometric structure between nodes. Importantly, we provide strict proof of convergence for the multiplication update rule used in our algorithm. We apply our proposed algorithm to various artificial and real-world networks and show that it is more effective for detecting overlapping communities. Furthermore, our study uncovers the intricate iterative process of system evolution toward convergence and investigates the impact of various variables on network detection. These findings provide insights into building more robust and operable complex systems.

A calibrated SVM based on weighted smooth GL1/2 for Alzheimer’s disease prediction

Alzheimer's disease (AD) is currently one of the mainstream senile diseases in the world. It is a key problem predicting the early stage of AD. Low accuracy recognition of AD and high redundancy brain lesions are the main obstacles. Traditionally, Group Lasso method can achieve good sparseness. But, redundancy inside group is ignored. This paper proposes an improved smooth classification framework which combines the weighted smooth GL_1/2 (wSGL_1/2) as feature selection method and a calibrated support vector machine (cSVM) as the classifier. wSGL_1/2 can make intra-group and inner-group features sparse, in which the group weights can further improve the efficiency of the model. cSVM can enhance the speed and stability of model by adding calibrated hinge function. Before feature selecting, an anatomical boundary-based clustering, called as ac-SLIC-AAL, is designed to make adjacent similar voxels into one group for accommodating the overall differences of all data. The cSVM model is fast convergence speed, high accuracy and good interpretability on AD classification, AD early diagnosis and MCI transition prediction. In experiments, all steps are tested respectively, including classifiers' comparison, feature selection verification, generalization verification and comparing with state-of-the-art methods. The results are supportive and satisfactory. The superior of the proposed model are verified globally. At the same time, the algorithm can point out the important brain areas in the MRI, which has important reference value for the doctor's predictive work. The source code and data is available at http://github.com/Hu-s-h/c-SVMForMRI.

A Simple but Universal Fully Linearized ADMM Algorithm for Optimization Based Image Reconstruction

Background and Objective

Optimization based image reconstruction algorithm is an advanced algorithm in medical imaging. However, the corresponding solving algorithm is challenging because the optimization model is usually large-scale and non-smooth. This work aims to devise a simple but universal solver for optimization models.

Methods

The alternating direction method of multipliers (ADMM) algorithm is a simple and effective solver of the optimization models. However, there always exists a sub-problem that has not closed-form solution. One may use gradient descent algorithm to solve this sub-problem, but the step-size selection via line search is time-consuming. Or, one may use fast Fourier transform (FFT) to get a closed-form solution if the system matrix and the sparse transform matrix are both of special structure. In this work, we propose a simple but universal fully linearized ADMM (FL-ADMM) algorithm that avoids line search to determine step-size and applies to system matrix and sparse transform of any structures.

Results

We derive the FL-ADMM algorithm instances for three total variation (TV) models in 2D computed tomography (CT). Further, we validate and evaluate one FL-ADMM algorithm and explore how the two important factors impact convergence rate. Also, we compare this algorithm with the Chambolle-Pock algorithm via real CT phantom reconstructions. These studies show that the FL-ADMM algorithm may accurately solve optimization models in image reconstruction.

Conclusion

The FL-ADMM algorithm is a simple, effective, convergent and universal solver of optimization models in image reconstruction. Compared to the existing ADMM algorithms, the new algorithm does not need time-consuming step-size line-search or special demand to system matrix and sparse transform. It is a rapid prototyping tool for optimization based image reconstruction.

Sparse non-convex regularization based explainable DBN in the analysis of brain abnormalities in schizophrenia

Deep belief networks have been widely used in medical image analysis. However, the high-dimensional but small-sample-size characteristic of medical image data makes the model prone to dimensional disaster and overfitting. Meanwhile, the traditional DBN is driven by performance and ignores the explainability which is important for medical image analysis. In this paper, a sparse non-convex based explainable deep belief network is proposed by combining DBN with non-convex sparsity learning. For sparsity, the non-convex regularization and Kullback-Leibler divergence penalty are embedded into DBN to obtain the sparse connection and sparse response representation of the network. It effectively reduces the complexity of the model and improves the generalization ability of the model. Considering explainability, the crucial features for decision-making are selected through the feature back-selection based on the row norm of each layer's weight after network training. We apply the model to schizophrenia data and demonstrate it achieves the best performance among several typical feature selection models. It reveals 28 functional connections highly correlated with schizophrenia, which provides an effective foundation for the treatment and prevention of schizophrenia and methodological assurance for similar brain disorders.

Unsupervised Learning for Salient Object Detection via Minimization of Bilinear Factor Matrix Norm

Saliency detection is an important but challenging task in the study of computer vision. In this article, we develop a new unsupervised learning approach for the saliency detection by an intrinsic regularization model, in which the Schatten-2/3 norm is integrated with the nonconvex sparse l_2/3 norm. The l_2/3 -norm is shown to be capable of detecting consistent values among sparse foreground by using image geometrical structure and feature similarity, while the Schatten-2/3 norm can capture the lower rank of background by matrix factorization. To improve effective performance of separation for Schatten-2/3-norm and l_2/3 -norm, a Laplacian regularization is adopted to the foreground for the smoothness. The proposed model essentially converts the required nonconvex optimization problem into the convex one, conducted by splitting the objective function based on singular value decomposition on one much smaller factor matrix and then optimized by using the alternating direction method of the multiplier. The convergence of the proposed algorithm is discussed in detail. Extensive experiments on three benchmark datasets demonstrate that our unsupervised learning approach is very competitive and appears to be more consistent across various salient objects than the current existing approaches.

Joint Sparse Collaborative Regression on Imaging Genetics Study of Schizophrenia

The imaging genetics approach generates large amount of high dimensional and multi-modal data, providing complementary information for comprehensive study of Schizophrenia, a complex mental disease. However, at the same time, the variety of these data in structures, resolutions, and formats makes their integrative study a forbidding task. In this paper, we propose a novel model called Joint Sparse Collaborative Regression (JSCoReg), which can extract class-specific features from different health conditions/disease classes. We first evaluate the performance of feature selection in terms of Receiver operating characteristic curve and the area under the ROC curve in the simulation experiment. We demonstrate that the JSCoReg model can achieve higher accuracy compared with similar models including Joint Sparse Canonical Correlation Analysis and Sparse Collaborative Regression. We then applied the JSCoReg model to the analysis of schizophrenia dataset collected from the Mind Clinical Imaging Consortium. The JSCoReg enables us to better identify biomarkers associated with schizophrenia, which are verified to be both biologically and statistically significant.

Matrix Completion via Non-Convex Relaxation and Adaptive Correlation Learning

The existing matrix completion methods focus on optimizing the relaxation of rank function such as nuclear norm, Schatten- p norm, etc. They usually need many iterations to converge. Moreover, only the low-rank property of matrices is utilized in most existing models and several methods that incorporate other knowledge are quite time-consuming in practice. To address these issues, we propose a novel non-convex surrogate that can be optimized by closed-form solutions, such that it empirically converges within dozens of iterations. Besides, the optimization is parameter-free and the convergence is proved. Compared with the relaxation of rank, the surrogate is motivated by optimizing an upper-bound of rank. We theoretically validate that it is equivalent to the existing matrix completion models. Besides the low-rank assumption, we intend to exploit the column-wise correlation for matrix completion, and thus an adaptive correlation learning, which is scaling-invariant, is developed. More importantly, after incorporating the correlation learning, the model can be still solved by closed-form solutions such that it still converges fast. Experiments show the effectiveness of the non-convex surrogate and adaptive correlation learning.

Nonconvex Structural Sparsity Residual Constraint for Image Restoration

This article proposes a novel nonconvex structural sparsity residual constraint (NSSRC) model for image restoration, which integrates structural sparse representation (SSR) with nonconvex sparsity residual constraint (NC-SRC). Although SSR itself is powerful for image restoration by combining the local sparsity and nonlocal self-similarity in natural images, in this work, we explicitly incorporate the novel NC-SRC prior into SSR. Our proposed approach provides more effective sparse modeling for natural images by applying a more flexible sparse representation scheme, leading to high-quality restored images. Moreover, an alternating minimizing framework is developed to solve the proposed NSSRC-based image restoration problems. Extensive experimental results on image denoising and image deblocking validate that the proposed NSSRC achieves better results than many popular or state-of-the-art methods over several publicly available datasets.

The Global Organizational Behavior Analysis for Financial Risk Management Utilizing Artificial Intelligence

Enterprise financial risks are analyzed utilizing the theory of organizational behavior, and a financial risk management system is constructed to improve the design and algorithm of the enterprise risk management system. Base on the CCER (China Center for Economic Research) database, the early warning model for enterprise financial risk management containing five indices is proposed for enterprises. Through Logistic regression analysis, the design principle of the financial risk management system based on AI (Artificial Intelligence) technology is explained. The proposed system innovatively introduces the AI integrated learning method, optimizes objective function through XGBoost (eXtreme Gradient Boosting) algorithm, and trains the model through BP (Backpropagation) NN (Neural Network). Finally, following comparative analysis, the effectiveness of the proposed method is verified.

Large-Scale Affine Matrix Rank Minimization With a Novel Nonconvex Regularizer

Low-rank minimization aims to recover a matrix of minimum rank subject to linear system constraint. It can be found in various data analysis and machine learning areas, such as recommender systems, video denoising, and signal processing. Nuclear norm minimization is a dominating approach to handle it. However, such a method ignores the difference among singular values of target matrix. To address this issue, nonconvex low-rank regularizers have been widely used. Unfortunately, existing methods suffer from different drawbacks, such as inefficiency and inaccuracy. To alleviate such problems, this article proposes a flexible model with a novel nonconvex regularizer. Such a model not only promotes low rankness but also can be solved much faster and more accurate. With it, the original low-rank problem can be equivalently transformed into the resulting optimization problem under the rank restricted isometry property (rank-RIP) condition. Subsequently, Nesterov's rule and inexact proximal strategies are adopted to achieve a novel algorithm highly efficient in solving this problem at a convergence rate of O(1/K) , with K being the iterate count. Besides, the asymptotic convergence rate is also analyzed rigorously by adopting the Kurdyka- ojasiewicz (KL) inequality. Furthermore, we apply the proposed optimization model to typical low-rank problems, including matrix completion, robust principal component analysis (RPCA), and tensor completion. Exhaustively empirical studies regarding data analysis tasks, i.e., synthetic data analysis, image recovery, personalized recommendation, and background subtraction, indicate that the proposed model outperforms state-of-the-art models in both accuracy and efficiency.

Underwater Image Enhancement With Hyper-Laplacian Reflectance Priors

Underwater image enhancement aims at improving the visibility and eliminating color distortions of underwater images degraded by light absorption and scattering in water. Recently, retinex variational models show remarkable capacity of enhancing images by estimating reflectance and illumination in a retinex decomposition course. However, ambiguous details and unnatural color still challenge the performance of retinex variational models on underwater image enhancement. To overcome these limitations, we propose a hyper-laplacian reflectance priors inspired retinex variational model to enhance underwater images. Specifically, the hyper-laplacian reflectance priors are established with the l_1/2 -norm penalty on first-order and second-order gradients of the reflectance. Such priors exploit sparsity-promoting and complete-comprehensive reflectance that is used to enhance both salient structures and fine-scale details and recover the naturalness of authentic colors. Besides, the l₂ norm is found to be suitable for accurately estimating the illumination. As a result, we turn a complex underwater image enhancement issue into simple subproblems that separately and simultaneously estimate the reflection and the illumination that are harnessed to enhance underwater images in a retinex variational model. We mathematically analyze and solve the optimal solution of each subproblem. In the optimization course, we develop an alternating minimization algorithm that is efficient on element-wise operations and independent of additional prior knowledge of underwater conditions. Extensive experiments demonstrate the superiority of the proposed method in both subjective results and objective assessments over existing methods. The code is available at: https://github.com/zhuangpeixian/HLRP.

Sparse signal reconstruction via collaborative neurodynamic optimization

In this paper, we formulate a mixed-integer problem for sparse signal reconstruction and reformulate it as a global optimization problem with a surrogate objective function subject to underdetermined linear equations. We propose a sparse signal reconstruction method based on collaborative neurodynamic optimization with multiple recurrent neural networks for scattered searches and a particle swarm optimization rule for repeated repositioning. We elaborate on experimental results to demonstrate the outperformance of the proposed approach against ten state-of-the-art algorithms for sparse signal reconstruction.

Fixed-Time Stable Neurodynamic Flow to Sparse Signal Recovery via Nonconvex L1-β2-Norm

This letter develops a novel fixed-time stable neurodynamic flow (FTSNF) implemented in a dynamical system for solving the nonconvex, nonsmooth model L1-β2, β∈[0,1] to recover a sparse signal. FTSNF is composed of many neuron-like elements running in parallel. It is very efficient and has provable fixed-time convergence. First, a closed-form solution of the proximal operator to model L1-β2, β∈[0,1] is presented based on the classic soft thresholding of the L1-norm. Next, the proposed FTSNF is proven to have a fixed-time convergence property without additional assumptions on the convexity and strong monotonicity of the objective functions. In addition, we show that FTSNF can be transformed into other proximal neurodynamic flows that have exponential and finite-time convergence properties. The simulation results of sparse signal recovery verify the effectiveness and superiority of the proposed FTSNF.

Development of a high-throughput SNP array for sea cucumber (Apostichopus japonicus) and its application in genomic selection with MCP regularized deep neural networks

High-throughput single nucleotide polymorphism (SNP) genotyping assays are powerful tools for genetic studies and genomic breeding applications for many species. Though large numbers of SNPs have been identified in sea cucumber (Apostichopus japonicus), but, as yet, no high-throughput genotyping platform is available for this species. In this study, we designed and developed a high-throughput 24 K SNP genotyping array named HaishenSNP24K for A. japonicus, based on the multi-objective-local optimization (MOLO) algorithm and HD-Marker genotyping method. The SNP array exhibited a relatively high genotyping call rate (> 96%), genotyping accuracy (>95%) and exhibited highly polymorphic in sea cucumber populations. In addition, we also assessed its application in genomic selection (GS). Deep neural networks (DNN) that can capture the complicated interactions of genes have been proposed as a promising tool in GS for SNP-based genomic prediction of complex traits in animal breeding. To overcome the problem of over-fitting when using the HaishenSNP24K array as high-dimensional DNN input, we developed minmax concave penalty (MCP) regularization for sparse deep neural networks (DNN-MCP) that finds an optimal sparse structure of a DNN by minimizing the square error subject to the non-convex penalty MCP on the parameters (weights and biases). Compared to two linear models, namely RR-GBLUP and Bayes B, and the nonlinear model DNN, DNN-MCP has greatly improved the genomic prediction ability for three quantitative traits (e.g., wet weight, dry weight and survival time) in the sea cucumber population. To the best of our knowledge, this is the first work to develop a high-throughput SNP array for A. japonicus and a new model DNN-MCP for genomic prediction of complex traits in GS. The present results provide evidence that supports the HaishenSNP24K array with DNN-MCP will be valuable for genetic studies and molecular breeding in A. japonicus.

Group Logistic Regression Models with Lp,q Regularization

In this paper, we proposed a logistic regression model with lp,q regularization that could give a group sparse solution. The model could be applied to variable-selection problems with sparse group structures. In the context of big data, the solutions for practical problems are often group sparse, so it is necessary to study this kind of model. We defined the model from three perspectives: theoretical, algorithmic and numeric. From the theoretical perspective, by introducing the notion of the group restricted eigenvalue condition, we gave the oracle inequality, which was an important property for the variable-selection problems. The global recovery bound was also established for the logistic regression model with lp,q regularization. From the algorithmic perspective, we applied the well-known alternating direction method of multipliers (ADMM) algorithm to solve the model. The subproblems for the ADMM algorithm were solved effectively. From the numerical perspective, we performed experiments for simulated data and real data in the factor stock selection. We employed the ADMM algorithm that we presented in the paper to solve the model. The numerical results were also presented. We found that the model was effective in terms of variable selection and prediction.

Online Reconstruction of Complex Networks From Streaming Data

The problem of reconstructing nonlinear and complex dynamical systems from available data or time series is prominent in many fields, including engineering, physical, computer, biological, and social sciences. Many methods have been proposed to address this problem and their performance is satisfactory. However, none of them can reconstruct network structure from large-scale real-time streaming data, which leads to the failure of real-time and online analysis or control of complex systems. In this article, to overcome the limitations of current methods, we first extend the network reconstruction problem (NRP) to online settings, and then develop a follow-the-regularized-leader (FTRL)-Proximal style method to address the online complex NRP; we refer to it as Online-NR. The performance of Online-NR is validated on synthetic evolutionary game network reconstruction datasets and eight real-world networks. The experimental results demonstrate that Online-NR can effectively solve the problem of online network reconstruction with large-scale real-time streaming data. Moreover, Online-NR outperforms or matches nine state-of-the-art network reconstruction methods.

Whispered Speech Conversion Based on the Inversion of Mel Frequency Cepstral Coefficient Features

A conversion method based on the inversion of Mel frequency cepstral coefficient (MFCC) features was proposed to convert whispered speech into normal speech. First, the MFCC features of whispered speech and normal speech were extracted and a matching relation between the MFCC feature parameters of whispered speech and normal speech was developed through the Gaussian mixture model (GMM). Then, the MFCC feature parameters of normal speech corresponding to whispered speech were obtained based on the GMM and, finally, whispered speech was converted into normal speech through the inversion of MFCC features. The experimental results showed that the cepstral distortion (CD) of the normal speech converted by the proposed method was 21% less than that of the normal speech converted by the linear predictive coefficient (LPC) features, the mean opinion score (MOS) was 3.56, and a satisfactory outcome in both intelligibility and sound quality was achieved.

Weighted Group Sparsity-Constrained Tensor Factorization for Hyperspectral Unmixing

Recently, unmixing methods based on nonnegative tensor factorization have played an important role in the decomposition of hyperspectral mixed pixels. According to the spatial prior knowledge, there are many regularizations designed to improve the performance of unmixing algorithms, such as the total variation (TV) regularization. However, these methods mostly ignore the similar characteristics among different spectral bands. To solve this problem, this paper proposes a group sparse regularization that uses the weighted constraint of the L2,1 norm, which can not only explore the similar characteristics of the hyperspectral image in the spectral dimension, but also keep the data smooth characteristics in the spatial dimension. In summary, a non-negative tensor factorization framework based on weighted group sparsity constraint is proposed for hyperspectral images. In addition, an effective alternating direction method of multipliers (ADMM) algorithm is used to solve the algorithm proposed in this paper. Compared with the existing popular methods, experiments conducted on three real datasets fully demonstrate the effectiveness and advancement of the proposed method.

Smooth Group L1/2 Regularization for Pruning Convolutional Neural Networks

In this paper, a novel smooth group L1/2 (SGL1/2) regularization method is proposed for pruning hidden nodes of the fully connected layer in convolution neural networks. Usually, the selection of nodes and weights is based on experience, and the convolution filter is symmetric in the convolution neural network. The main contribution of SGL1/2 is to try to approximate the weights to 0 at the group level. Therefore, we will be able to prune the hidden node if the corresponding weights are all close to 0. Furthermore, the feasibility analysis of this new method is carried out under some reasonable assumptions due to the smooth function. The numerical results demonstrate the superiority of the SGL1/2 method with respect to sparsity, without damaging the classification performance.

Three-Dimensional Sparse SAR Imaging with Generalized Lq Regularization

Three-dimensional (3D) synthetic aperture radar (SAR) imaging provides complete 3D spatial information, which has been used in environmental monitoring in recent years. Compared with matched filtering (MF) algorithms, the regularization technique can improve image quality. However, due to the substantial computational cost, the existing observation-matrix-based sparse imaging algorithm is difficult to apply to large-scene and 3D reconstructions. Therefore, in this paper, novel 3D sparse reconstruction algorithms with generalized Lq-regularization are proposed. First, we combine majorization–minimization (MM) and L1 regularization (MM-L1) to improve SAR image quality. Next, we combine MM and L1/2 regularization (MM-L1/2) to achieve high-quality 3D images. Then, we present the algorithm which combines MM and L0 regularization (MM-L0) to obtain 3D images. Finally, we present a generalized MM-Lq algorithm (GMM-Lq) for sparse SAR imaging problems with arbitrary q0≤q≤1 values. The proposed algorithm can improve the performance of 3D SAR images, compared with existing regularization techniques, and effectively reduce the amount of calculation needed. Additionally, the reconstructed complex image retains the phase information, which makes the reconstructed SAR image still suitable for interferometry applications. Simulation and experimental results verify the effectiveness of the algorithms.

A Robust Tensor-Based Submodule Clustering for Imaging Data Using l12 Regularization and Simultaneous Noise Recovery via Sparse and Low Rank Decomposition Approach

The massive generation of data, which includes images and videos, has made data management, analysis, information extraction difficult in recent years. To gather relevant information, this large amount of data needs to be grouped. Real-life data may be noise corrupted during data collection or transmission, and the majority of them are unlabeled, allowing for the use of robust unsupervised clustering techniques. Traditional clustering techniques, which vectorize the images, are unable to keep the geometrical structure of the images. Hence, a robust tensor-based submodule clustering method based on l12 regularization with improved clustering capability is formulated. The l12 induced tensor nuclear norm (TNN), integrated into the proposed method, offers better low rankness while retaining the self-expressiveness property of submodules. Unlike existing methods, the proposed method employs a simultaneous noise removal technique by twisting the lateral image slices of the input data tensor into frontal slices and eliminates the noise content in each image, using the principles of the sparse and low rank decomposition technique. Experiments are carried out over three datasets with varying amounts of sparse, Gaussian and salt and pepper noise. The experimental results demonstrate the superior performance of the proposed method over the existing state-of-the-art methods.

Tangent vector-based gradient method with l12-regularization: Iterative half thresholding algorithm for CS-MRI

OBJECT

This paper presents a new method using tangent vector-based l₁₂-regularization for compressed sensing MR image reconstruction.

MATERIALS AND METHODS

The proposed method with l₁₂-regularization is tested on four datasets: (i) 1-D sparse signal (ii) numerical cardiac phantom, (iii & iv) two sets of in-vivo cardiac MRI datasets acquired using 30 receiver coil elements with Cartesian and radial trajectories on 3T scanner. The results are compared with standard CS reconstruction, which utilizes l₁-regularization. The experiments were also conducted for two different types of samplings: (i) cartesian sub-sampling and (ii) 2D random Gaussian sub-sampling.

RESULTS

The quality of the reconstructed images is validated through Root Mean Square Error (RMSE) and Peak Signal-to-Noise Ratio (PSNR). The results show that the proposed method outperforms the standard CS reconstructions in our experiments with an improvement of 54.8% in RMSE and 14.3% in terms of PSNR. Moreover, the Gaussian random sub-sampling-based image reconstruction results are better than the Cartesian sub-sampling-based reconstruction results.

CONCLUSION

The results show that the proposed method yields a good sparse signal approximation and superior convergence behavior, which implies a promising technique for the reconstruction of cardiac MR images as compared to the conventional CS algorithm.

Simultaneous Patch-Group Sparse Coding with Dual-Weighted ℓp Minimization for Image Restoration

Sparse coding (SC) models have been proven as powerful tools applied in image restoration tasks, such as patch sparse coding (PSC) and group sparse coding (GSC). However, these two kinds of SC models have their respective drawbacks. PSC tends to generate visually annoying blocking artifacts, while GSC models usually produce over-smooth effects. Moreover, conventional ℓ1 minimization-based convex regularization was usually employed as a standard scheme for estimating sparse signals, but it cannot achieve an accurate sparse solution under many realistic situations. In this paper, we propose a novel approach for image restoration via simultaneous patch-group sparse coding (SPG-SC) with dual-weighted ℓp minimization. Specifically, in contrast to existing SC-based methods, the proposed SPG-SC conducts the local sparsity and nonlocal sparse representation simultaneously. A dual-weighted ℓp minimization-based non-convex regularization is proposed to improve the sparse representation capability of the proposed SPG-SC. To make the optimization tractable, a non-convex generalized iteration shrinkage algorithm based on the alternating direction method of multipliers (ADMM) framework is developed to solve the proposed SPG-SC model. Extensive experimental results on two image restoration tasks, including image inpainting and image deblurring, demonstrate that the proposed SPG-SC outperforms many state-of-the-art algorithms in terms of both objective and perceptual quality.

Correntropy-induced metric with Laplacian kernel for robust fluorescence molecular tomography

Fluorescence molecular tomography (FMT), which is used to visualize the three-dimensional distribution of fluorescence probe in small animals via the reconstruction method, has become a promising imaging technique in preclinical research. However, the classical reconstruction criterion is formulated based on the squared l ₂-norm distance metric, leaving it prone to being influenced by the presence of outliers. In this study, we propose a robust distance based on the correntropy-induced metric with a Laplacian kernel (CIML). The proposed metric satisfies the conditions of distance metric function and contains first and higher order moments of samples. Moreover, we demonstrate important properties of the proposed metric such as nonnegativity, nonconvexity, and boundedness, and analyze its robustness from the perspective of M-estimation. The proposed metric includes and extends the traditional metrics such as l ₀-norm and l ₁-norm metrics by setting an appropriate parameter. We show that, in reconstruction, the metric is a sparsity-promoting penalty. To reduce the negative effects of noise and outliers, a novel robust reconstruction framework is presented with the proposed correntropy-based metric. The proposed CIML model retains the advantages of the traditional model and promotes robustness. However, the nonconvexity of the proposed metric renders the CIML model difficult to optimize. Furthermore, an effective iterative algorithm for the CIML model is designed, and we present a theoretical analysis of its ability to converge. Numerical simulation and in vivo mouse experiments were conducted to evaluate the CIML method's performance. The experimental results show that the proposed method achieved more accurate fluorescent target reconstruction than the state-of-the-art methods in most cases, which illustrates the feasibility and robustness of the CIML method.