Depth Selection for Deep ReLU Nets in Feature Extraction and Generalization

Exploring Novel Fentanyl Analogues Using a Graph-Based Transformer Model

The structures of fentanyl and its analogues are easy to be modified and few types have been included in database so far, which allow criminals to avoid the supervision of relevant departments. This paper introduces a molecular graph-based transformer model, which is combined with a data augmentation method based on substructure replacement to generate novel fentanyl analogues. 140,000 molecules were generated, and after a set of screening, 36,799 potential fentanyl analogues were finally obtained. We calculated the molecular properties of 36,799 potential fentanyl analogues. The results showed that the model could learn some properties of original fentanyl molecules. We compared the generated molecules from transformer model and data augmentation method based on substructure replacement with those generated by the other two molecular generation models based on deep learning, and found that the model in this paper can generate more novel potential fentanyl analogues. Finally, the findings of the paper indicate that transformer model based on molecular graph helps us explore the structure of potential fentanyl molecules as well as understand distribution of original molecules of fentanyl.

Rational Design of Full-Color Fluorescent C3N Quantum Dots

Carbon-based quantum dots (QDs) exhibit unique photoluminescence due to size-dependent quantum confinement, giving rise to fascinating full-color emission properties. Accurate emission calculations using time-dependent density functional theory are a time-costing and expensive process. Herein, we employed an artificial neural network (ANN) combined with statistical learning to establish the relationship between geometrical/electronic structures of ground states and emission wavelength for C3N QDs. The emission energy of these QDs can be doubly modulated by size and edge effects, which are governed by the number of C4N2 rings and the CH group, respectively. Moreover, these two structural characteristics also determine the phonon vibration mode of C3N QDs to harmonize the emission intensity and lifetime of hot electrons in the electron-hole recombination process, as indicated by nonadiabatic molecular dynamics simulation. These computational results provide a general approach to atomically precise design the full-color fluorescent carbon-based QDs with targeted functions and high performance.

Trends and hotspots in research on medical images with deep learning: a bibliometric analysis from 2013 to 2023

Background

With the rapid development of the internet, the improvement of computer capabilities, and the continuous advancement of algorithms, deep learning has developed rapidly in recent years and has been widely applied in many fields. Previous studies have shown that deep learning has an excellent performance in image processing, and deep learning-based medical image processing may help solve the difficulties faced by traditional medical image processing. This technology has attracted the attention of many scholars in the fields of computer science and medicine. This study mainly summarizes the knowledge structure of deep learning-based medical image processing research through bibliometric analysis and explores the research hotspots and possible development trends in this field.

Methods

Retrieve the Web of Science Core Collection database using the search terms "deep learning," "medical image processing," and their synonyms. Use CiteSpace for visual analysis of authors, institutions, countries, keywords, co-cited references, co-cited authors, and co-cited journals.

Results

The analysis was conducted on 562 highly cited papers retrieved from the database. The trend chart of the annual publication volume shows an upward trend. Pheng-Ann Heng, Hao Chen, and Klaus Hermann Maier-Hein are among the active authors in this field. Chinese Academy of Sciences has the highest number of publications, while the institution with the highest centrality is Stanford University. The United States has the highest number of publications, followed by China. The most frequent keyword is "Deep Learning," and the highest centrality keyword is "Algorithm." The most cited author is Kaiming He, and the author with the highest centrality is Yoshua Bengio.

Conclusion

The application of deep learning in medical image processing is becoming increasingly common, and there are many active authors, institutions, and countries in this field. Current research in medical image processing mainly focuses on deep learning, convolutional neural networks, classification, diagnosis, segmentation, image, algorithm, and artificial intelligence. The research focus and trends are gradually shifting toward more complex and systematic directions, and deep learning technology will continue to play an important role.

Approximation of smooth functionals using deep ReLU networks

In recent years, deep neural networks have been employed to approximate nonlinear continuous functionals F defined on Lp([-1,1]s) for 1≤p≤∞. However, the existing theoretical analysis in the literature either is unsatisfactory due to the poor approximation results, or does not apply to the rectified linear unit (ReLU) activation function. This paper aims to investigate the approximation power of functional deep ReLU networks in two settings: F is continuous with restrictions on the modulus of continuity, and F has higher order Fréchet derivatives. A novel functional network structure is proposed to extract features of higher order smoothness harbored by the target functional F. Quantitative rates of approximation in terms of the depth, width and total number of weights of neural networks are derived for both settings. We give logarithmic rates when measuring the approximation error on the unit ball of a Hölder space. In addition, we establish nearly polynomial rates (i.e., rates of the form exp-a(logM)b with a>0,0 Collapse

Key Words

• Approximation theory
• Deep learning theory
• Fréchet derivative
• Polynomial rates
• ReLU
• Smooth functionals

Collapse

MESH Headings

• Neural Networks, Computer
• Algorithms

Collapse

Grants Collapse

Affiliation(s)

Linhao Song School of Mathematical Science, Beihang University, Beijing, China; School of Data Science, City University of Hong Kong, Kowloon, Hong Kong.
Ying Liu Laboratory for AI-Powered Financial Technologies, Hong Kong Science Park, Shatin, New Territories, Hong Kong.
Jun Fan Department of Mathematics, Hong Kong Baptist University, Kowloon, Hong Kong.
Ding-Xuan Zhou School of Mathematics and Statistics, University of Sydney, Sydney, NSW 2006, Australia.

Collapse

Realization of Spatial Sparseness by Deep ReLU Nets With Massive Data

The great success of deep learning poses urgent challenges for understanding its working mechanism and rationality. The depth, structure, and massive size of the data are recognized to be three key ingredients for deep learning. Most of the recent theoretical studies for deep learning focus on the necessity and advantages of depth and structures of neural networks. In this article, we aim at rigorous verification of the importance of massive data in embodying the outperformance of deep learning. In particular, we prove that the massiveness of data is necessary for realizing the spatial sparseness, and deep nets are crucial tools to make full use of massive data in such an application. All these findings present the reasons why deep learning achieves great success in the era of big data though deep nets and numerous network structures have been proposed at least 20 years ago.

Towards Reduced CNNs for De-Noising Phase Images Corrupted with Speckle Noise

Digital holography is a very efficient technique for 3D imaging and the characterization of changes at the surfaces of objects. However, during the process of holographic interferometry, the reconstructed phase images suffer from speckle noise. In this paper, de-noising is addressed with phase images corrupted with speckle noise. To do so, DnCNN residual networks with different depths were built and trained with various holographic noisy phase data. The possibility of using a network pre-trained on natural images with Gaussian noise is also investigated. All models are evaluated in terms of phase error with HOLODEEP benchmark data and with three unseen images corresponding to different experimental conditions. The best results are obtained using a network with only four convolutional blocks and trained with a wide range of noisy phase patterns.