Physics-incorporated convolutional recurrent neural networks for source identification and forecasting of dynamical systems

PEPNet: A barotropic primitive equations-based network for wind speed prediction

In wind speed prediction technologies, deep learning-based methods have achieved promising advantages. However, most existing methods focus on learning implicit knowledge in a data-driven manner but neglect some explicit knowledge from the physical theory of meteorological dynamics, failing to make stable and long-term predictions. In this paper, we explore introducing explicit physical knowledge into neural networks and propose Physical Equations Predictive Network (PEPNet) for multi-step wind speed predictions. In PEPNet, a new neural block called the Augmented Neural Barotropic Equations (ANBE) block is designed as its key component, which aims to capture the wind dynamics by combining barotropic primitive equations and deep neural networks. Specifically, the ANBE block adopts a two-branch structure to model wind dynamics, where one branch is physic-based and the other is data-driven-based. The physic-based branch constructs temporal partial derivatives of meteorological elements (including u-component wind, v-component wind, and geopotential height) in a new Neural Barotropic Equations Unit (NBEU). The NBEU is developed based on the barotropic primitive equations mode in numerical weather prediction (NWP). Besides, considering that the barotropic primitive mode is a crude assumption of atmospheric motion, another data-driven-based branch is developed in the ANBE block, which aims at capturing meteorological dynamics beyond barotropic primitive equations. Finally, the PEPNet follows a time-variant structure to enhance the model's capability to capture wind dynamics over time. To evaluate the predictive performance of PEPNet, we have conducted several experiments on two real-world datasets. Experimental results show that the proposed method outperforms the state-of-the-art techniques and achieve optimal performance.

The value of deep neural networks in the pathological classification of thyroid tumors

BACKGROUND

To explore the distinguishing diagnostic value and clinical application potential of deep neural networks (DNN) for pathological images of thyroid tumors.

METHODS

A total of 799 pathological thyroid images of 559 patients with thyroid tumors were retrospectively analyzed. The pathological types included papillary thyroid carcinoma (PTC), medullary thyroid carcinoma (MTC), follicular thyroid carcinoma (FTC), adenomatous goiter, adenoma, and normal thyroid gland. The dataset was divided into a training set and a test set. Resnet50, Resnext50, EfficientNet, and Densenet121 were trained using the training set data and tested with the test set data to determine the diagnostic efficiency of different pathology types and to further analyze the causes of misdiagnosis.

RESULTS

The recall, precision, negative predictive value (NPV), accuracy, specificity, and F1 scores of the four models ranged from 33.33% to 100.00%. The area under curve (AUC) ranged from 0.822 to 0.994, and the Kappa coefficient ranged from 0.7508 to 0.7713. However, the performance of diagnosing FTC, adenoma, and adenomatous goiter was slightly inferior to other types of pathological tissues.

CONCLUSION

The DNN model achieved satisfactory results in the task of classifying thyroid tumors by learning thyroid pathology images. These results indicate the potential of the DNN model for the efficient diagnosis of thyroid tumor histopathology.

HybridBranchNet: A novel structure for branch hybrid convolutional neural networks architecture

ConvNet deep neural networks are developed with a consistent structure. The availability of abundant resources helps these structures to be scaled and redesigned in different sizes so that they can be optimized for different applications. By increasing one or more dimensions of the network, such as depth, resolution and width, the number of trainable network parameters will increase and, as a result, the accuracy and performance It should be noted that the backtracking of the convolutional neural network will improve. However, but increasing the number of network parameters increases the complexity of the network, which is not desirable. Therefore, adjusting the structure of the network, increasing the speed, and reducing the number of network parameters along with ensuring accuracy optimization will be important. This study aims to examine a branch network structure systematically, which can lead to better performance. In this study, in order to increase the speed, to reduce the size of the convolutional network model, and to increase the accuracy optimization, a new scaling method, which optimally designs all dimensions of depth, width, and resolution, is proposed based on a branch neural network. A family of HybridBranchNet networks, which is more accurate and efficient than ConvNets, has been created along with this design. HybridBranchNet3 has a classification accuracy of 83.1%. The proposed model was compared with a family of EfficientNet convolutional networks. The comparison results revealed that the proposed network exceeded the mentioned models in terms of accuracy and speed by 1.03% and 39%, respectively. They also showed that the number of trainable parameters is 13% less than that of the EfficientNet network. The proposed method has an accuracy of 92.3% in the CIFAR-100 dataset and 98.8% in the Flowers-102 dataset. Although the architectures such as CoAtNet have slightly higher classification accuracy than the proposed method, they have a greater number of parameters that cannot be used in a conventional system.

Deep learning-accelerated computational framework based on Physics Informed Neural Network for the solution of linear elasticity

The paper presents an efficient and robust data-driven deep learning (DL) computational framework developed for linear continuum elasticity problems. The methodology is based on the fundamentals of the Physics Informed Neural Networks (PINNs). For an accurate representation of the field variables, a multi-objective loss function is proposed. It consists of terms corresponding to the residual of the governing partial differential equations (PDE), constitutive relations derived from the governing physics, various boundary conditions, and data-driven physical knowledge fitting terms across randomly selected collocation points in the problem domain. To this end, multiple densely connected independent artificial neural networks (ANNs), each approximating a field variable, are trained to obtain accurate solutions. Several benchmark problems including the Airy solution to elasticity and the Kirchhoff-Love plate problem are solved. Performance in terms of accuracy and robustness illustrates the superiority of the current framework showing excellent agreement with analytical solutions. The present work combines the benefits of the classical methods depending on the physical information available in analytical relations with the superior capabilities of the DL techniques in the data-driven construction of lightweight, yet accurate and robust neural networks. The models developed herein can significantly boost computational speed using minimal network parameters with easy adaptability in different computational platforms.

WDMNet: Modeling diverse variations of regional wind speed for multi-step predictions

Regional wind speed prediction plays an important role in the development of wind power, which is usually recorded in the form of two orthogonal components, namely U-wind and V-wind. The regional wind speed has the characteristics of diverse variations, which are reflected in three aspects: (1) The spatially diverse variations of regional wind speed indicate that wind speed has different dynamic patterns at different positions; (2) The distinct variations between U-wind and V-wind denote that U-wind and V-wind at the same position exhibit different dynamic patterns; (3) The non-stationary variations of wind speed represent that the intermittent and chaotic nature of wind speed. In this paper, we propose a novel framework named Wind Dynamics Modeling Network (WDMNet) to model the diverse variations of regional wind speed and make accurate multi-step predictions. To jointly capture the spatially diverse variations and the distinct variations between U-wind and V-wind, WDMNet leverages a new neural block called Involution Gated Recurrent Unit Partial Differential Equation (Inv-GRU-PDE) as its key component. The block adopts involution to model spatially diverse variations and separately constructs hidden driven PDEs of U-wind and V-wind. The construction of PDEs in this block is achieved by a new Involution PDE (InvPDE) layers. Besides, a deep data-driven model is also introduced in Inv-GRU-PDE block as the complement to the constructed hidden PDEs for sufficiently modeling regional wind dynamics. Finally, to effectively capture the non-stationary variations of wind speed, WDMNet follows a time-variant structure for multi-step predictions. Comprehensive experiments have been conducted on two real-world datasets. Experimental results demonstrate the effectiveness and superiority of the proposed method over state-of-the-art techniques.

Unravelled multilevel transformation networks for predicting sparsely observed spatio-temporal dynamics

In this paper, we address the problem of predicting complex, nonlinear spatio-temporal dynamics when available data are recorded at irregularly spaced sparse spatial locations. Most of the existing deep learning models for modelling spatio-temporal dynamics are either designed for data in a regular grid or struggle to uncover the spatial relations from sparse and irregularly spaced data sites. We propose a deep learning model that learns to predict unknown spatio-temporal dynamics using data from sparsely-distributed data sites. We base our approach on the radial basis function (RBF) collocation method which is often used for meshfree solution of partial differential equations. The RBF framework allows us to unravel the observed spatio-temporal function and learn the spatial interactions among data sites on the RBF-space. The learned spatial features are then used to compose multilevel transformations of the raw observations and predict its evolution in future time steps. We demonstrate the advantage of our approach using both synthetic and real-world climate data. This article is part of the theme issue 'Data-driven prediction in dynamical systems'.