Neural networks with a continuous squashing function in the output are universal approximators

Artificial Intelligence in Pediatric Nephrology-A Call for Action

Artificial intelligence is playing an increasingly important role in many fields of clinical care to assist health care providers in patient management. In adult-focused nephrology, artificial intelligence is beginning to be used to improve clinical care, hemodialysis prescriptions, and follow-up of transplant recipients. This article provides an overview of medical artificial intelligence applications relevant to pediatric nephrology. We describe the core concepts of artificial intelligence and machine learning and cover the basics of neural networks and deep learning. We also discuss some examples for clinical applications of artificial intelligence in pediatric nephrology, including neonatal kidney function, early recognition of acute kidney injury, renally cleared drug dosing, intrapatient variability, urinary tract infection workup in infancy, and longitudinal disease progression. Furthermore, we consider the future of artificial intelligence in clinical pediatric nephrology and its potential impact on medical practice and address the ethical issues artificial intelligence raises in terms of clinical decision-making, health care provider-patient relationship, patient privacy, and data collection. This article also represents a call for action involving those of us striving to provide optimal services for children, adolescents, and young adults with chronic conditions.

Safety Analysis of a Certifiable Air Data System Based on Synthetic Sensors for Flow Angle Estimation

This work deals with the safety analysis of an air data system (ADS) partially based on synthetic sensors. The ADS is designed for the small aircraft transportation (SAT) community and is suitable for future unmanned aerial vehicles and urban air mobility applications. The ADS’s main innovation is based on estimation of the flow angles (angle-of-attack and angle-of-sideslip) using synthetic sensors instead of classical vanes (or sensors), whereas pressure and temperature are directly measured with Pitot and temperature probes. As the air data system is a safety-critical system, safety analyses are performed and the results are compared with the safety objectives required by the aircraft integrator. The present paper introduces the common aeronautical procedures for system safety assessment applied to a safety critical system partially based on synthetic sensors. The mean time between failures of ADS’s sub-parts are estimated on a statistical basis in order to evaluate the failure rate of the ADS’s functions. The proposed safety analysis is also useful in identifying the most critical air data system parts and sub-parts. Possible technological gaps to be filled to achieve the airworthiness safety objectives with nonredundant architectures are also identified.

Preliminary Design of a Model-Free Synthetic Sensor for Aerodynamic Angle Estimation for Commercial Aviation

Heterogeneity of the small aircraft category (e.g., small air transport (SAT), urban air mobility (UAM), unmanned aircraft system (UAS)), modern avionic solution (e.g., fly-by-wire (FBW)) and reduced aircraft (A/C) size require more compact, integrated, digital and modular air data system (ADS) able to measure data from the external environment. The MIDAS project, funded in the frame of the Clean Sky 2 program, aims to satisfy those recent requirements with an ADS certified for commercial applications. The main pillar lays on a smart fusion between COTS solutions and analytical sensors (patented technology) for the identification of the aerodynamic angles. The identification involves both flight dynamic relationships and data-driven state observer(s) based on neural techniques, which are deterministic once the training is completed. As this project will bring analytical sensors on board of civil aircraft as part of a redundant system for the very first time, design activities documented in this work have a particular focus on airworthiness certification aspects. At this maturity level, simulated data are used, real flight test data will be used in the next stages. Data collection is described both for the training and test aspects. Training maneuvers are defined aiming to excite all dynamic modes, whereas test maneuvers are collected aiming to validate results independently from the training set and all autopilot configurations. Results demonstrate that an alternate solution is possible enabling significant savings in terms of computational effort and lines of codes but they show, at the same time, that a better training strategy may be beneficial to cope with the new neural network architecture.

Prediction of load in a long bone using an artificial neural network prediction algorithm

The hierarchical nature of bone makes it a difficult material to fully comprehend. The equine third metacarpal (MC3) bone experiences nonuniform surface strains, which are a measure of displacement induced by loads. This paper investigates the use of an artificial neural network expert system to quantify MC3 bone loading. Previous studies focused on determining the response of bone using load, bone geometry, mechanical properties, and constraints as input parameters. This is referred to as a forward problem and is generally solved using numerical techniques such as finite element analysis (FEA). Conversely, an inverse problem has to be solved to quantify load from the measurements of strain and displacement. Commercially available FEA packages, without manipulating their underlying algebraic formulae, are incapable of completing a solution to the inverse problem. In this study, an artificial neural network (ANN) was employed to quantify the load required to produce the MC3 displacement and surface strains determined experimentally. Nine hydrated MC3 bones from thoroughbred horses were loaded in compression in an MTS machine. Ex-vivo experiments measured strain readings from one three-gauge rosette and three distinct single-element gauges at different locations on the MC3 midshaft, associated displacement, and load exposure time. Horse age and bone side (left or right limb) were also recorded for each MC3 bone. This information was used to construct input variables for the ANN model. The ability of this expert system to predict the MC3 loading was investigated. The ANN prediction offered excellent reliability for the prediction of load in the MC3 bones investigated, i.e. R² ≥ 0.98.

Monitoring of total positive end-expiratory pressure during mechanical ventilation by artificial neural networks

Ventilation treatment of acute lung injury (ALI) requires the application of positive airway pressure at the end of expiration (PEEP_app) to avoid lung collapse. However, the total pressure exerted on the alveolar walls (PEEP_tot) is the sum of PEEP_app and intrinsic PEEP (PEEP_i), a hidden component. To measure PEEP_tot, ventilation must be discontinued with an end-expiratory hold maneuver (EEHM). We hypothesized that artificial neural networks (ANN) could estimate the PEEP_tot from flow and pressure tracings during ongoing mechanical ventilation. Ten pigs were mechanically ventilated, and the time constant of their respiratory system (τ_RS) was measured. We shortened their expiratory time (TE) according to multiples of τ_RS, obtaining different respiratory patterns (R_pat). Pressure (P_AW) and flow (V'_AW) at the airway opening during ongoing mechanical ventilation were simultaneously recorded, with and without the addition of external resistance. The last breath of each R_pat included an EEHM, which was used to compute the reference PEEP_tot. The entire protocol was repeated after the induction of ALI with i.v. injection of oleic acid, and 382 tracings were obtained. The ANN had to extract the PEEP_tot, from the tracings without an EEHM. ANN agreement with reference PEEP_tot was assessed with the Bland-Altman method. Bland Altman analysis of estimation error by ANN showed -0.40 ± 2.84 (expressed as bias ± precision) and ±5.58 as limits of agreement (data expressed as cmH₂O). The ANNs estimated the PEEP_tot well at different levels of PEEP_app under dynamic conditions, opening up new possibilities in monitoring PEEP_i in critically ill patients who require ventilator treatment.

On Training Efficiency and Computational Costs of a Feed Forward Neural Network: A Review

A comprehensive review on the problem of choosing a suitable activation function for the hidden layer of a feed forward neural network has been widely investigated. Since the nonlinear component of a neural network is the main contributor to the network mapping capabilities, the different choices that may lead to enhanced performances, in terms of training, generalization, or computational costs, are analyzed, both in general-purpose and in embedded computing environments. Finally, a strategy to convert a network configuration between different activation functions without altering the network mapping capabilities will be presented.

Modelling evolutionary cell behaviour using neural networks: application to tumour growth

In this paper, we present a modelling framework for cellular evolution that is based on the notion that a cell's behaviour is driven by interactions with other cells and its immediate environment. We equip each cell with a phenotype that determines its behaviour and implement a decision mechanism to allow evolution of this phenotype. This decision mechanism is modelled using feed-forward neural networks, which have been suggested as suitable models of cell signalling pathways. The environmental variables are presented as inputs to the network and result in a response that corresponds to the phenotype of the cell. The response of the network is determined by the network parameters, which are subject to mutations when the cells divide. This approach is versatile as there are no restrictions on what the input or output nodes represent, they can be chosen to represent any environmental variables and behaviours that are of importance to the cell population under consideration. This framework was implemented in an individual-based model of solid tumour growth in order to investigate the impact of the tissue oxygen concentration on the growth and evolutionary dynamics of the tumour. Our results show that the oxygen concentration affects the tumour at the morphological level, but more importantly has a direct impact on the evolutionary dynamics. When the supply of oxygen is limited we observe a faster divergence away from the initial genotype, a higher population diversity and faster evolution towards aggressive phenotypes. The implementation of this framework suggests that this approach is well suited for modelling systems where evolution plays an important role and where a changing environment exerts selection pressure on the evolving population.

Computational properties of networks of synchronous groups of spiking neurons

We demonstrate a model in which synchronously firing ensembles of neurons are networked to produce computational results. Each ensemble is a group of biological integrate-and-fire spiking neurons, with probabilistic interconnections between groups. An analogy is drawn in which each individual processing unit of an artificial neural network corresponds to a neuronal group in a biological model. The activation value of a unit in the artificial neural network corresponds to the fraction of active neurons, synchronously firing, in a biological neuronal group. Weights of the artificial neural network correspond to the product of the interconnection density between groups, the group size of the presynaptic group, and the postsynaptic potential heights in the synchronous group model. All three of these parameters can modulate connection strengths between neuronal groups in the synchronous group models. We give an example of nonlinear classification (XOR) and a function approximation example in which the capability of the artificial neural network can be captured by a neural network model with biological integrate-and-fire neurons configured as a network of synchronously firing ensembles of such neurons. We point out that the general function approximation capability proven for feedforward artificial neural networks appears to be approximated by networks of neuronal groups that fire in synchrony, where the groups comprise integrate-and-fire neurons. We discuss the advantages of this type of model for biological systems, its possible learning mechanisms, and the associated timing relationships.

An evolutionary hybrid cellular automaton model of solid tumour growth

We propose a cellular automaton model of solid tumour growth, in which each cell is equipped with a micro-environment response network. This network is modelled using a feed-forward artificial neural network, that takes environmental variables as an input and from these determines the cellular behaviour as the output. The response of the network is determined by connection weights and thresholds in the network, which are subject to mutations when the cells divide. As both available space and nutrients are limited resources for the tumour, this gives rise to clonal evolution where only the fittest cells survive. Using this approach we have investigated the impact of the tissue oxygen concentration on the growth and evolutionary dynamics of the tumour. The results show that the oxygen concentration affects the selection pressure, cell population diversity and morphology of the tumour. A low oxygen concentration in the tissue gives rise to a tumour with a fingered morphology that contains aggressive phenotypes with a small apoptotic potential, while a high oxygen concentration in the tissue gives rise to a tumour with a round morphology containing less evolved phenotypes. The tissue oxygen concentration thus affects the tumour at both the morphological level and on the phenotype level.

A multivariate prediction model for microarray cross-hybridization

Background

Expression microarray analysis is one of the most popular molecular diagnostic techniques in the post-genomic era. However, this technique faces the fundamental problem of potential cross-hybridization. This is a pervasive problem for both oligonucleotide and cDNA microarrays; it is considered particularly problematic for the latter. No comprehensive multivariate predictive modeling has been performed to understand how multiple variables contribute to (cross-) hybridization.

Results

We propose a systematic search strategy using multiple multivariate models [multiple linear regressions, regression trees, and artificial neural network analyses (ANNs)] to select an effective set of predictors for hybridization. We validate this approach on a set of DNA microarrays with cytochrome p450 family genes. The performance of our multiple multivariate models is compared with that of a recently proposed third-order polynomial regression method that uses percent identity as the sole predictor. All multivariate models agree that the 'most contiguous base pairs between probe and target sequences,' rather than percent identity, is the best univariate predictor. The predictive power is improved by inclusion of additional nonlinear effects, in particular target GC content, when regression trees or ANNs are used.

Conclusion

A systematic multivariate approach is provided to assess the importance of multiple sequence features for hybridization and of relationships among these features. This approach can easily be applied to larger datasets. This will allow future developments of generalized hybridization models that will be able to correct for false-positive cross-hybridization signals in expression experiments.

Normalization and analysis of residual variation in two-dimensional gel electrophoresis for quantitative differential proteomics

Although two-dimensional gel electrophoresis (2-DE) has long been a favorite experimental method to screen proteomes, its reproducibility is seldom analyzed with the assistance of quantitative error models. The lack of models of residual distributions that can be used to assign likelihood to differential expression reflects the difficulty in tackling the combined effect of variability in spot intensity and uncertain recognition of the same spot in different gels. In this report we have analyzed a series of four triplicate two-dimensional gels of chicken embryo heart samples at two distinct development stages to produce such a model of residual distribution. In order to achieve this reference error model, a nonparametric procedure for consistent spot intensity normalization had to be established, and is also reported here. In addition to variability in normalized intensity due to various sources, the residual variation between replicates was observed to be compounded by failure to identify the spot itself (gel alignment). The mixed effect is reflected by variably skewed bimodal density distributions of residuals. The extraction of a global error model that accommodated such distribution was achieved empirically by machine learning, specifically by bootstrapped artificial neural networks. The model described is being used to assign confidence values to observed variations in arbitrary 2-DE gels in order to quantify the degree of over-expression and under-expression of protein spots.

Feedforward sigmoidal networks--equicontinuity and fault-tolerance properties

Sigmoidal feedforward artificial neural networks (FFANNs) have been established to be universal approximators of continuous functions. The universal approximation results are summarized to identify the function sets represented by the sigmoidal FFANNs with the universal approximation properties. The equicontinuous properties of the identified sets is analyzed. The equicontinuous property is related to the fault tolerance of the sigmoidal FFANNs. The generally used arbitrary weight sigmoidal FFANNs are shown to be nonequicontinuous sets. A class of bounded weight sigmoidal FFANNs is established to be equicontinuous. The fault-tolerance behavior of the networks is analyzed and error bounds for the induced errors established.

On the Partitioning Capabilities of Feedforward Neural Networks with Sigmoid Nodes

In this letter, the capabilities of feedforward neural networks (FNNs) on the realization and approximation of functions of the form g: R1 → A, which partition the R1 space into polyhedral sets, each one being assigned to one out of the c classes of A, are investigated. More specifically, a constructive proof is given for the fact that FNNs consisting of nodes having sigmoid output functions are capable of approximating any function g with arbitrary accuracy. Also, the capabilities of FNNs consisting of nodes having the hard limiter as output function are reviewed. In both cases, the two-class as well as the multiclass cases are considered.