Information Field Theory and Artificial Intelligence

Information field theory (IFT), the information theory for fields, is a mathematical framework for signal reconstruction and non-parametric inverse problems. Artificial intelligence (AI) and machine learning (ML) aim at generating intelligent systems, including such for perception, cognition, and learning. This overlaps with IFT, which is designed to address perception, reasoning, and inference tasks. Here, the relation between concepts and tools in IFT and those in AI and ML research are discussed. In the context of IFT, fields denote physical quantities that change continuously as a function of space (and time) and information theory refers to Bayesian probabilistic logic equipped with the associated entropic information measures. Reconstructing a signal with IFT is a computational problem similar to training a generative neural network (GNN) in ML. In this paper, the process of inference in IFT is reformulated in terms of GNN training. In contrast to classical neural networks, IFT based GNNs can operate without pre-training thanks to incorporating expert knowledge into their architecture. Furthermore, the cross-fertilization of variational inference methods used in IFT and ML are discussed. These discussions suggest that IFT is well suited to address many problems in AI and ML research and application.

Geometric Variational Inference

Efficiently accessing the information contained in non-linear and high dimensional probability distributions remains a core challenge in modern statistics. Traditionally, estimators that go beyond point estimates are either categorized as Variational Inference (VI) or Markov-Chain Monte-Carlo (MCMC) techniques. While MCMC methods that utilize the geometric properties of continuous probability distributions to increase their efficiency have been proposed, VI methods rarely use the geometry. This work aims to fill this gap and proposes geometric Variational Inference (geoVI), a method based on Riemannian geometry and the Fisher information metric. It is used to construct a coordinate transformation that relates the Riemannian manifold associated with the metric to Euclidean space. The distribution, expressed in the coordinate system induced by the transformation, takes a particularly simple form that allows for an accurate variational approximation by a normal distribution. Furthermore, the algorithmic structure allows for an efficient implementation of geoVI which is demonstrated on multiple examples, ranging from low-dimensional illustrative ones to non-linear, hierarchical Bayesian inverse problems in thousands of dimensions.

A physics perspective on lidar data assimilation for mobile robots

This paper presents a new algorithm for lidar data assimilation relying on a new forward model. Current mapping algorithms suffer from multiple shortcomings, which can be related to the lack of clear forward model. In order to address these issues, we provide a mathematical framework where we show how the use of coarse model parameters results in a new data assimilation problem. Understanding this new problem proves essential to derive sound inference algorithms. We introduce a model parameter specifically tailored for lidar data assimilation, which closely relates to the local mean free path. Using this new model parameter, we derive its associated forward model and we provide the resulting mapping algorithm. We further discuss how our proposed algorithm relates to usual occupancy grid mapping. Finally, we present an example with real lidar measurements.

Cancer detection through Electrical Impedance Tomography and optimal control theory: theoretical and computational analysis

The Inverse Electrical Impedance Tomography (EIT) problem on recovering electrical conductivity tensor and potential in the body based on the measurement of the boundary voltages on the $ m $ electrodes for a given electrode current is analyzed. A PDE constrained optimal control framework in Besov space is developed, where the electrical conductivity tensor and boundary voltages are control parameters, and the cost functional is the norm difference of the boundary electrode current from the given current pattern and boundary electrode voltages from the measurements. The novelty of the control-theoretic model is its adaptation to the clinical situation when additional "voltage-to-current" measurements can increase the size of the input data from $ m $ up to $ m! $ while keeping the size of the unknown parameters fixed. The existence of the optimal control and Fréchet differentiability in the Besov space along with optimality condition is proved. Numerical analysis of the simulated model example in the 2D case demonstrates that by increasing the number of input boundary electrode currents from $ m $ to $ m^2 $ through additional "voltage-to-current" measurements the resolution of the electrical conductivity of the body identified via gradient method in Besov space framework is significantly improved.

Multi-Scale Learned Iterative Reconstruction

Model-based learned iterative reconstruction methods have recently been shown to outperform classical reconstruction algorithms. Applicability of these methods to large scale inverse problems is however limited by the available memory for training and extensive training times, the latter due to computationally expensive forward models. As a possible solution to these restrictions we propose a multi-scale learned iterative reconstruction scheme that computes iterates on discretisations of increasing resolution. This procedure does not only reduce memory requirements, it also considerably speeds up reconstruction and training times, but most importantly is scalable to large scale inverse problems with non-trivial forward operators, such as those that arise in many 3D tomographic applications. In particular, we propose a hybrid network that combines the multiscale iterative approach with a particularly expressive network architecture which in combination exhibits excellent scalability in 3D. Applicability of the algorithm is demonstrated for 3D cone beam computed tomography from real measurement data of an organic phantom. Additionally, we examine scalability and reconstruction quality in comparison to established learned reconstruction methods in two dimensions for low dose computed tomography on human phantoms.

The influence of numerical error on parameter estimation and uncertainty quantification for advective PDE models

Advective partial differential equations can be used to describe many scientific processes. Two significant sources of error that can cause difficulties in inferring parameters from experimental data on these processes include (i) noise from the measurement and collection of experimental data and (ii) numerical error in approximating the forward solution to the advection equation. How this second source of error alters parameter estimation and uncertainty quantification during an inverse problem methodology is not well understood. As a step towards a better understanding of this problem, we present both analytical and computational results concerning how a least squares cost function and parameter estimator behave in the presence of numerical error in approximating solutions to the underlying advection equation. We investigate residual patterns to derive an autocorrelative statistical model that can improve parameter estimation and confidence interval computation for first order methods. Building on our results and their general nature, we provide guidelines for practitioners to determine when numerical or experimental error is the main source of error in their inference, along with suggestions of how to efficiently improve their results.

Image Reconstruction in Electrical Impedance Tomography Based on Structure-Aware Sparse Bayesian Learning

Electrical impedance tomography (EIT) is developed to investigate the internal conductivity changes of an object through a series of boundary electrodes, and has become increasingly attractive in a broad spectrum of applications. However, the design of optimal tomography image reconstruction algorithms has not achieved the adequate level of progress and matureness. In this paper, we propose an efficient and high-resolution EIT image reconstruction method in the framework of sparse Bayesian learning. Significant performance improvement is achieved by imposing structure-aware priors on the learning process to incorporate the prior knowledge that practical conductivity distribution maps exhibit clustered sparsity and intra-cluster continuity. The proposed method not only achieves high-resolution estimation and preserves the shape information even in low signal-to-noise ratio scenarios but also avoids the time-consuming parameter tuning process. The effectiveness of the proposed algorithm is validated through comparisons with state-of-the-art techniques using extensive numerical simulation and phantom experiment results.