End-to-End Multi-task Learning Architecture for Brain Tumor Analysis with Uncertainty Estimation in MRI Images

Brain tumors are a threat to life for every other human being, be it adults or children. Gliomas are one of the deadliest brain tumors with an extremely difficult diagnosis. The reason is their complex and heterogenous structure which gives rise to subjective as well as objective errors. Their manual segmentation is a laborious task due to their complex structure and irregular appearance. To cater to all these issues, a lot of research has been done and is going on to develop AI-based solutions that can help doctors and radiologists in the effective diagnosis of gliomas with the least subjective and objective errors, but an end-to-end system is still missing. An all-in-one framework has been proposed in this research. The developed end-to-end multi-task learning (MTL) architecture with a feature attention module can classify, segment, and predict the overall survival of gliomas by leveraging task relationships between similar tasks. Uncertainty estimation has also been incorporated into the framework to enhance the confidence level of healthcare practitioners. Extensive experimentation was performed by using combinations of MRI sequences. Brain tumor segmentation (BraTS) challenge datasets of 2019 and 2020 were used for experimental purposes. Results of the best model with four sequences show 95.1% accuracy for classification, 86.3% dice score for segmentation, and a mean absolute error (MAE) of 456.59 for survival prediction on the test data. It is evident from the results that deep learning-based MTL models have the potential to automate the whole brain tumor analysis process and give efficient results with least inference time without human intervention. Uncertainty quantification confirms the idea that more data can improve the generalization ability and in turn can produce more accurate results with less uncertainty. The proposed model has the potential to be utilized in a clinical setup for the initial screening of glioma patients.

Active Learning in Brain Tumor Segmentation with Uncertainty Sampling and Annotation Redundancy Restriction

Deep learning models have demonstrated great potential in medical imaging but are limited by the expensive, large volume of annotations required. To address this, we compared different active learning strategies by training models on subsets of the most informative images using real-world clinical datasets for brain tumor segmentation and proposing a framework that minimizes the data needed while maintaining performance. Then, 638 multi-institutional brain tumor magnetic resonance imaging scans were used to train three-dimensional U-net models and compare active learning strategies. Uncertainty estimation techniques including Bayesian estimation with dropout, bootstrapping, and margins sampling were compared to random query. Strategies to avoid annotating similar images were also considered. We determined the minimum data necessary to achieve performance equivalent to the model trained on the full dataset (α = 0.05). Bayesian approximation with dropout at training and testing showed results equivalent to that of the full data model (target) with around 30% of the training data needed by random query to achieve target performance (p = 0.018). Annotation redundancy restriction techniques can reduce the training data needed by random query to achieve target performance by 20%. We investigated various active learning strategies to minimize the annotation burden for three-dimensional brain tumor segmentation. Dropout uncertainty estimation achieved target performance with the least annotated data.

Evidence-based uncertainty-aware semi-supervised medical image segmentation

Semi-Supervised Learning (SSL) has demonstrated great potential to reduce the dependence on a large set of annotated data, which is challenging to collect in clinical practice. One of the most important SSL methods is to generate pseudo labels from the unlabeled data using a network model trained with labeled data, which will inevitably introduce false pseudo labels into the training process and potentially jeopardize performance. To address this issue, uncertainty-aware methods have emerged as a promising solution and have gained considerable attention recently. However, current uncertainty-aware methods usually face the dilemma of balancing the additional computational cost, uncertainty estimation accuracy, and theoretical basis in a unified training paradigm. To address this issue, we propose to integrate the Dempster-Shafer Theory of Evidence (DST) into SSL-based medical image segmentation, dubbed EVidential Inference Learning (EVIL). EVIL performs as a novel consistency regularization-based training paradigm, which enforces consistency on predictions perturbed by two networks with different parameters to enhance generalization Additionally, EVIL provides a theoretically assured solution for precise uncertainty quantification within a single forward pass. By discarding highly unreliable pseudo labels after uncertainty estimation, trustworthy pseudo labels can be generated and incorporated into subsequent model training. The experimental results demonstrate that the proposed approach performs competitively when benchmarked against several state-of-the-art methods on public datasets, i.e., ACDC, MM-WHS, and MonuSeg. The code can be found at https://github.com/CYYukio/EVidential-Inference-Learning.

U-PASS: An uncertainty-guided deep learning pipeline for automated sleep staging

With the increasing prevalence of machine learning in critical fields like healthcare, ensuring the safety and reliability of these systems is crucial. Estimating uncertainty plays a vital role in enhancing reliability by identifying areas of high and low confidence and reducing the risk of errors. This study introduces U-PASS, a specialized human-centered machine learning pipeline tailored for clinical applications, which effectively communicates uncertainty to clinical experts and collaborates with them to improve predictions. U-PASS incorporates uncertainty estimation at every stage of the process, including data acquisition, training, and model deployment. Training is divided into a supervised pre-training step and a semi-supervised recording-wise finetuning step. We apply U-PASS to the challenging task of sleep staging and demonstrate that it systematically improves performance at every stage. By optimizing the training dataset, actively seeking feedback from domain experts for informative samples, and deferring the most uncertain samples to experts, U-PASS achieves an impressive expert-level accuracy of 85% on a challenging clinical dataset of elderly sleep apnea patients. This represents a significant improvement over the starting point at 75% accuracy. The largest improvement gain is due to the deferral of uncertain epochs to a sleep expert. U-PASS presents a promising AI approach to incorporating uncertainty estimation in machine learning pipelines, improving their reliability and unlocking their potential in clinical settings.

Dual consistency regularization with subjective logic for semi-supervised medical image segmentation

Semi-supervised learning plays a vital role in computer vision tasks, particularly in medical image analysis. It significantly reduces the time and cost involved in labeling data. Current methods primarily focus on consistency regularization and the generation of pseudo labels. However, due to the model's poor awareness of unlabeled data, aforementioned methods may misguide the model. To alleviate this problem, we propose a dual consistency regularization with subjective logic for semi-supervised medical image segmentation. Specifically, we introduce subjective logic into our semi-supervised medical image segmentation task to estimate uncertainty, and based on the consistency hypothesis, we construct dual consistency regularization under weak and strong perturbations to guide the model's learning from unlabeled data. To evaluate the performance of the proposed method, we performed experiments on three widely used datasets: ACDC, LA, and Pancreas. Experiments show that the proposed method achieved improvement compared with other state-of-the-art (SOTA) methods.

Uncertainty-aware MR-based CT synthesis for robust proton therapy planning of brain tumour

BACKGROUND AND PURPOSE

Deep learning techniques excel in MR-based CT synthesis, but missing uncertainty prediction limits its clinical use in proton therapy. We developed an uncertainty-aware framework and evaluated its efficiency in robust proton planning.

MATERIALS AND METHODS

A conditional generative-adversarial network was trained on 64 brain tumour patients with paired MR-CT images to generate synthetic CTs (sCT) from combined T1-T2 MRs of three orthogonal planes. A Bayesian neural network predicts Laplacian distributions for all voxels with parameters (μ, b). A robust proton plan was optimized using three sCTs of μ and μ±b. The dosimetric differences between the plan from sCT (sPlan) and the recalculated plan (rPlan) on planning CT (pCT) were quantified for each patient. The uncertainty-aware robust plan was compared to conventional robust (global ± 3 %) and non-robust plans.

RESULTS

In 8-fold cross-validation, sCT-pCT image differences (Mean-Absolute-Error) were 80.84 ± 9.84HU (body), 35.78 ± 6.07HU (soft tissues) and 221.88 ± 31.69HU (bones), with Dice scores of 90.33 ± 2.43 %, 95.13 ± 0.80 %, and 85.53 ± 4.16 %, respectively. The uncertainty distribution positively correlated with absolute prediction error (Correlation Coefficient: 0.62 ± 0.01). The uncertainty-conditioned robust optimisation improved the rPlan-sPlan agreement, e.g., D95 absolute difference (CTV) was 1.10 ± 1.24 % compared to conventional (1.64 ± 2.71 %) and non-robust (2.08 ± 2.96 %) optimisation. This trend was consistent across all target and organs-at-risk indexes.

CONCLUSION

The enhanced framework incorporates 3D uncertainty prediction and generates high-quality sCTs from MR images. The framework also facilitates conditioned robust optimisation, bolstering proton plan robustness against network prediction errors. The innovative feature of uncertainty visualisation and robust analyses contribute to evaluating sCT clinical utility for individual patients.

Anatomically-aware uncertainty for semi-supervised image segmentation

Semi-supervised learning relaxes the need of large pixel-wise labeled datasets for image segmentation by leveraging unlabeled data. A prominent way to exploit unlabeled data is to regularize model predictions. Since the predictions of unlabeled data can be unreliable, uncertainty-aware schemes are typically employed to gradually learn from meaningful and reliable predictions. Uncertainty estimation methods, however, rely on multiple inferences from the model predictions that must be computed for each training step, which is computationally expensive. Moreover, these uncertainty maps capture pixel-wise disparities and do not consider global information. This work proposes a novel method to estimate segmentation uncertainty by leveraging global information from the segmentation masks. More precisely, an anatomically-aware representation is first learnt to model the available segmentation masks. The learnt representation thereupon maps the prediction of a new segmentation into an anatomically-plausible segmentation. The deviation from the plausible segmentation aids in estimating the underlying pixel-level uncertainty in order to further guide the segmentation network. The proposed method consequently estimates the uncertainty using a single inference from our representation, thereby reducing the total computation. We evaluate our method on two publicly available segmentation datasets of left atria in cardiac MRIs and of multiple organs in abdominal CTs. Our anatomically-aware method improves the segmentation accuracy over the state-of-the-art semi-supervised methods in terms of two commonly used evaluation metrics.

Reliable prediction intervals with directly optimized inductive conformal regression for deep learning

By generating prediction intervals (PIs) to quantify the uncertainty of each prediction in deep learning regression, the risk of wrong predictions can be effectively controlled. High-quality PIs need to be as narrow as possible, whilst covering a preset proportion of real labels. At present, many approaches to improve the quality of PIs can effectively reduce the width of PIs, but they do not ensure that enough real labels are captured. Inductive Conformal Predictor (ICP) is an algorithm that can generate effective PIs which is theoretically guaranteed to cover a preset proportion of data. However, typically ICP is not directly optimized to yield minimal PI width. In this study, we propose Directly Optimized Inductive Conformal Regression (DOICR) for neural networks that takes only the average width of PIs as the loss function and increases the quality of PIs through an optimized scheme, under the validity condition that sufficient real labels are captured in the PIs. Benchmark experiments show that DOICR outperforms current state-of-the-art algorithms for regression problems using underlying Deep Neural Network structures for both tabular and image data.

Nuclei instance segmentation from histopathology images using Bayesian dropout based deep learning

BACKGROUND

The deterministic deep learning models have achieved state-of-the-art performance in various medical image analysis tasks, including nuclei segmentation from histopathology images. The deterministic models focus on improving the model prediction accuracy without assessing the confidence in the predictions.

METHODS

We propose a semantic segmentation model using Bayesian representation to segment nuclei from the histopathology images and to further quantify the epistemic uncertainty. We employ Bayesian approximation with Monte-Carlo (MC) dropout during the inference time to estimate the model's prediction uncertainty.

RESULTS

We evaluate the performance of the proposed approach on the PanNuke dataset, which consists of 312 visual fields from 19 organ types. We compare the nuclei segmentation accuracy of our approach with that of a fully convolutional neural network, U-Net, SegNet, and the state-of-the-art Hover-net. We use F1-score and intersection over union (IoU) as the evaluation metrics. The proposed approach achieves a mean F1-score of 0.893 ± 0.008 and an IoU value of 0.868 ± 0.003 on the test set of the PanNuke dataset. These results outperform the Hover-net, which has a mean F1-score of 0.871 ± 0.010 and an IoU value of 0.840 ± 0.032.

CONCLUSIONS

The proposed approach, which incorporates Bayesian representation and Monte-Carlo dropout, demonstrates superior performance in segmenting nuclei from histopathology images compared to existing models such as U-Net, SegNet, and Hover-net. By considering the epistemic uncertainty, our model provides a more reliable estimation of the prediction confidence. These findings highlight the potential of Bayesian deep learning for improving medical image analysis tasks and can contribute to the development of more accurate and reliable computer-aided diagnostic systems.

Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise

In the past years, deep learning has seen an increase in usage in the domain of histopathological applications. However, while these approaches have shown great potential, in high-risk environments deep learning models need to be able to judge their uncertainty and be able to reject inputs when there is a significant chance of misclassification. In this work, we conduct a rigorous evaluation of the most commonly used uncertainty and robustness methods for the classification of Whole Slide Images, with a focus on the task of selective classification, where the model should reject the classification in situations in which it is uncertain. We conduct our experiments on tile-level under the aspects of domain shift and label noise, as well as on slide-level. In our experiments, we compare Deep Ensembles, Monte-Carlo Dropout, Stochastic Variational Inference, Test-Time Data Augmentation as well as ensembles of the latter approaches. We observe that ensembles of methods generally lead to better uncertainty estimates as well as an increased robustness towards domain shifts and label noise, while contrary to results from classical computer vision benchmarks no systematic gain of the other methods can be shown. Across methods, a rejection of the most uncertain samples reliably leads to a significant increase in classification accuracy on both in-distribution as well as out-of-distribution data. Furthermore, we conduct experiments comparing these methods under varying conditions of label noise. Lastly, we publish our code framework to facilitate further research on uncertainty estimation on histopathological data.

ADNet++: A few-shot learning framework for multi-class medical image volume segmentation with uncertainty-guided feature refinement

A major barrier to applying deep segmentation models in the medical domain is their typical data-hungry nature, requiring experts to collect and label large amounts of data for training. As a reaction, prototypical few-shot segmentation (FSS) models have recently gained traction as data-efficient alternatives. Nevertheless, despite the recent progress of these models, they still have some essential shortcomings that must be addressed. In this work, we focus on three of these shortcomings: (i) the lack of uncertainty estimation, (ii) the lack of a guiding mechanism to help locate edges and encourage spatial consistency in the segmentation maps, and (iii) the models' inability to do one-step multi-class segmentation. Without modifying or requiring a specific backbone architecture, we propose a modified prototype extraction module that facilitates the computation of uncertainty maps in prototypical FSS models, and show that the resulting maps are useful indicators of the model uncertainty. To improve the segmentation around boundaries and to encourage spatial consistency, we propose a novel feature refinement module that leverages structural information in the input space to help guide the segmentation in the feature space. Furthermore, we demonstrate how uncertainty maps can be used to automatically guide this feature refinement. Finally, to avoid ambiguous voxel predictions that occur when images are segmented class-by-class, we propose a procedure to perform one-step multi-class FSS. The efficiency of our proposed methodology is evaluated on two representative datasets for abdominal organ segmentation (CHAOS dataset and BTCV dataset) and one dataset for cardiac segmentation (MS-CMRSeg dataset). The results show that our proposed methodology significantly (one-sided Wilcoxon signed rank test, p<0.05) improves the baseline, increasing the overall dice score with +5.2, +5.1, and +2.8 percentage points for the CHAOS dataset, the BTCV dataset, and the MS-CMRSeg dataset, respectively.

Clinical target volume delineation quality assurance for MRI-guided prostate radiotherapy using deep learning with uncertainty estimation

BACKGROUND AND PURPOSE

Previous studies on automatic delineation quality assurance (QA) have mostly focused on CT-based planning. As MRI-guided radiotherapy is increasingly utilized in prostate cancer treatment, there is a need for more research on MRI-specific automatic QA. This work proposes a clinical target volume (CTV) delineation QA framework based on deep learning (DL) for MRI-guided prostate radiotherapy.

MATERIALS AND METHODS

The proposed workflow utilized a 3D dropblock ResUnet++ (DB-ResUnet++) to generate multiple segmentation predictions via Monte Carlo dropout which were used to compute an average delineation and area of uncertainty. A logistic regression (LR) classifier was employed to classify the manual delineation as pass or discrepancy based on the spatial association between the manual delineation and the network's outputs. This approach was evaluated on a multicentre MRI-only prostate radiotherapy dataset and compared with our previously published QA framework based on AN-AG Unet.

RESULTS

The proposed framework achieved an area under the receiver operating curve (AUROC) of 0.92, a true positive rate (TPR) of 0.92 and a false positive rate of 0.09 with an average processing time per delineation of 1.3 min. Compared with our previous work using AN-AG Unet, this method generated fewer false positive detections at the same TPR with a much faster processing speed.

CONCLUSION

To the best of our knowledge, this is the first study to propose an automatic delineation QA tool using DL with uncertainty estimation for MRI-guided prostate radiotherapy, which can potentially be used for reviewing prostate CTV delineation in multicentre clinical trials.

Analysis of predictive performance and reliability of classifiers for quality assessment of medical evidence revealed important variation by medical area

OBJECTIVES

A major obstacle in deployment of models for automated quality assessment is their reliability. To analyze their calibration and selective classification performance.

STUDY DESIGN AND SETTING

We examine two systems for assessing the quality of medical evidence, EvidenceGRADEr and RobotReviewer, both developed from Cochrane Database of Systematic Reviews (CDSR) to measure strength of bodies of evidence and risk of bias (RoB) of individual studies, respectively. We report their calibration error and Brier scores, present their reliability diagrams, and analyze the risk-coverage trade-off in selective classification.

RESULTS

The models are reasonably well calibrated on most quality criteria (expected calibration error [ECE] 0.04-0.09 for EvidenceGRADEr, 0.03-0.10 for RobotReviewer). However, we discover that both calibration and predictive performance vary significantly by medical area. This has ramifications for the application of such models in practice, as average performance is a poor indicator of group-level performance (e.g., health and safety at work, allergy and intolerance, and public health see much worse performance than cancer, pain, and anesthesia, and Neurology). We explore the reasons behind this disparity.

CONCLUSION

Practitioners adopting automated quality assessment should expect large fluctuations in system reliability and predictive performance depending on the medical area. Prospective indicators of such behavior should be further researched.

Gaussian Processes for real-time 3D motion and uncertainty estimation during MR-guided radiotherapy

Respiratory motion during radiotherapy causes uncertainty in the tumor's location, which is typically addressed by an increased radiation area and a decreased dose. As a result, the treatments' efficacy is reduced. The recently proposed hybrid MR-linac scanner holds the promise to efficiently deal with such respiratory motion through real-time adaptive MR-guided radiotherapy (MRgRT). For MRgRT, motion-fields should be estimated from MR-data and the radiotherapy plan should be adapted in real-time according to the estimated motion-fields. All of this should be performed with a total latency of maximally 200 ms, including data acquisition and reconstruction. A measure of confidence in such estimated motion-fields is highly desirable, for instance to ensure the patient's safety in case of unexpected and undesirable motion. In this work, we propose a framework based on Gaussian Processes to infer 3D motion-fields and uncertainty maps in real-time from only three readouts of MR-data. We demonstrated an inference frame rate up to 69 Hz including data acquisition and reconstruction, thereby exploiting the limited amount of required MR-data. Additionally, we designed a rejection criterion based on the motion-field uncertainty maps to demonstrate the framework's potential for quality assurance. The framework was validated in silico and in vivo on healthy volunteer data (n=5) acquired using an MR-linac, thereby taking into account different breathing patterns and controlled bulk motion. Results indicate end-point-errors with a 75th percentile below 1 mm in silico, and a correct detection of erroneous motion estimates with the rejection criterion. Altogether, the results show the potential of the framework for application in real-time MR-guided radiotherapy with an MR-linac.

Quantifying uncertainty of marine water quality forecasts for environmental management using a dynamic multi-factor analysis and multi-resolution ensemble approach

Unpredictable climate change and human activities pose enormous challenges to assessing the water quality components in the marine environment. Accurately quantifying the uncertainty of water quality forecasts can help decision-makers implement more scientific water pollution management strategies. This work introduces a new method of uncertainty quantification driven by point prediction for solving the engineering problem of water quality forecasting under the influence of complex environmental factors. The constructed multi-factor correlation analysis system can dynamically adjust the combined weight of environmental indicators according to the performance, thereby increasing the interpretability of data fusion. The designed singular spectrum analysis is utilized to reduce the volatility of the original water quality data. The real-time decomposition technique cleverly avoids the problem of data leakage. The multi-resolution-multi-objective optimization ensemble method is adopted to absorb the characteristics of different resolution data, so as to mine deeper potential information. Experimental studies are conducted using 6 actual water quality high-resolution signals with 21,600 sampling points from the Pacific islands and corresponding low-resolution signals with 900 sampling points, including temperature, salinity, turbidity, chlorophyll, dissolved oxygen, and oxygen saturation. The results illustrate that the model is superior to the existing model in quantifying the uncertainty of water quality prediction.

Calibrating segmentation networks with margin-based label smoothing

Despite the undeniable progress in visual recognition tasks fueled by deep neural networks, there exists recent evidence showing that these models are poorly calibrated, resulting in over-confident predictions. The standard practices of minimizing the cross-entropy loss during training promote the predicted softmax probabilities to match the one-hot label assignments. Nevertheless, this yields a pre-softmax activation of the correct class that is significantly larger than the remaining activations, which exacerbates the miscalibration problem. Recent observations from the classification literature suggest that loss functions that embed implicit or explicit maximization of the entropy of predictions yield state-of-the-art calibration performances. Despite these findings, the impact of these losses in the relevant task of calibrating medical image segmentation networks remains unexplored. In this work, we provide a unifying constrained-optimization perspective of current state-of-the-art calibration losses. Specifically, these losses could be viewed as approximations of a linear penalty (or a Lagrangian term) imposing equality constraints on logit distances. This points to an important limitation of such underlying equality constraints, whose ensuing gradients constantly push towards a non-informative solution, which might prevent from reaching the best compromise between the discriminative performance and calibration of the model during gradient-based optimization. Following our observations, we propose a simple and flexible generalization based on inequality constraints, which imposes a controllable margin on logit distances. Comprehensive experiments on a variety of public medical image segmentation benchmarks demonstrate that our method sets novel state-of-the-art results on these tasks in terms of network calibration, whereas the discriminative performance is also improved. The code is available at https://github.com/Bala93/MarginLoss.

Labeling confidence for uncertainty-aware histology image classification

Deep learning-based models applied to digital pathology require large, curated datasets with high-quality (HQ) annotations to perform correctly. In many cases, recruiting expert pathologists to annotate large databases is not feasible, and it is necessary to collect additional labeled data with varying label qualities, e.g., pathologists-in-training (henceforth, non-expert annotators). Learning from datasets with noisy labels is more challenging in medical applications since medical imaging datasets tend to have instance-dependent noise and suffer from high inter/intra-observer variability. In this paper, we design an uncertainty-driven labeling strategy with which we generate soft labels from 10 non-expert annotators for multi-class skin cancer classification. Based on this soft annotation, we propose an uncertainty estimation-based framework to handle these noisy labels. This framework is based on a novel formulation using a dual-branch min-max entropy calibration to penalize inexact labels during the training. Comprehensive experiments demonstrate the promising performance of our labeling strategy. Results show a consistent improvement by using soft labels with standard cross-entropy loss during training (∼4.0% F1-score) and increases when calibrating the model with the proposed min-max entropy calibration (∼6.6% F1-score). These improvements are produced at negligible cost, both in terms of annotation and calculation.

Predictive uncertainty estimation for out-of-distribution detection in digital pathology

Machine learning model deployment in clinical practice demands real-time risk assessment to identify situations in which the model is uncertain. Once deployed, models should be accurate for classes seen during training while providing informative estimates of uncertainty to flag abnormalities and unseen classes for further analysis. Although recent developments in uncertainty estimation have resulted in an increasing number of methods, a rigorous empirical evaluation of their performance on large-scale digital pathology datasets is lacking. This work provides a benchmark for evaluating prevalent methods on multiple datasets by comparing the uncertainty estimates on both in-distribution and realistic near and far out-of-distribution (OOD) data on a whole-slide level. To this end, we aggregate uncertainty values from patch-based classifiers to whole-slide level uncertainty scores. We show that results found in classical computer vision benchmarks do not always translate to the medical imaging setting. Specifically, we demonstrate that deep ensembles perform best at detecting far-OOD data but can be outperformed on a more challenging near-OOD detection task by multi-head ensembles trained for optimal ensemble diversity. Furthermore, we demonstrate the harmful impact OOD data can have on the performance of deployed machine learning models. Overall, we show that uncertainty estimates can be used to discriminate in-distribution from OOD data with high AUC scores. Still, model deployment might require careful tuning based on prior knowledge of prospective OOD data.

DragNet: Learning-based deformable registration for realistic cardiac MR sequence generation from a single frame

Deformable image registration (DIR) can be used to track cardiac motion. Conventional DIR algorithms aim to establish a dense and non-linear correspondence between independent pairs of images. They are, nevertheless, computationally intensive and do not consider temporal dependencies to regulate the estimated motion in a cardiac cycle. In this paper, leveraging deep learning methods, we formulate a novel hierarchical probabilistic model, termed DragNet, for fast and reliable spatio-temporal registration in cine cardiac magnetic resonance (CMR) images and for generating synthetic heart motion sequences. DragNet is a variational inference framework, which takes an image from the sequence in combination with the hidden states of a recurrent neural network (RNN) as inputs to an inference network per time step. As part of this framework, we condition the prior probability of the latent variables on the hidden states of the RNN utilised to capture temporal dependencies. We further condition the posterior of the motion field on a latent variable from hierarchy and features from the moving image. Subsequently, the RNN updates the hidden state variables based on the feature maps of the fixed image and the latent variables. Different from traditional methods, DragNet performs registration on unseen sequences in a forward pass, which significantly expedites the registration process. Besides, DragNet enables generating a large number of realistic synthetic image sequences given only one frame, where the corresponding deformations are also retrieved. The probabilistic framework allows for computing spatio-temporal uncertainties in the estimated motion fields. Our results show that DragNet performance is comparable with state-of-the-art methods in terms of registration accuracy, with the advantage of offering analytical pixel-wise motion uncertainty estimation across a cardiac cycle and being a motion generator. We will make our code publicly available.

Quantification and dosimetry of small volumes including associated uncertainty estimation

BACKGROUND

Accurate quantification of radioactivity in a source of interest relies on accurate registration between SPECT and anatomical images, and appropriate correction of partial volume effects (PVEs). For small volumes, exact registration between the two imaging modalities and recovery factors used to correct for PVE are unreliable. There is currently no guidance relating to quantification or the associated uncertainty estimation for small volumes.

MATERIAL AND METHODS

A method for quantification of small sources of interest is proposed, which uses multiple oversized volumes of interest. The method was applied to three Na[¹³¹I]I activity distributions where a Na[¹³¹I]I capsule was situated within a cylindrical phantom containing either zero background, uniform background or non-uniform background and to a scenario with small lesions placed in an anthropomorphic phantom. The Na[¹³¹I]I capsule and lesions were quantified using the proposed method and compared with measurements made using two alternative quantification methods. The proposed method was also applied to assess the absorbed dose delivered to a bone metastasis following [¹³¹I]mIBG therapy for neuroblastoma including the associated uncertainty estimation.

RESULTS

The method is accurate across a range of activities and in varied radioactivity distributions. Median percentage errors using the proposed method in no background, uniform backgrounds and non-uniform backgrounds were - 0.4%, - 0.3% and 1.7% with median associated uncertainties of 1.4%, 1.4% and 1.6%, respectively. The technique is more accurate and robust when compared to currently available alternative methods.

CONCLUSIONS

The proposed method provides a reliable and accurate method for quantification of sources of interest, which are less than three times the spatial resolution of the imaging system. The method may be of use in absorbed dose calculation in cases of bone metastasis, lung metastasis or thyroid remnants.

Distance-based detection of out-of-distribution silent failures for Covid-19 lung lesion segmentation

Automatic segmentation of ground glass opacities and consolidations in chest computer tomography (CT) scans can potentially ease the burden of radiologists during times of high resource utilisation. However, deep learning models are not trusted in the clinical routine due to failing silently on out-of-distribution (OOD) data. We propose a lightweight OOD detection method that leverages the Mahalanobis distance in the feature space and seamlessly integrates into state-of-the-art segmentation pipelines. The simple approach can even augment pre-trained models with clinically relevant uncertainty quantification. We validate our method across four chest CT distribution shifts and two magnetic resonance imaging applications, namely segmentation of the hippocampus and the prostate. Our results show that the proposed method effectively detects far- and near-OOD samples across all explored scenarios.

Uncertainty estimation for margin detection in cancer surgery using mass spectrometry

PURPOSE

Rapid evaporative ionization mass spectrometry (REIMS) is an emerging technology for clinical margin detection. Deployment of REIMS depends on construction of reliable deep learning models that can categorize tissue according to its metabolomic signature. Challenges associated with developing these models include the presence of noise during data acquisition and the variance in tissue signatures between patients. In this study, we propose integration of uncertainty estimation in deep models to factor predictive confidence into margin detection in cancer surgery.

METHODS

iKnife is used to collect 693 spectra of cancer and healthy samples acquired from 91 patients during basal cell carcinoma resection. A Bayesian neural network and two baseline models are trained on these data to perform classification as well as uncertainty estimation. The samples with high estimated uncertainty are then removed, and new models are trained using the clean data. The performance of proposed and baseline models, with different ratios of filtered data, is then compared.

RESULTS

The data filtering does not improve the performance of the baseline models as they cannot provide reliable estimations of uncertainty. In comparison, the proposed model demonstrates a statistically significant improvement in average balanced accuracy (75.2%), sensitivity (74.1%) and AUC (82.1%) after removing uncertain training samples. We also demonstrate that if highly uncertain samples are predicted and removed from the test data, sensitivity further improves to 88.2%.

CONCLUSIONS

This is the first study that applies uncertainty estimation to inform model training and deployment for tissue recognition in cancer surgery. Uncertainty estimation is leveraged in two ways: by factoring a measure of input noise in training the models and by including predictive confidence in reporting the outputs. We empirically show that considering uncertainty for model development can help improve the overall accuracy of a margin detection system using REIMS.

Uncertainty-aware dynamic integration for multi-omics classification of tumors

PURPOSE

Omics data are crucial for medical diagnosis as it contains intrinsic biomedical information. Multi-omics integrated analysis has become a new direction for scientists to explore life mechanisms. Nevertheless, the quality of complex omics data often varies greatly due to different samples or even different omics types, it is challenging to dynamically capture the uncertainty for different kinds of omics data.

METHODS

This paper proposes a uncertainty-aware dynamic integration framework for multi-omics classification. The framework consists of three modules: deep embedding, confidence prediction, and downstream tasks. The deep embedding module extract key information from multi-omics data to obtain a low-dimensional feature representation which is used to train downstream tasks. Combined with the deep embedding module, the confidence prediction module is used to dynamically capture the uncertainty of the data. We introduce "confidNet" to assign a confidence value for each type of omics data, which is used for dynamic integration between multi-omics.

RESULTS

Compared with other integration methods, the proposed method can contain more crucial biomedical information in the obtained low-dimensional representation. Our framework realizes reliable integration among multiple omics, and it can still achieve high accuracy on small sample data sets. We have verified the effectiveness of the model in a large number of experiments.

CONCLUSION

Our framework can be widely applied to high-dimensional omics data and has great potential to facilitate medical decision-making and biological analysis.

Blood-brain barrier penetration prediction enhanced by uncertainty estimation

Blood–brain barrier is a pivotal factor to be considered in the process of central nervous system (CNS) drug development, and it is of great significance to rapidly explore the blood–brain barrier permeability (BBBp) of compounds in silico in early drug discovery process. Here, we focus on whether and how uncertainty estimation methods improve in silico BBBp models. We briefly surveyed the current state of in silico BBBp prediction and uncertainty estimation methods of deep learning models, and curated an independent dataset to determine the reliability of the state-of-the-art algorithms. The results exhibit that, despite the comparable performance on BBBp prediction between graph neural networks-based deep learning models and conventional physicochemical-based machine learning models, the GROVER-BBBp model shows greatly improvement when using uncertainty estimations. In particular, the strategy combined Entropy and MC-dropout can increase the accuracy of distinguishing BBB + from BBB − to above 99% by extracting predictions with high confidence level (uncertainty score < 0.1). Case studies on preclinical/clinical drugs for Alzheimer’ s disease and marketed antitumor drugs that verified by literature proved the application value of uncertainty estimation enhanced BBBp prediction model, that may facilitate the drug discovery in the field of CNS diseases and metastatic brain tumors.

Comparison of Bayesian, k-Nearest Neighbor and Gaussian process regression methods for quantifying uncertainty of suspended sediment concentration prediction

Suspended sediment transport in river system is a complex process influenced by many factors that their interactions lead to nonlinear and high scatter of concentration-discharge relationships. This makes the model prediction subject to high uncertainty and providing one value as the model prediction is somehow useless and cannot provide adequate information about the model accuracy and associated uncertainty. Current study compares the efficiency of Bayesian (i.e. Bayesian segmented linear regression (BSLR) and Bayesian linear model (BLR)), Gaussian Process Regression (GPR) and k-Nearest Neighbor (k-NN) in quantifying uncertainty of the suspended sediment concentration prediction in three watersheds namely Arazkoseh, Oghan and Jajrood located in Iran. Three input combinations including, contemporary discharge, slow and quick flow components and contemporary, one and two antecedent days discharge, were used. The BSLR model was able to identify threshold value, furthermore, pre-threshold and post-threshold slopes of BSLR model indicated that for Arazkoseh watershed channel and for Oghan and Jajrood watersheds, upland area are dominate sediment sources. In all three studied cases, given prediction interval width and the percent of enclosed observed data by prediction interval, k-NN model provided more reliable prediction interval. Moreover, separation stream flow into slow and quick flow components lead to improved performance of GPR and k-NN models in the studied watersheds, and the best results for Arazkoseh and Oghan watersheds were obtained when slow and quick flow components were used as the model input.

Detecting out-of-distribution samples via variational auto-encoder with reliable uncertainty estimation

Variational autoencoders (VAEs) are influential generative models with rich representation capabilities from the deep neural network architecture and Bayesian method. However, VAE models have a weakness that assign a higher likelihood to out-of-distribution (OOD) inputs than in-distribution (ID) inputs. To address this problem, a reliable uncertainty estimation is considered to be critical for in-depth understanding of OOD inputs. In this study, we propose an improved noise contrastive prior (INCP) to be able to integrate into the encoder of VAEs, called INCPVAE. INCP is scalable, trainable and compatible with VAEs, and it also adopts the merits from the INCP for uncertainty estimation. Experiments on various datasets demonstrate that compared to the standard VAEs, our model is superior in uncertainty estimation for the OOD data and is robust in anomaly detection tasks. The INCPVAE model obtains reliable uncertainty estimation for OOD inputs and solves the OOD problem in VAE models.

Probabilistic Random Forest improves bioactivity predictions close to the classification threshold by taking into account experimental uncertainty

Measurements of protein–ligand interactions have reproducibility limits due to experimental errors. Any model based on such assays will consequentially have such unavoidable errors influencing their performance which should ideally be factored into modelling and output predictions, such as the actual standard deviation of experimental measurements (σ) or the associated comparability of activity values between the aggregated heterogenous activity units (i.e., K_i versus IC₅₀ values) during dataset assimilation. However, experimental errors are usually a neglected aspect of model generation. In order to improve upon the current state-of-the-art, we herein present a novel approach toward predicting protein–ligand interactions using a Probabilistic Random Forest (PRF) classifier. The PRF algorithm was applied toward in silico protein target prediction across ~ 550 tasks from ChEMBL and PubChem. Predictions were evaluated by taking into account various scenarios of experimental standard deviations in both training and test sets and performance was assessed using fivefold stratified shuffled splits for validation. The largest benefit in incorporating the experimental deviation in PRF was observed for data points close to the binary threshold boundary, when such information was not considered in any way in the original RF algorithm. For example, in cases when σ ranged between 0.4–0.6 log units and when ideal probability estimates between 0.4–0.6, the PRF outperformed RF with a median absolute error margin of ~ 17%. In comparison, the baseline RF outperformed PRF for cases with high confidence to belong to the active class (far from the binary decision threshold), although the RF models gave errors smaller than the experimental uncertainty, which could indicate that they were overtrained and/or over-confident. Finally, the PRF models trained with putative inactives decreased the performance compared to PRF models without putative inactives and this could be because putative inactives were not assigned an experimental pXC₅₀ value, and therefore they were considered inactives with a low uncertainty (which in practice might not be true). In conclusion, PRF can be useful for target prediction models in particular for data where class boundaries overlap with the measurement uncertainty, and where a substantial part of the training data is located close to the classification threshold.

Effect of setting data collection parameters on the reliability of a circular dichroism spectrum

Circular dichroism (CD) spectroscopy is a well-established biophysical technique used to investigate the structure of molecules. The analysis of a protein CD spectrum depends on the quality of the original CD data, which can be affected by the sample purity, background absorption of the additives/solvent/buffer, the choice of the parameters used for data collection, etc. In this paper, the CD spectrum of myoglobin was used as a model to exploit how variations on each data collection parameter could affect the final protein CD spectrum and, the subsequent effect of them on the quantitative analysis of protein secondary structure. Bioinformatics analysis carried out with SESCA package and PDBMD2CD server predicted a theoretical myoglobin CD spectrum, and a Monte Carlo-like model was implemented to estimate the uncertainty in secondary structure predictions performed with CDSSTR, Selcon 3 and ContinLL algorithms. An inappropriate choice of data collection parameters can lead to a misinterpretation of the CD data in terms of the protein structural content.

Iterative confidence relabeling with deep ConvNets for organ segmentation with partial labels

Training deep ConvNets requires large labeled datasets. However, collecting pixel-level labels for medical image segmentation is very expensive and requires a high level of expertise. In addition, most existing segmentation masks provided by clinical experts focus on specific anatomical structures. In this paper, we propose a method dedicated to handle such partially labeled medical image datasets. We propose a strategy to identify pixels for which labels are correct, and to train Fully Convolutional Neural Networks with a multi-label loss adapted to this context. In addition, we introduce an iterative confidence self-training approach inspired by curriculum learning to relabel missing pixel labels, which relies on selecting the most confident prediction with a specifically designed confidence network that learns an uncertainty measure which is leveraged in our relabeling process. Our approach, INERRANT for Iterative coNfidencE Relabeling of paRtial ANnoTations, is thoroughly evaluated on two public datasets (TCAI and LITS), and one internal dataset with seven abdominal organ classes. We show that INERRANT robustly deals with partial labels, performing similarly to a model trained on all labels even for large missing label proportions. We also highlight the importance of our iterative learning scheme and the proposed confidence measure for optimal performance. Finally we show a practical use case where a limited number of completely labeled data are enriched by publicly available but partially labeled data.

Improving Uncertainty Estimation With Semi-Supervised Deep Learning for COVID-19 Detection Using Chest X-Ray Images

In this work we implement a COVID-19 infection detection system based on chest X-ray images with uncertainty estimation. Uncertainty estimation is vital for safe usage of computer aided diagnosis tools in medical applications. Model estimations with high uncertainty should be carefully analyzed by a trained radiologist. We aim to improve uncertainty estimations using unlabelled data through the MixMatch semi-supervised framework. We test popular uncertainty estimation approaches, comprising Softmax scores, Monte-Carlo dropout and deterministic uncertainty quantification. To compare the reliability of the uncertainty estimates, we propose the usage of the Jensen-Shannon distance between the uncertainty distributions of correct and incorrect estimations. This metric is statistically relevant, unlike most previously used metrics, which often ignore the distribution of the uncertainty estimations. Our test results show a significant improvement in uncertainty estimates when using unlabelled data. The best results are obtained with the use of the Monte Carlo dropout method.

Multi-model ensemble simulated non-point source pollution based on Bayesian model averaging method and model uncertainty analysis

Watershed models are cost-effective and powerful tools for evaluating and controlling non-point source pollution (NPSP), while the reliability of watershed models in a management context depends largely on inherent uncertainties in model predictions. The objective of this study is to present the use of multi-model ensemble applied to streamflow, total nitrogen (TN), and total phosphorus (TP) simulation and quantify the uncertainty resulting from model structure. In this study, three watershed models, which have different structures in simulating NPSP, were selected to conduct watershed monthly streamflow, TN load, and TP load ensemble simulation and 90% credible intervals based on Bayesian model averaging (BMA) method. The result using the observed data of the Yixunhe watershed revealed that the coefficient of determination and Nash-Sutcliffe coefficient of the BMA model simulate streamflow, TN load, and TP load were better than that of the single model. The higher the efficiency of a single model is, the greater the weight during the BMA ensemble simulation is. The 90% credible interval of BMA has a high coverage of measured values in this study. This indicates that the BMA method can not only provide simulation with better precision through ensemble simulation but also provide quantitative evaluation of the model structure through interval, which could offer rich information of the NPSP simulation and management.

Dithiothreitol-based oxidative potential for airborne particulate matter: an estimation of the associated uncertainty

Oxidative stress is considered as one of the main mechanisms by which airborne particles produce adverse health effects. Several methods to estimate the oxidative potential (OP) of particulate matter (PM) have been proposed. Among them, the dithiothreitol (DTT) assay has gained popularity due to its simplicity and overall low implementation cost. Usually, the estimations of OP^DTT are based on n-replicates of a set of samples and their associated standard deviation. However, interlaboratory comparisons of OP^DTT can be difficult and lead to misinterpretations. This work presents an estimation of the total uncertainty for the OP^DTT measurement of PM₁₀ and PM_2.5 samples collected in Santiago (Chile), based on recommendations by the Joint Committee for Guides in Metrology and Eurachem. The expanded uncertainty expressed as a percentage of the mass-normalized OP^DTT measurements was 18.0% and 16.3% for PM₁₀ and PM_2.5 samples respectively. The dominating contributor to the total uncertainty was identified (i.e., DTT consumption rate, related to the regression and repeatability of experimental data), while the volumetric operations (i.e., pipettes) were also important. The results showed that, although the OP measured following the DTT assay has been successfully used to estimate the potential health impacts of airborne PM, uncertainty estimations must be considered before interpreting the results.

Robust control design for an industrial wind turbine with HIL simulations

This paper describes the design and implementation of a H_∞-based robust controller for a commercial 2MW variable-speed variable-pitch wind turbine. The single controller is able to deliver specification performance for the entire full load region. Various aspects of modeling and design procedures including reduced order model validation, torsional mode damping of drivetrain and uncertainty estimation are explained in detail. To make the design more reliable and industrially attractive, simple routines for the derivation of required weighting functions are introduced. To investigate the benefits of the suggested design, its closed-loop performance is compared with the wind turbine's baseline controller and an alternative state-space controller proposed in the literature. This is first evaluated using a FAST-based simulator under IEC-61400 compatible scenarios. It is then verified by a real-life hardware in the loop simulator using an industrial PLC.

Modeling study on the time-varying process of sediment trapping in vegetative filter strips

Vegetative filter strips (VFS) play an important role in reducing erosion, retaining runoff, and trapping sediment. The bed surface of a VFS is dynamic during continuous sediment deposition, which makes it difficult to determine the sediment transport capacity of overland flow in real time. This study described this process from a systematic perspective. Based on self-feedback theory, an ordinary differential equation was established with a self-inhibition correction factor and its analytical solution was obtained. A sediment-trapping-process model of VFS (STPMOD-VFS) was proposed. Then, the sediment trapping processes of three experiments under different conditions were predicted using the STPMOD-VFS, and the prediction was considered highly satisfactory. The uncertainty estimation results showed the STPMOD-VFS had significant 'equifinality' performance. The sensitivity of different parameters of the STPMOD-VFS was found significantly different. The parameters related to the sediment delivery rate of silt-laden inflow and those that control the attenuation coefficients of the instantaneous sediment trapping efficiency of the VFS were sensitive, while those parameters that control the initial sediment outflow rate were less sensitive. Finally, an equation describing the sediment trapping process by a VFS under a dynamic sediment inflow rate was built and solved, and a general expression for the VFS sediment trapping time-varying process was given. The findings of this study could help in evaluation of sediment processes on grassed hillslopes.

Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation

Although having achieved great success in medical image segmentation, deep learning-based approaches usually require large amounts of well-annotated data, which can be extremely expensive in the field of medical image analysis. Unlabeled data, on the other hand, is much easier to acquire. Semi-supervised learning and unsupervised domain adaptation both take the advantage of unlabeled data, and they are closely related to each other. In this paper, we propose uncertainty-aware multi-view co-training (UMCT), a unified framework that addresses these two tasks for volumetric medical image segmentation. Our framework is capable of efficiently utilizing unlabeled data for better performance. We firstly rotate and permute the 3D volumes into multiple views and train a 3D deep network on each view. We then apply co-training by enforcing multi-view consistency on unlabeled data, where an uncertainty estimation of each view is utilized to achieve accurate labeling. Experiments on the NIH pancreas segmentation dataset and a multi-organ segmentation dataset show state-of-the-art performance of the proposed framework on semi-supervised medical image segmentation. Under unsupervised domain adaptation settings, we validate the effectiveness of this work by adapting our multi-organ segmentation model to two pathological organs from the Medical Segmentation Decathlon Datasets. Additionally, we show that our UMCT-DA model can even effectively handle the challenging situation where labeled source data is inaccessible, demonstrating strong potentials for real-world applications.

Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks

Despite the state-of-the-art performance for medical image segmentation, deep convolutional neural networks (CNNs) have rarely provided uncertainty estimations regarding their segmentation outputs, e.g., model (epistemic) and image-based (aleatoric) uncertainties. In this work, we analyze these different types of uncertainties for CNN-based 2D and 3D medical image segmentation tasks at both pixel level and structure level. We additionally propose a test-time augmentation-based aleatoric uncertainty to analyze the effect of different transformations of the input image on the segmentation output. Test-time augmentation has been previously used to improve segmentation accuracy, yet not been formulated in a consistent mathematical framework. Hence, we also propose a theoretical formulation of test-time augmentation, where a distribution of the prediction is estimated by Monte Carlo simulation with prior distributions of parameters in an image acquisition model that involves image transformations and noise. We compare and combine our proposed aleatoric uncertainty with model uncertainty. Experiments with segmentation of fetal brains and brain tumors from 2D and 3D Magnetic Resonance Images (MRI) showed that 1) the test-time augmentation-based aleatoric uncertainty provides a better uncertainty estimation than calculating the test-time dropout-based model uncertainty alone and helps to reduce overconfident incorrect predictions, and 2) our test-time augmentation outperforms a single-prediction baseline and dropout-based multiple predictions.

Investigation of thermal conversion characteristics and performance evaluation of co-combustion of pine sawdust and lignite coal using TGA, artificial neural network modeling and likelihood method

(Co-)combustion of pine sawdust (PS) and lignite coal (LC) were investigated using artificial neural networks (ANN), particle swarm optimization (PSO), and Monte Carlo simulation (MC) as a function of blend ratio, heating rate, and temperature via thermal conversion characteristics. The order of degraded compounds in terms of hemi-cellulosic and lignin-based compounds demonstrated the main oxidation and degradation mechanism of co-combustion of PS and LC. The best prediction (R² of 99.99%) was obtained by ANN28 model. Operating conditions of 90LC10PS, 425 °C, and 19 °C min^-1 were determined by PSO as optimum levels with TG value of 67.5%. Once three-replicated validation experiments were performed under PSO-optimized conditions, mean TG values ware observed as 67.5% with a standard deviation of ±0.4%. Consequently, MC was used to identify the stochastic variability and uncertainty associated with ANN models that were derived to predict TG values.

Quantitative error prediction of medical image registration using regression forests

Predicting registration error can be useful for evaluation of registration procedures, which is important for the adoption of registration techniques in the clinic. In addition, quantitative error prediction can be helpful in improving the registration quality. The task of predicting registration error is demanding due to the lack of a ground truth in medical images. This paper proposes a new automatic method to predict the registration error in a quantitative manner, and is applied to chest CT scans. A random regression forest is utilized to predict the registration error locally. The forest is built with features related to the transformation model and features related to the dissimilarity after registration. The forest is trained and tested using manually annotated corresponding points between pairs of chest CT scans in two experiments: SPREAD (trained and tested on SPREAD) and inter-database (including three databases SPREAD, DIR-Lab-4DCT and DIR-Lab-COPDgene). The results show that the mean absolute errors of regression are 1.07  ±  1.86 and 1.76  ±  2.59 mm for the SPREAD and inter-database experiment, respectively. The overall accuracy of classification in three classes (correct, poor and wrong registration) is 90.7% and 75.4%, for SPREAD and inter-database respectively. The good performance of the proposed method enables important applications such as automatic quality control in large-scale image analysis.

Validation and uncertainty estimation for trace amounts determination of 25 drugs used in hospital chemotherapy compounding units

The validation and uncertainty assessment of the analytical method developed for the simultaneous determination of 25 anticancer drugs commonly handled in hospital pharmacy was reported. Selected compounds were 5-fluorouracil, cytarabine, fludarabine phosphate, ganciclovir, gemcitabine, dacarbazine, methotrexate, pemetrexed, busulfan, raltitrexed, etoposide phosphate, topotecan, ifosfamide, cyclophosphamide, irinotecan, doxorubicin, epirubicin, daunorubicin, idarubicin, vincristine, vinblastine, vinorelbine, docetaxel and paclitaxel. Accuracy and uncertainty profiles were obtained for all compounds. Quantitative performances were satisfactory in term of specificity, sensitivity, precision and accuracy. Repeatability (1.9-25.4%) and intermediate precision (2.7-29%) were determined for all target compounds. Lower limits of quantification between 1 and 25 ng/mL were obtained. Uncertainty associated to measurement of routine samples was evaluated. The multi-targeted method was specific and reliable and was successfully applied to wipe samples from hospital pharmacy chemotherapy compounding unit and to the determination of outside contamination of vials from pharmaceutical manufacturers.

Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks

PURPOSE

Optical imaging is evolving as a key technique for advanced sensing in the operating room. Recent research has shown that machine learning algorithms can be used to address the inverse problem of converting pixel-wise multispectral reflectance measurements to underlying tissue parameters, such as oxygenation. Assessment of the specific hardware used in conjunction with such algorithms, however, has not properly addressed the possibility that the problem may be ill-posed.

METHODS

We present a novel approach to the assessment of optical imaging modalities, which is sensitive to the different types of uncertainties that may occur when inferring tissue parameters. Based on the concept of invertible neural networks, our framework goes beyond point estimates and maps each multispectral measurement to a full posterior probability distribution which is capable of representing ambiguity in the solution via multiple modes. Performance metrics for a hardware setup can then be computed from the characteristics of the posteriors.

RESULTS

Application of the assessment framework to the specific use case of camera selection for physiological parameter estimation yields the following insights: (1) estimation of tissue oxygenation from multispectral images is a well-posed problem, while (2) blood volume fraction may not be recovered without ambiguity. (3) In general, ambiguity may be reduced by increasing the number of spectral bands in the camera.

CONCLUSION

Our method could help to optimize optical camera design in an application-specific manner.

Classification of samples from NMR-based metabolomics using principal components analysis and partial least squares with uncertainty estimation

Recent progress in metabolomics has been aided by the development of analysis techniques such as gas and liquid chromatography coupled with mass spectrometry (GC-MS and LC-MS) and nuclear magnetic resonance (NMR) spectroscopy. The vast quantities of data produced by these techniques has resulted in an increase in the use of machine algorithms that can aid in the interpretation of this data, such as principal components analysis (PCA) and partial least squares (PLS). Techniques such as these can be applied to biomarker discovery, interlaboratory comparison, and clinical diagnoses. However, there is a lingering question whether the results of these studies can be applied to broader sets of clinical data, usually taken from different data sources. In this work, we address this question by creating a metabolomics workflow that combines a previously published consensus analysis procedure ( https://doi.org/10.1016/j.chemolab.2016.12.010 ) with PCA and PLS models using uncertainty analysis based on bootstrapping. This workflow is applied to NMR data that come from an interlaboratory comparison study using synthetic and biologically obtained metabolite mixtures. The consensus analysis identifies trusted laboratories, whose data are used to create classification models that are more reliable than without. With uncertainty analysis, the reliability of the classification can be rigorously quantified, both for data from the original set and from new data that the model is analyzing. Graphical abstract ᅟ.

Determination of cadmium in sediments by diluted HCI extraction and isotope dilution ICP-MS

Isotope dilution ICP-MS is proposed to measure the mass fraction of Cd extracted by diluted HCl in marine sediments, using a fast and simple extraction procedure based on ultrasonic probe agitation. The ¹¹¹Cd isotope was added before the extraction to achieve isotope equilibration with native Cd solubilized from the sample. The parameters affecting trueness and precision of isotope ratio measurements were evaluated carefully and subsequently corrected in order to minimize errors; they were: detector dead time, spectral interferences, mass discrimination factor and optimum sample/spike ratio. The mass fraction of Cd extracted was compared with the sum of the certified contents of the three steps of the sequential extraction procedure of the Standards, Measurements and Testing Programme (SM&T) analysing the BCR 701 sediment to validate the method. The certified and measured values agreed, giving a measured / certified mass fraction ratio of 1.05. Further, the extraction procedure itself was studied by adding the enriched isotope after the extraction step, which allowed verifying that analyte losses occurred during this process. Two additional reference sediments with certified total cadmium contents were also analysed. The method provided very good precision (0.9%, RSD) and a low detection limit, 1.8 ng g^-1. The procedural uncertainty budget was estimated following the EURACHEM Guide by means of the 'GUM Workbench' software, obtaining a relative expanded uncertainty of 1.5%. The procedure was applied to determine the bioaccessible mass fraction of Cd in sediments from two environmentally and economically important areas of Galicia (rias of Arousa and Vigo, NW of Spain).

Simultaneous determination of acrylamide and 4-hydroxy-2,5-dimethyl-3(2H)-furanone in baby food by liquid chromatography-tandem mass spectrometry

A liquid chromatography triple quadrupole mass spectrometry method was developed and validated for the simultaneous determination of acrylamide and 4-hydroxy-2,5-dimethyl-3(2H)-furanone (HDMF) in baby food. The sample preparation involves acetonitrile-based extraction combined with dispersive primary secondary amine (PSA) cleanup and cation-exchange solid-phase extraction (SPE), which promotes efficient removal of matrix interferences. Analytical selectivity and sensitivity were achieved for the quantification of acrylamide and HDMF in complex matrices such as fruit, cereal and milk-based baby foods; furthermore, adequate linearity (range 10-300μgkg^-1) in solvent and matrix-matched calibration curves, and appropriate recoveries (94-110%) and precision (RSD≤10%), under repeatability and within-laboratory reproducibility conditions, were also obtained. Expanded measurement uncertainty was estimated at the 20μgkg^-1 level (limit of quantification) on the basis of data obtained from in-house validation, with values of 25.5 and 16.5% for acrylamide and HDMF, respectively. The fitness for purpose of developed method was verified by analyzing 15 commercial baby foods available in the Brazilian market. Acrylamide was detected in one plum-based baby food (35μgkg^-1) while HDMF in 67% of the samples analyzed (levels between 25 and 262μgkg^-1).

Calculation spreadsheet for uncertainty estimation of measurement results in gamma-ray spectrometry and its validation for quality assurance purpose

An Excel calculation spreadsheet has been developed to estimate the uncertainty of measurement results in γ-ray spectrometry. It considers all relevant uncertainty components and calculates the combined standard uncertainty of the measurement result. The calculation spreadsheet has been validated using two independent open access software and is available for download free of charge at: https://nucleus.iaea.org/rpst/ReferenceProducts/Analytical_Methods/index.htm. It provides a simple and easy-to-use template for estimating the uncertainty of γ-ray spectrometry measurement results and supports the radioanalytical laboratories seeking accreditation for their measurements using γ-ray spectrometry.

Bayesian Monte Carlo and maximum likelihood approach for uncertainty estimation and risk management: Application to lake oxygen recovery model

Model uncertainty estimation and risk assessment is essential to environmental management and informed decision making on pollution mitigation strategies. In this study, we apply a probabilistic methodology, which combines Bayesian Monte Carlo simulation and Maximum Likelihood estimation (BMCML) to calibrate a lake oxygen recovery model. We first derive an analytical solution of the differential equation governing lake-averaged oxygen dynamics as a function of time-variable wind speed. Statistical inferences on model parameters and predictive uncertainty are then drawn by Bayesian conditioning of the analytical solution on observed daily wind speed and oxygen concentration data obtained from an earlier study during two recovery periods on a eutrophic lake in upper state New York. The model is calibrated using oxygen recovery data for one year and statistical inferences were validated using recovery data for another year. Compared with essentially two-step, regression and optimization approach, the BMCML results are more comprehensive and performed relatively better in predicting the observed temporal dissolved oxygen levels (DO) in the lake. BMCML also produced comparable calibration and validation results with those obtained using popular Markov Chain Monte Carlo technique (MCMC) and is computationally simpler and easier to implement than the MCMC. Next, using the calibrated model, we derive an optimal relationship between liquid film-transfer coefficient for oxygen and wind speed and associated 95% confidence band, which are shown to be consistent with reported measured values at five different lakes. Finally, we illustrate the robustness of the BMCML to solve risk-based water quality management problems, showing that neglecting cross-correlations between parameters could lead to improper required BOD load reduction to achieve the compliance criteria of 5 mg/L.

Robust fractional order sliding mode control of doubly-fed induction generator (DFIG)-based wind turbines

Wind power plants have nonlinear dynamics and contain many uncertainties such as unknown nonlinear disturbances and parameter uncertainties. Thus, it is a difficult task to design a robust reliable controller for this system. This paper proposes a novel robust fractional-order sliding mode (FOSM) controller for maximum power point tracking (MPPT) control of doubly fed induction generator (DFIG)-based wind energy conversion system. In order to enhance the robustness of the control system, uncertainties and disturbances are estimated using a fractional order uncertainty estimator. In the proposed method a continuous control strategy is developed to achieve the chattering free fractional order sliding-mode control, and also no knowledge of the uncertainties and disturbances or their bound is assumed. The boundedness and convergence properties of the closed-loop signals are proven using Lyapunov׳s stability theory. Simulation results in the presence of various uncertainties were carried out to evaluate the effectiveness and robustness of the proposed control scheme.

Assessment of uncertainty in constructed wetland treatment performance and load estimation methods

Constructed wetlands (CWs) are commonly established to reduce pollution load from different sources. In environmental permits, the load remaining after CW purification is typically estimated through concentration and flow measurements. This load monitoring is often carried out using long water quality sampling intervals, which causes uncertainty in load estimation. In this study, a large suspended solids (SSs) and dissolved organic carbon (DOC) dataset was used to quantify the uncertainty in load estimation at the inlet and outlet of a CW with different sampling frequencies (sampling every 1, 2, 3 or 4 weeks). A method to reduce the uncertainty by dividing the CW flow duration curve (FDC) into four equal categories and assigning mean/median concentration for each category according to the measured concentrations was also tested. The results showed that estimated SS load was associated with considerable uncertainty and that this uncertainty increased with lower sampling frequency. The FDC method was able to decrease the uncertainty, but much still remained, especially when concentrations of the measured variable showed great variation. In such cases, sensor technology might be a feasible option for further reducing the uncertainty.

Assessment of measurement result uncertainty in determination of (210)Pb with the focus on matrix composition effect in gamma-ray spectrometry

Reference materials were used to assess measurement result uncertainty in determination of (210)Pb by gamma-ray spectrometry, liquid scintillation counting, or indirectly by alpha-particle spectrometry, using its daughter (210)Po in radioactive equilibrium. Combined standard uncertainties of (210)Pb massic activities obtained by liquid scintillation counting are in the range 2-12%, depending on matrices and massic activity values. They are in the range 1-3% for the measurement of its daughter (210)Po using alpha-particle spectrometry. Three approaches (direct computation of counting efficiency and efficiency transfer approaches based on the computation and, respectively, experimental determination of the efficiency transfer factors) were applied for the evaluation of (210)Pb using gamma-ray spectrometry. Combined standard uncertainties of gamma-ray spectrometry results were found in the range 2-17%. The effect of matrix composition on self-attenuation was investigated and a detailed assessment of uncertainty components was performed.

Rapid determination of residues of pesticides in honey by µGC-ECD and GC-MS/MS: Method validation and estimation of measurement uncertainty according to document No. SANCO/12571/2013

A simple and straightforward method for simultaneous determination of residues of 13 pesticides in honey samples (acrinathrin, bifenthrin, bromopropylate, cyhalothrin-lambda, cypermethrin, chlorfenvinphos, chlorpyrifos, coumaphos, deltamethrin, fluvalinate-tau, malathion, permethrin and tetradifon) from different pesticide classes has been developed and validated. The analytical method provides dissolution of honey in water and an extraction of pesticide residues by n-Hexane followed by clean-up on a Florisil SPE column. The extract was evaporated and taken up by a solution of an injection internal standard (I-IS), ethion, and finally analyzed by capillary gas chromatography with electron capture detection (GC-µECD). Identification for qualitative purpose was conducted by gas chromatography with triple quadrupole mass spectrometer (GC-MS/MS). A matrix-matched calibration curve was performed for quantitative purposes by plotting the area ratio (analyte/I-IS) against concentration using a GC-µECD instrument. According to document No. SANCO/12571/2013, the method was validated by testing the following parameters: linearity, matrix effect, specificity, precision, trueness (bias) and measurement uncertainty. The analytical process was validated analyzing blank honey samples spiked at levels equal to and greater than 0.010 mg/kg (limit of quantification). All parameters were satisfactorily compared with the values established by document No. SANCO/12571/2013. The analytical performance was verified by participating in eight multi-residue proficiency tests organized by BIPEA, obtaining satisfactory z-scores in all 70 determinations. Measurement uncertainty was estimated according to the top-down approaches described in Appendix C of the SANCO document using the within-laboratory reproducibility relative standard deviation combined with laboratory bias using the proficiency test data.

Contamination of the southern Baltic Sea waters by the residues of selected pharmaceuticals: Method development and field studies

In this study the occurrence of thirteen pharmaceuticals in seawaters collected from southern Baltic Sea was evaluated for the first time. It was performed by applying newly developed analytical procedure. The method was characterized in terms of its basic validation parameters as well as matrix effects, extraction efficiency and absolute recovery. The results were expressed as result ± expanded uncertainty, which was estimated according to the Guide to the Expression of Uncertainty in Measurement. Additionally, in order to verify the influence of variable parameters of the analyzed samples on method performance parameters, chemometric analysis was carried out. The obtained results revealed that residues of pharmaceuticals were present in seawaters at a concentration level of ng L(-1). Trimethoprim, sulfamethoxazole and enrofloxacin were most frequently detected compounds. The highest concentration was determined for ketoprofen (135.0 ± 10.9 ng L(-1)). Marine pollution potential hotspots were found in enclosed or semi-enclosed bodies of water.