A Hybrid Approach for Approximating the Ideal Observer for Joint Signal Detection and Estimation Tasks by Use of Supervised Learning and Markov-Chain Monte Carlo Methods

The ideal observer (IO) sets an upper performance limit among all observers and has been advocated for assessing and optimizing imaging systems. For general joint detection and estimation (detection-estimation) tasks, estimation ROC (EROC) analysis has been established for evaluating the performance of observers. However, in general, it is difficult to accurately approximate the IO that maximizes the area under the EROC curve. In this study, a hybrid method that employs machine learning is proposed to accomplish this. Specifically, a hybrid approach is developed that combines a multi-task convolutional neural network and a Markov-Chain Monte Carlo (MCMC) method in order to approximate the IO for detection-estimation tasks. Unlike traditional MCMC methods, the hybrid method is not limited to use of specific utility functions. In addition, a purely supervised learning-based sub-ideal observer is proposed. Computer-simulation studies are conducted to validate the proposed method, which include signal-known-statistically/background-known-exactly and signal-known-statistically/background-known-statistically tasks. The EROC curves produced by the proposed method are compared to those produced by the MCMC approach or analytical computation when feasible. The proposed method provides a new approach for approximating the IO and may advance the application of EROC analysis for optimizing imaging systems.

Risk analysis, ideal observers, and receiver operating characteristic curves for tasks that combine detection and estimation

Previously published work on joint estimation/detection tasks has focused on the area under the estimation receiver operating characteristic (EROC) curve as a figure of merit (FOM) for these tasks in imaging. Another FOM for these joint tasks is the Bayesian risk, where a cost is assigned to all detection outcomes and to the estimation errors, and then averaged over all sources of randomness in the object ensemble and the imaging system. Important elements of the cost function, which are not included in standard EROC analysis, are that the cost for a false positive depends on the estimate produced for the parameter vector, and the cost for a false negative depends on the true value of the parameter vector. The ideal observer in this setting, which minimizes the risk, is derived for two applications. In the first application, a parameter vector is estimated only in the case of a signal present classification. For the second application, parameter vectors are estimated for either classification, and these vectors may have different dimensions. In both applications, a risk-based estimation receiver operating characteristic curve is defined and an expression for the area under this curve is given. It is also shown that, for some observers, this area may be estimated from a two alternative forced choice test. Finally, if the classifier is optimized for a given estimator, then it is shown that the slope of the risk-based estimation receiver operating characteristic curve at each point is the negative of the ratio of the prior probabilities for the two classes.

Optimal Joint Detection and Estimation That Maximizes ROC-Type Curves

Combined detection-estimation tasks are frequently encountered in medical imaging. Optimal methods for joint detection and estimation are of interest because they provide upper bounds on observer performance, and can potentially be utilized for imaging system optimization, evaluation of observer efficiency, and development of image formation algorithms. We present a unified Bayesian framework for decision rules that maximize receiver operating characteristic (ROC)-type summary curves, including ROC, localization ROC (LROC), estimation ROC (EROC), free-response ROC (FROC), alternative free-response ROC (AFROC), and exponentially-transformed FROC (EFROC) curves, succinctly summarizing previous results. The approach relies on an interpretation of ROC-type summary curves as plots of an expected utility versus an expected disutility (or penalty) for signal-present decisions. We propose a general utility structure that is flexible enough to encompass many ROC variants and yet sufficiently constrained to allow derivation of a linear expected utility equation that is similar to that for simple binary detection. We illustrate our theory with an example comparing decision strategies for joint detection-estimation of a known signal with unknown amplitude. In addition, building on insights from our utility framework, we propose new ROC-type summary curves and associated optimal decision rules for joint detection-estimation tasks with an unknown, potentially-multiple, number of signals in each observation.