"How much realism is needed?" - the wrong question in silico imagers have been asking

Observer-study-based approaches to quantitatively evaluate the realism of synthetic medical images

Objective.Synthetic images generated by simulation studies have a well-recognized role in developing and evaluating imaging systems and methods. However, for clinically relevant development and evaluation, the synthetic images must be clinically realistic and, ideally, have the same distribution as that of clinical images. Thus, mechanisms that can quantitatively evaluate this clinical realism and, ideally, the similarity in distributions of the real and synthetic images, are much needed.Approach.We investigated two observer-study-based approaches to quantitatively evaluate the clinical realism of synthetic images. In the first approach, we presented a theoretical formalism for the use of an ideal-observer study to quantitatively evaluate the similarity in distributions between the real and synthetic images. This theoretical formalism provides a direct relationship between the area under the receiver operating characteristic curve, AUC, for an ideal observer and the distributions of real and synthetic images. The second approach is based on the use of expert-human-observer studies to quantitatively evaluate the realism of synthetic images. In this approach, we developed a web-based software to conduct two-alternative forced-choice (2-AFC) experiments with expert human observers. The usability of this software was evaluated by conducting a system usability scale (SUS) survey with seven expert human readers and five observer-study designers. Further, we demonstrated the application of this software to evaluate a stochastic and physics-based image-synthesis technique for oncologic positron emission tomography (PET). In this evaluation, the 2-AFC study with our software was performed by six expert human readers, who were highly experienced in reading PET scans, with years of expertise ranging from 7 to 40 years (median: 12 years, average: 20.4 years).Main results.In the ideal-observer-study-based approach, we theoretically demonstrated that the AUC for an ideal observer can be expressed, to an excellent approximation, by the Bhattacharyya distance between the distributions of the real and synthetic images. This relationship shows that a decrease in the ideal-observer AUC indicates a decrease in the distance between the two image distributions. Moreover, a lower bound of ideal-observer AUC = 0.5 implies that the distributions of synthetic and real images exactly match. For the expert-human-observer-study-based approach, our software for performing the 2-AFC experiments is available athttps://apps.mir.wustl.edu/twoafc. Results from the SUS survey demonstrate that the web application is very user friendly and accessible. As a secondary finding, evaluation of a stochastic and physics-based PET image-synthesis technique using our software showed that expert human readers had limited ability to distinguish the real images from the synthetic images.Significance.This work addresses the important need for mechanisms to quantitatively evaluate the clinical realism of synthetic images. The mathematical treatment in this paper shows that quantifying the similarity in the distribution of real and synthetic images is theoretically possible by using an ideal-observer-study-based approach. Our developed software provides a platform for designing and performing 2-AFC experiments with human observers in a highly accessible, efficient, and secure manner. Additionally, our results on the evaluation of the stochastic and physics-based image-synthesis technique motivate the application of this technique to develop and evaluate a wide array of PET imaging methods.

Performance evaluation of digital breast tomosynthesis systems: physical methods and experimental data

Digital breast tomosynthesis (DBT) has become a well-established breast imaging technique, whose performance has been investigated in many clinical studies, including a number of prospective clinical trials. Results from these studies generally point to non-inferiority in terms of microcalcification detection and superior mass-lesion detection for DBT imaging compared to digital mammography (DM). This modality has become an essential tool in the clinic for assessment and ad-hoc screening but is not yet implemented in most breast screening programmes at a state or national level. While evidence on the clinical utility of DBT has been accumulating, there has also been progress in the development of methods for technical performance assessment and quality control of these imaging systems. DBT is a relatively complicated ‘pseudo-3D’ modality whose technical assessment poses a number of difficulties. This paper reviews methods for the technical performance assessment of DBT devices, starting at the component level in part one and leading up to discussion of system evaluation with physical test objects in part two. We provide some historical and basic theoretical perspective, often starting from methods developed for DM imaging. Data from a multi-vendor comparison are also included, acquired under the medical physics quality control protocol developed by EUREF and currently being consolidated by a European Federation of Organisations for Medical Physics working group. These data and associated methods can serve as a reference for the development of reference data and provide some context for clinical studies.

Advances in Computational Human Phantoms and Their Applications in Biomedical Engineering - A Topical Review

Over the past decades, significant improvements have been made in the field of computational human phantoms (CHPs) and their applications in biomedical engineering. Their sophistication has dramatically increased. The very first CHPs were composed of simple geometric volumes, e.g., cylinders and spheres, while current CHPs have a high resolution, cover a substantial range of the patient population, have high anatomical accuracy, are poseable, morphable, and are augmented with various details to perform functionalized computations. Advances in imaging techniques and semi-automated segmentation tools allow fast and personalized development of CHPs. These advances open the door to quickly develop personalized CHPs, inherently including the disease of the patient. Because many of these CHPs are increasingly providing data for regulatory submissions of various medical devices, the validity, anatomical accuracy, and availability to cover the entire patient population is of utmost importance. The article is organized into two main sections: the first section reviews the different modeling techniques used to create CHPs, whereas the second section discusses various applications of CHPs in biomedical engineering. Each topic gives an overview, a brief history, recent developments, and an outlook into the future.

Organ doses evaluation for chest computed tomography procedures with TL dosimeters: Comparison with Monte Carlo simulations

Purpose

To evaluate organ doses in routine and low‐dose chest computed tomography (CT) protocols using an experimental methodology. To compare experimental results with results obtained by the National Cancer Institute dosimetry system for CT (NCICT) organ dose calculator. To address the differences on organ dose measurements using tube current modulation (TCM) and fixed tube current protocols.

Methods

An experimental approach to evaluate organ doses in pediatric and adult anthropomorphic phantoms using thermoluminescent dosimeters (TLDs) was employed in this study. Several analyses were performed in order to establish the best way to achieve the main results in this investigation. The protocols used in this study were selected after an analysis of patient data collected from the Institute of Radiology of the School of Medicine of the University of São Paulo (InRad). The image quality was evaluated by a radiologist from this institution. Six chest adult protocols and four chest pediatric protocols were evaluated. Lung doses were evaluated for the adult phantom and lung and thyroid doses were evaluated for the pediatric phantom. The irradiations were performed using both a GE and a Philips CT scanner. Finally, organ doses measured with dosimeters were compared with Monte Carlo simulations performed with NCICT.

Results

After analyzing the data collected from all CT examinations performed during a period of 3 yr, the authors identified that adult and pediatric chest CT are among the most applied protocol in patients in that clinical institution, demonstrating the relevance on evaluating organ doses due to these examinations. With regards to the scan parameters adopted, the authors identified that using 80 kV instead of 120 kV for a pediatric chest routine CT, with TCM in both situations, can lead up to a 28.7% decrease on the absorbed dose. Moreover, in comparison to the standard adult protocol, which is performed with fixed mAs, TCM, and ultra low‐dose protocols resulted in dose reductions of up to 35.0% and 90.0%, respectively. Finally, the percent differences found between experimental and Monte Carlo simulated organ doses were within a 20% interval.

Conclusions

The results obtained in this study measured the impact on the absorbed dose in routine chest CT by changing several scan parameters while the image quality could be potentially preserved.