Benchmarking of fluorescence cameras through the use of a composite phantom

AAPM Task Group Report 311: Guidance for performance evaluation of fluorescence-guided surgery systems

The last decade has seen a large growth in fluorescence-guided surgery (FGS) imaging and interventions. With the increasing number of clinical specialties implementing FGS, the range of systems with radically different physical designs, image processing approaches, and performance requirements is expanding. This variety of systems makes it nearly impossible to specify uniform performance goals, yet at the same time, utilization of different devices in new clinical procedures and trials indicates some need for common knowledge bases and a quality assessment paradigm to ensure that effective translation and use occurs. It is feasible to identify key fundamental image quality characteristics and corresponding objective test methods that should be determined such that there are consistent conventions across a variety of FGS devices. This report outlines test methods, tissue simulating phantoms and suggested guidelines, as well as personnel needs and professional knowledge bases that can be established. This report frames the issues with guidance and feedback from related societies and agencies having vested interest in the outcome, coming from an independent scientific group formed from academics and international federal agencies for the establishment of these professional guidelines.

Impact of Surgical Lights on the Performance of Fluorescence-Guided Surgery Systems: A Pilot Study

Fluorescence-guided surgery can aid in the intraoperative visualization of target tissues, with promising applications in human and veterinary surgical oncology. The aim of this study was to evaluate the performances of two fluoresce camera systems, IC-Flow^TM and Visionsense^TM VS3 Iridum, for the detection of two non-targeted (ICG and IRDye-800) and two targeted fluorophores (Angiostamp^TM and FAP-Cyan) under different room light conditions, including ambient light, new generation LED, and halogen artificial light sources, which are commonly used in operating theaters. Six dilutions of the fluorophores were imaged in phantom kits using the two camera systems. The limit of detection (LOD) and mean signal-to-background ratio (mSBR) were determined. The highest values of mSBR and a lower LOD were obtained in dark conditions for both systems. Under room lights, the capabilities decreased, but the mSBR remained greater than 3 (=clearly detectable signal). LOD and mSBR worsened under surgical lights for both camera systems, with a greater impact from halogen bulbs on Visionsense^TM VS3 Iridium and of the LED lights on IC-Flow due to a contribution of these lights in the near-infrared spectrum. When considering implementing FGS into the clinical routine, surgeons should cautiously evaluate the spectral contribution of the lights in the operating theater.

3D-Printed Tumor Phantoms for Assessment of In Vivo Fluorescence Imaging Analysis Methods

PURPOSE

Interventional fluorescence imaging is increasingly being utilized to quantify cancer biomarkers in both clinical and preclinical models, yet absolute quantification is complicated by many factors. The use of optical phantoms has been suggested by multiple professional organizations for quantitative performance assessment of fluorescence guidance imaging systems. This concept can be further extended to provide standardized tools to compare and assess image analysis metrics.

PROCEDURES

3D-printed fluorescence phantoms based on solid tumor models were developed with representative bio-mimicking optical properties. Phantoms were produced with discrete tumors embedded with an NIR fluorophore of fixed concentration and either zero or 3% non-specific fluorophore in the surrounding material. These phantoms were first imaged by two fluorescence imaging systems using two methods of image segmentation, and four assessment metrics were calculated to demonstrate variability in the quantitative assessment of system performance. The same analysis techniques were then applied to one tumor model with decreasing tumor fluorophore concentrations.

RESULTS

These anatomical phantom models demonstrate the ability to use 3D printing to manufacture anthropomorphic shapes with a wide range of reduced scattering (μ_s': 0.24-1.06 mm^-1) and absorption (μ_a: 0.005-0.14 mm^-1) properties. The phantom imaging and analysis highlight variability in the measured sensitivity metrics associated with tumor visualization.

CONCLUSIONS

3D printing techniques provide a platform for demonstrating complex biological models that introduce real-world complexities for quantifying fluorescence image data. Controlled iterative development of these phantom designs can be used as a tool to advance the field and provide context for consensus-building beyond performance assessment of fluorescence imaging platforms, and extend support for standardizing how quantitative metrics are extracted from imaging data and reported in literature.

Current and Future Applications of Fluorescence Guidance in Orthopaedic Surgery

Fluorescence-guided surgery (FGS) is an evolving field that seeks to identify important anatomic structures or physiologic phenomena with helpful relevance to the execution of surgical procedures. Fluorescence labeling occurs generally via the administration of fluorescent reporters that may be molecularly targeted, enzyme-activated, or untargeted, vascular probes. Fluorescence guidance has substantially changed care strategies in numerous surgical fields; however, investigation and adoption in orthopaedic surgery have lagged. FGS shows the potential for improving patient care in orthopaedics via several applications including disease diagnosis, perfusion-based tissue healing capacity assessment, infection/tumor eradication, and anatomic structure identification. This review highlights current and future applications of fluorescence guidance in orthopaedics and identifies key challenges to translation and potential solutions.

Performance of two clinical fluorescence imaging systems with different targeted and non-targeted near-infrared fluorophores: a cadaveric explorative study

Introduction

Near-infrared (NIR) fluorescence-guided surgery is increasingly utilized in humans and pets. As clinical imaging systems are optimized for Indocyanine green (ICG) detection, the usage of targeted dyes necessitates the validation of these systems for each dye. We investigated the impact of skin pigmentation and tissue overlay on the sensitivity of two NIR cameras (IC-Flow^TM, Visionsense^TM VS3 Iridum) for the detection of non-targeted (ICG, IRDye800) and targeted (Angiostamp^TM, FAP-Cyan) NIR fluorophores in an ex vivo big animal model.

Methods

We quantitatively measured the limit of detection (LOD) and signal-to-background ratio (SBR) and implemented a semi-quantitative visual score to account for subjective interpretation of images by the surgeon.

Results

Visionsense^TM VS3 Iridum outperformed IC-Flow^TM in terms of LOD and SBR for the detection of all dyes except FAP-Cyan. Median SBR was negatively affected by skin pigmentation and tissue overlay with both camera systems. Level of agreement between quantitative and semi-quantitative visual score and interobserver agreement were better with Visionsense^TM VS3 Iridum.

Conclusion

The overlay of different tissue types and skin pigmentation may negatively affect the ability of the two tested camera systems to identify nanomolar concentrations of targeted-fluorescent dyes and should be considered when planning surgical applications.

Quantitative tumor depth determination using dual wavelength excitation fluorescence

Quantifying solid tumor margins with fluorescence-guided surgery approaches is a challenge, particularly when using near infrared (NIR) wavelengths due to increased penetration depths. An NIR dual wavelength excitation fluorescence (DWEF) approach was developed that capitalizes on the wavelength-dependent attenuation of light in tissue to determine fluorophore depth. A portable dual wavelength excitation fluorescence imaging system was built and tested in parallel with an NIR tumor-targeting fluorophore in tissue mimicking phantoms, chicken tissue, and in vivo mouse models of breast cancer. The system showed high accuracy in all experiments. The low cost and simplicity of this approach make it ideal for clinical use.

Development of a microstructured tissue phantom with adaptable optical properties for use with microscopes and fluorescence lifetime imaging systems

OBJECTIVES

For the development and validation of diagnostic procedures based on microscopic methods, knowledge about the imaging depth and achievable resolution in tissue is crucial. This poses the challenge to develop a microscopic artificial phantom focused on the microscopic instead of the macroscopic optical tissue characteristics.

METHODS

As existing artificial tissue phantoms designed for image forming systems are primarily targeted at wide field applications, they are unsuited for reaching the formulated objective. Therefore, a microscopy- and microendoscopy-suited artificial tissue phantom was developed and characterized. It is based on a microstructured glass surface coated with fluorescent beads at known depths covered by a scattering agent with modifiable optical properties. The phantom was examined with different kinds of microscopy systems in order to characterize its quality and stability and to demonstrate its usefulness for instrument comparison, for example, regarding structural as well as fluorescence lifetime analysis.

RESULTS

The analysis of the manufactured microstructured glass surfaces showed high regularity in their physical dimensions in accordance with the specifications. Measurements of the optical parameters of the scattering medium were consistent with simulations. The fluorescent beads coating proved to be stable for a respectable period of time (about a week). The developed artificial tissue phantom was successfully used to detect differences in image quality between a research microscope and an endoscopy based system. Plausible causes for the observed differences could be derived based on the well known microstructure of the phantom.

CONCLUSIONS

The artificial tissue phantom is well suited for the intended use with microscopic and microendoscopic systems. Due to its configurable design, it can be adapted to a wide range of applications. It is especially targeted at the characterization and calibration of clinical imaging systems that often lack extensive positioning capabilities such as an intrinsic z-stage.

Conversion of imager-specific response to tissue phantom fluorescence into system of units-traceable units

SIGNIFICANCE

The fluorescence-guided imaging for surgical intervention community recognizes the need for performance standards for these imaging devices. Tissue phantoms are used to track an imager's performance as a fluorescence detector, but imager-specific units are of limited utility.

AIM

Tissue phantoms can be calibrated to be traceable to the international system of units (SI) and in turn be used to calibrate imagers such that fluorescence measurements can be reported in universally accepted units.

APPROACH

The radiometry to convert imager-specific arbitrary digital counts to SI-traceable unit of watts is described in this paper.

RESULTS

An example of an imager calibration is included.

CONCLUSIONS

Calibrated tissue phantoms become a tool for metrological traceability.

Criteria for the design of tissue-mimicking phantoms for the standardization of biophotonic instrumentation

A lack of accepted standards and standardized phantoms suitable for the technical validation of biophotonic instrumentation hinders the reliability and reproducibility of its experimental outputs. In this Perspective, we discuss general criteria for the design of tissue-mimicking biophotonic phantoms, and use these criteria and state-of-the-art developments to critically review the literature on phantom materials and on the fabrication of phantoms. By focusing on representative examples of standardization in diffuse optical imaging and spectroscopy, fluorescence-guided surgery and photoacoustic imaging, we identify unmet needs in the development of phantoms and a set of criteria (leveraging characterization, collaboration, communication and commitment) for the standardization of biophotonic instrumentation.

Standardization and implementation of fluorescence molecular endoscopy in the clinic

SIGNIFICANCE

Near-infrared fluorescence molecular endoscopy (NIR-FME) is an innovative technique allowing for in vivo visualization of molecular processes in hollow organs. Despite its potential for clinical translation, NIR-FME still faces challenges, for example, the lack of consensus in performing quality control and standardization of procedures and systems. This may hamper the clinical approval of the technology by authorities and its acceptance by endoscopists. Until now, several clinical trials using NIR-FME have been performed. However, most of these trials had different study designs, making comparison difficult.

AIM

We describe the need for standardization in NIR-FME, provide a pathway for setting up a standardized clinical study, and describe future perspectives for NIR-FME. Body: Standardization is challenging due to many parameters. Invariable parameters refer to the hardware specifications. Variable parameters refer to movement or tissue optical properties. Phantoms can be of aid when defining the influence of these variables or when standardizing a procedure.

CONCLUSION

There is a need for standardization in NIR-FME and hurdles still need to be overcome before a widespread clinical implementation of NIR-FME can be realized. When these hurdles are overcome, clinical outcomes can be compared and systems can be benchmarked, enabling clinical implementation.

3D printing fluorescent material with tunable optical properties

The 3D printing of fluorescent materials could help develop, validate, and translate imaging technologies, including systems for fluorescence-guided surgery. Despite advances in 3D printing techniques for optical targets, no comprehensive method has been demonstrated for the simultaneous incorporation of fluorophores and fine-tuning of absorption and scattering properties. Here, we introduce a photopolymer-based 3D printing method for manufacturing fluorescent material with tunable optical properties. The results demonstrate the ability to 3D print various individual fluorophores at reasonably high fluorescence yields, including IR-125, quantum dots, methylene blue, and rhodamine 590. Furthermore, tuning of the absorption and reduced scattering coefficients is demonstrated within the relevant mamalian soft tissue coefficient ranges of 0.005-0.05 mm-1 and 0.2-1.5 mm-1, respectively. Fabrication of fluorophore-doped biomimicking and complex geometric structures validated the ability to print feature sizes less than 200 μm. The presented methods and optical characterization techniques provide the foundation for the manufacturing of solid 3D printed fluorescent structures, with direct relevance to biomedical optics and the broad adoption of fast manufacturing methods in fluorescence imaging.

Propagation of structured light through tissue-mimicking phantoms

Optical interrogation of tissues is broadly considered in biomedical applications. Nevertheless, light scattering by tissue limits the resolution and accuracy achieved when investigating sub-surface tissue features. Light carrying optical angular momentum or complex polarization profiles, offers different propagation characteristics through scattering media compared to light with unstructured beam profiles. Here we discuss the behaviour of structured light scattered by tissue-mimicking phantoms. We study the spatial and the polarization profile of the scattered modes as a function of a range of optical parameters of the phantoms, with varying scattering and absorption coefficients and of different lengths. These results show the non-trivial trade-off between the advantages of structured light profiles and mode broadening, stimulating further investigations in this direction.

Performance test methods for near-infrared fluorescence imaging

PURPOSE

Near-infrared fluorescence (NIRF) imaging using exogenous contrast has gained much attention as a technique for enhancing visualization of vasculature using untargeted agents, as well as for the detection and localization of cancer with targeted agents. In order to address the emerging need for standardization of NIRF imaging technologies, it is necessary to identify the best practices suitable for objective, quantitative testing of key image quality characteristics. Toward the development of a battery of test methods that are rigorous yet applicable to a wide variety of devices, we have evaluated techniques for phantom design, measurement, and calculation of specific performance metrics.

METHODS

Using a NIRF imaging system for indocyanine green imaging, providing excitation at 780 nm and detection above 830 nm, we explored methods to evaluate uniformity, field of view, spectral crosstalk, spatial resolution, depth of field, sensitivity, linearity, and penetration depth. These measurements were performed using fluorophore-doped multiwell plate and high turbidity planar phantoms, as well as a 3D-printed multichannel phantom and a USAF 1951 resolution target.

RESULTS AND CONCLUSIONS

Based on a wide range of approaches described in medical and fluorescence imaging literature, we have developed and demonstrated a cohesive battery of test methods for evaluation of fluorescence image quality in wide-field imagers. We also propose a number of key metrics that can facilitate direct, quantitative comparison of device performance. These methods have the potential to facilitate more uniform evaluation and inter-comparison of clinical and preclinical imaging systems than is typically achieved, with the long-term goal of establishing international standards for fluorescence image quality assessment.

Indocyanine green matching phantom for fluorescence-guided surgery imaging system characterization and performance assessment

SIGNIFICANCE

Expanded use of fluorescence-guided surgery with devices approved for use with indocyanine green (ICG) has led to a range of commercial systems available. There is a compelling need to be able to independently characterize system performance and allow for cross-system comparisons.

AIM

The goal of this work is to expand on previous proposed fluorescence imaging standard designs to develop a long-term stable phantom that spectrally matches ICG characteristics and utilizes 3D printing technology for incorporating tissue-equivalent materials.

APPROACH

A batch of test targets was created to assess ICG concentration sensitivity in the 0.3- to 1000-nM range, tissue-equivalent depth sensitivity down to 6 mm, and spatial resolution with a USAF test chart. Comparisons were completed with a range of systems that have significantly different imaging capabilities and applications, including the Li-Cor® Odyssey, Li-Cor® Pearl, PerkinElmer® Solaris, and Stryker® Spy Elite.

RESULTS

Imaging of the ICG-matching phantoms with all four commercially available systems showed the ability to benchmark system performance and allow for cross-system comparisons. The fluorescence tests were able to assess differences in the detectable concentrations of ICG with sensitivity differences >10× for preclinical and clinical systems. Furthermore, the tests successfully assessed system differences in the depth-signal decay rate, as well as resolution performance and image artifacts. The manufacturing variations, photostability, and mechanical design of the tests showed promise in providing long-term stable standards for fluorescence imaging.

CONCLUSIONS

The presented ICG-matching phantom provides a major step toward standardizing performance characterization and cross-system comparisons for devices approved for use with ICG. The developed hybrid manufacturing platform can incorporate long-term stable fluorescing agents with 3D printed tissue-equivalent material. Further, long-term testing of the phantom and refinements to the manufacturing process are necessary for future implementation as a widely adopted fluorescence imaging standard.

Optical and Optoacoustic Imaging

The present chapter summarizes progress with optical methods that go beyond human vision. The focus is on two particular technologies: fluorescence molecular imaging and optoacoustic (photoacoustic) imaging. The rationale for the selection of these two methods is that in contrast to optical microscopy techniques, both fluorescence and optoacoustic imaging can achieve large fields of view, i.e., spanning several centimeters in two or three dimensions. Such fields of views relate better to human vision and can visualize large parts of tissue, a necessary premise for clinical detection. Conversely, optical microscopy methods only scan millimeter-sized dimensions or smaller. With such operational capacity, optical microscopy methods need to be guided by another visualization technique in order to scan a very specific area in tissue and typically only provide superficial measurements, i.e., information from depths that are of the order of 0.05-1 mm. This practice has generally limited their clinical applicability to some niche applications, such as optical coherence tomography of the retina. On the other hand, fluorescence molecular imaging and optoacoustic imaging emerge as more global optical imaging methods with wide applications in surgery, endoscopy, and non-invasive clinical imaging, as summarized in the following. The current progress in this field is based on a volume of recent review and other literature that highlights key advances achieved in technology and biomedical applications. Context and figures from references from the authors of this chapter have been used here, as it reflects our general view of the current status of the field.

Fluorescence imaging reversion using spatially variant deconvolution

Fluorescence imaging opens new possibilities for intraoperative guidance and early cancer detection, in particular when using agents that target specific disease features. Nevertheless, photon scattering in tissue degrades image quality and leads to ambiguity in fluorescence image interpretation and challenges clinical translation. We introduce the concept of capturing the spatially-dependent impulse response of an image and investigate Spatially Adaptive Impulse Response Correction (SAIRC), a method that is proposed for improving the accuracy and sensitivity achieved. Unlike classical methods that presume a homogeneous spatial distribution of optical properties in tissue, SAIRC explicitly measures the optical heterogeneity in tissues. This information allows, for the first time, the application of spatially-dependent deconvolution to correct the fluorescence images captured in relation to their modification by photon scatter. Using experimental measurements from phantoms and animals, we investigate the improvement in resolution and quantification over non-corrected images. We discuss how the proposed method is essential for maximizing the performance of fluorescence molecular imaging in the clinic.

Multi-Parametric Standardization of Fluorescence Imaging Systems Based on a Composite Phantom

OBJECTIVE

Fluorescence molecular imaging (FMI) has emerged as a promising tool for surgical guidance in oncology, with one of the few remaining challenges being the ability to offer quality control and data referencing. This paper investigates the use of a novel composite phantom to correct and benchmark FMI systems.

METHODS

This paper extends on previous work by describing a phantom design that can provide a more complete assessment of FMI systems through quantification of dynamic range and determination of spatial illumination patterns for both reflectance and fluorescence imaging. Various performance metrics are combined into a robust and descriptive "system benchmarking score," enabling not only the comprehensive comparison of different systems, but also for the first time, correction of the acquired data.

RESULTS

We show that systems developed for targeted fluorescence imaging can achieve benchmarking scores of up to 70%, while clinically available systems optimized for indocyanine green are limited to 50%, mostly due to greater leakage of ambient and excitation illumination and lower resolution. The image uniformity can also be approximated and employed for image flat-fielding, an important milestone toward data referencing. In addition, we demonstrate composite phantom use in assessing the performance of a surgical microscope and of a raster-scan imaging system.

CONCLUSION

Our results suggest that the new phantom has the potential to support high-fidelity FMI through benchmarking and image correction.

SIGNIFICANCE

Standardization of the FMI is a necessary process for establishing good imaging practices in clinical environments and for enabling high-fidelity imaging across patients and multi-center imaging studies.