Towards an Implantable Fluorescence Image Sensor for Real-Time Monitoring of Immune Response in Cancer Therapy

Toward a Wireless Image Sensor for Real-Time Fluorescence Microscopy in Cancer Therapy

We present a mm-sized, ultrasonically powered lensless CMOS image sensor as a progress towards wireless fluorescence microscopy. Access to biological information within the tissue has the potential to provide insights guiding diagnosis and treatment across numerous medical conditions including cancer therapy. This information, in conjunction with current clinical imaging techniques that have limitations in obtaining images continuously and lack wireless compatibility, can improve continual detection of multicell clusters deep within tissue. The proposed platform incorporates a 2.4 × 4.7 mm2 integrated circuit (IC) fabricated in TSMC 0.18 µm, a micro laser diode (µLD), a single piezoceramic and off-chip storage capacitors. The IC consists of a 36 × 40 array of capacitive trans-impedance amplifier-based pixels, wireless power management and communication via ultrasound and a laser driver all controlled by a Finite State Machine. The piezoceramic harvests energy from the acoustic waves at a depth of 2 cm to power up the IC and transfer 11.5 kbits/frame via backscattering. During Charge-Up, the off-chip capacitor stores charge to later supply a high-power 78 mW µLD during Imaging. Proof of concept of the imaging front end is shown by imaging distributions of CD8 T-cells, an indicator of the immune response to cancer, ex vivo, in the lymph nodes of a functional immune system (BL6 mice) against colorectal cancer consistent with the results of a fluorescence microscope. The overall system performance is verified by detecting 140 µm features on a USAF resolution target with 32 ms exposure time and 389 ms ultrasound backscattering.

Multicolor fluorescence microscopy for surgical guidance using a chip-scale imager with a low-NA fiber optic plate and a multi-bandpass interference filter

In curative-intent cancer surgery, intraoperative fluorescence imaging of both diseased and healthy tissue can help to ensure successful removal of all gross and microscopic disease with minimal damage to neighboring critical structures, such as nerves. Current fluorescence-guided surgery (FGS) systems, however, rely on bulky and rigid optics that incur performance-limiting trade-offs between sensitivity and maneuverability. Moreover, many FGS systems are incapable of multiplexed imaging. As a result, clinical FGS is currently limited to millimeter-scale detection of a single fluorescent target. Here we present a scalable, lens-less fluorescence imaging chip, VISION, capable of sensitive and multiplexed detection within a compact form factor. Central to VISION is a novel optical frontend design combining a low-numerical-aperture fiber optic plate (LNA-FOP) and a multi-bandpass interference filter, which is affixed to a custom CMOS image sensor. The LNA-FOP acts as a planar collimator to improve resolution and compensate for the angle-sensitivity of the interference filter, enabling high-resolution and multiplexed fluorescence imaging without lenses. We show VISION is capable of detecting tumor foci of less than 100 cells at near video framerates and, as proof of principle, can simultaneously visualize both tumor and nerves in ex vivo prostate tissue.

Towards A Wireless Image Sensor for Real-Time Fluorescence Microscopy in Cancer Therapy

We present a mm-sized, ultrasonically powered lensless CMOS image sensor as a progress towards wireless fluorescence microscopy. Access to biological information within the tissue has the potential to provide insights guiding diagnosis and treatment across numerous medical conditions including cancer therapy. This information, in conjunction with current clinical imaging techniques that have limitations in obtaining images continuously and lack wireless compatibility, can improve continual detection of multicell clusters deep within tissue. The proposed platform incorporates a 2.4×4.7 mm2 integrated circuit (IC) fabricated in TSMC 0.18 μm, a micro laser diode (μLD), a single piezoceramic and off-chip storage capacitors. The IC consists of a 36×40 array of capacitive trans-impedance amplifier-based pixels, wireless power management and communication via ultrasound and a laser driver all controlled by a Finite State Machine. The piezoceramic harvests energy from the acoustic waves at a depth of 2 cm to power up the IC and transfer 11.5 kbits/frame via backscattering. During Charge-Up, the off-chip capacitor stores charge to later supply a high-power 78 mW μLD during Imaging. Proof of concept of the imaging front end is shown by imaging distributions of CD8 T-cells, an indicator of the immune response to cancer, ex vivo, in the lymph nodes of a functional immune system (BL6 mice) against colorectal cancer consistent with the results of a fluorescence microscope. The overall system performance is verified by detecting 140 μm features on a USAF resolution target with 32 ms exposure time and 389 ms ultrasound backscattering.

Concept of a fully-implantable system to monitor tumor recurrence

Current treatment for glioblastoma includes tumor resection followed by radiation, chemotherapy, and periodic post-operative examinations. Despite combination therapies, patients face a poor prognosis and eventual recurrence, which often occurs at the resection site. With standard MRI imaging surveillance, histologic changes may be overlooked or misinterpreted, leading to erroneous conclusions about the course of adjuvant therapy and subsequent interventions. To address these challenges, we propose an implantable system for accurate continuous recurrence monitoring that employs optical sensing of fluorescently labeled cancer cells and is implanted in the resection cavity during the final stage of tumor resection. We demonstrate the feasibility of the sensing principle using miniaturized system components, optical tissue phantoms, and porcine brain tissue in a series of experimental trials. Subsequently, the system electronics are extended to include circuitry for wireless energy transfer and power management and verified through electromagnetic field, circuit simulations and test of an evaluation board. Finally, a holistic conceptual system design is presented and visualized. This novel approach to monitor glioblastoma patients is intended to early detect recurrent cancerous tissue and enable personalization and optimization of therapy thus potentially improving overall prognosis.

Fully Integrated Ultra-thin Intraoperative Micro-imager for Cancer Detection Using Upconverting Nanoparticles

PURPOSE

Intraoperative detection and removal of microscopic residual disease (MRD) remain critical to the outcome of cancer surgeries. Today's minimally invasive surgical procedures require miniaturization and surgical integration of highly sensitive imagers to seamlessly integrate into the modern clinical workflow. However, current intraoperative imagers remain cumbersome and still heavily dependent on large lenses and rigid filters, precluding further miniaturization and integration into surgical tools.

PROCEDURES

We have successfully engineered a chip-scale intraoperative micro-imager array-without optical filters or lenses-integrated with lanthanide-based alloyed upconverting nanoparticles (aUCNPs) to achieve tissue imaging using a single micro-chip. This imaging platform is able to leverage the unique optical properties of aUCNPs (long luminescent lifetime, high-efficiency upconversion, no photobleaching) by utilizing a time-resolved imaging method to acquire images using a 36-by-80-pixel, 2.3 mm [Formula: see text] 4.8 mm silicon-based electronic imager micro-chip, that is, less than 100-µm thin. Each pixel incorporates a novel architecture enabling automated background measurement and cancellation. We have validated the performance, spatial resolution, and the background cancellation scheme of the imaging platform, using resolution test targets and mouse prostate tumor sample intratumorally injected with aUCNPs. To demonstrate the ability to image MRD, or tumor margins, we evaluated the imaging platform in visualizing a single-cell thin section of the injected prostate tumor sample.

RESULTS

Tested on USAF resolution targets, the imager is able to achieve a resolution of 71 µm. We have also demonstrated successful background cancellation, achieving a signal-to-background ratio of 8 when performing ex vivo imaging on aUCNP-injected prostate tumor sample, improved from originally 0.4. The performance of the imaging platform on single-cell layer sections was also evaluated and the sensor achieved a signal-to-background ratio of 4.3 in resolving cell clusters with sizes as low as 200 cells.

CONCLUSION

The imaging system proposed here is a scalable chip-scale ultra-thin alternative for bulky conventional intraoperative imagers. Its novel pixel architecture and background correction scheme enable visualization of microscopic-scale residual disease while remaining completely free of lenses and filters, achieving an ultra-miniaturized form factor-critical for intraoperative settings.

3D Reconstruction of cellular images from microfabricated imagers using fully-adaptive deep neural networks

Millimeter-scale multi-cellular level imagers enable various applications, ranging from intraoperative surgical navigation to implantable sensors. However, the tradeoffs for miniaturization compromise resolution, making extracting 3D cell locations challenging—critical for tumor margin assessment and therapy monitoring. This work presents three machine-learning-based modules that extract spatial information from single image acquisitions using custom-made millimeter-scale imagers. The neural networks were trained on synthetically-generated (using Perlin noise) cell images. The first network is a convolutional neural network estimating the depth of a single layer of cells, the second is a deblurring module correcting for the point spread function (PSF). The final module extracts spatial information from a single image acquisition of a 3D specimen and reconstructs cross-sections, by providing a layered “map” of cell locations. The maximum depth error of the first module is 100 µm, with 87% test accuracy. The second module’s PSF correction achieves a least-square-error of only 4%. The third module generates a binary “cell” or “no cell” per-pixel labeling with an accuracy ranging from 89% to 85%. This work demonstrates the synergy between ultra-small silicon-based imagers that enable in vivo imaging but face a trade-off in spatial resolution, and the processing power of neural networks to achieve enhancements beyond conventional linear optimization techniques.