Explainable liver tumor delineation in surgical specimens using hyperspectral imaging and deep learning

SpeCamX: mobile app that turns unmodified smartphones into multispectral imagers

We present the development of SpeCamX, a mobile application that enables an unmodified smartphone into a multispectral imager. Multispectral imaging provides detailed spectral information about objects or scenes, but its accessibility has been limited due to its specialized requirements for the device. SpeCamX overcomes this limitation by utilizing the RGB photographs captured by smartphones and converting them into multispectral images spanning a range of 420 to 680 nm without a need for internal modifications or external attachments. The app also includes plugin functions for extracting medical information from the resulting multispectral data cube. In a clinical study, SpeCamX was used to implement an augmented smartphone bilirubinometer, predicting blood bilirubin levels (BBL) with superior performance in accuracy, efficiency and stability compared to default smartphone cameras. This innovative technology democratizes multispectral imaging, making it accessible to a wider audience and opening new possibilities for both medical and non-medical applications.

Application of 3D Bioprinting in Liver Diseases

Liver diseases are the primary reason for morbidity and mortality in the world. Owing to a shortage of organ donors and postoperative immune rejection, patients routinely suffer from liver failure. Unlike 2D cell models, animal models, and organoids, 3D bioprinting can be successfully employed to print living tissues and organs that contain blood vessels, bone, and kidney, heart, and liver tissues and so on. 3D bioprinting is mainly classified into four types: inkjet 3D bioprinting, extrusion-based 3D bioprinting, laser-assisted bioprinting (LAB), and vat photopolymerization. Bioinks for 3D bioprinting are composed of hydrogels and cells. For liver 3D bioprinting, hepatic parenchymal cells (hepatocytes) and liver nonparenchymal cells (hepatic stellate cells, hepatic sinusoidal endothelial cells, and Kupffer cells) are commonly used. Compared to conventional scaffold-based approaches, marked by limited functionality and complexity, 3D bioprinting can achieve accurate cell settlement, a high resolution, and more efficient usage of biomaterials, better mimicking the complex microstructures of native tissues. This method will make contributions to disease modeling, drug discovery, and even regenerative medicine. However, the limitations and challenges of this method cannot be ignored. Limitation include the requirement of diverse fabrication technologies, observation of drug dynamic response under perfusion culture, the resolution to reproduce complex hepatic microenvironment, and so on. Despite this, 3D bioprinting is still a promising and innovative biofabrication strategy for the creation of artificial multi-cellular tissues/organs.

Signal to noise ratio quantifies the contribution of spectral channels to classification of human head and neck tissues ex vivo using deep learning and multispectral imaging

SIGNIFICANCE

Accurate identification of tissues is critical for performing safe surgery. Combining multispectral imaging (MSI) with deep learning is a promising approach to increasing tissue discrimination and classification. Evaluating the contributions of spectral channels to tissue discrimination is important for improving MSI systems.

AIM

Develop a metric to quantify the contributions of individual spectral channels to tissue classification in MSI.

APPROACH

MSI was integrated into a digital operating microscope with three sensors and seven illuminants. Two convolutional neural network (CNN) models were trained to classify 11 head and neck tissue types using white light (RGB) or MSI images. The signal to noise ratio (SNR) of spectral channels was compared with the impact of channels on tissue classification performance as determined using CNN visualization methods.

RESULTS

Overall tissue classification accuracy was higher with use of MSI images compared with RGB images, both for classification of all 11 tissue types and binary classification of nerve and parotid ( p < 0.001 ). Removing spectral channels with SNR > 20 reduced tissue classification accuracy.

CONCLUSIONS

The spectral channel SNR is a useful metric for both understanding CNN tissue classification and quantifying the contributions of different spectral channels in an MSI system.

Leberchirurgie 4.0 - OP-Planung, Volumetrie, Navigation und Virtuelle Realität

Durch die Optimierung der konservativen Behandlung, die Verbesserung der bildgebenden Verfahren und die Weiterentwicklung der Operationstechniken haben sich das operative Spektrum sowie der Maßstab für die Resektabilität in Bezug auf die Leberchirurgie in den letzten Jahrzehnten deutlich verändert.

Dank zahlreicher technischer Entwicklungen, insbesondere der 3-dimensionalen Segmentierung, kann heutzutage die präoperative Planung und die Orientierung während der Operation selbst, vor allem bei komplexen Eingriffen, unter Berücksichtigung der patientenspezifischen Anatomie erleichtert werden.

Neue Technologien wie 3-D-Druck, virtuelle und augmentierte Realität bieten zusätzliche Darstellungsmöglichkeiten für die individuelle Anatomie. Verschiedene intraoperative Navigationsmöglichkeiten sollen die präoperative Planung im Operationssaal verfügbar machen, um so die Patientensicherheit zu erhöhen.

Dieser Übersichtsartikel soll einen Überblick über den gegenwärtigen Stand der verfügbaren Technologien sowie einen Ausblick in den Operationssaal der Zukunft geben.

Detecting Asymptomatic Infections of Rice Bacterial Leaf Blight Using Hyperspectral Imaging and 3-Dimensional Convolutional Neural Network With Spectral Dilated Convolution

Rice is one of the most important food crops for human beings. Its total production ranks third in the grain crop output. Bacterial Leaf Blight (BLB), as one of the three major diseases of rice, occurs every year, posing a huge threat to rice production and safety. There is an asymptomatic period between the infection and the onset periods, and BLB will spread rapidly and widely under suitable conditions. Therefore, accurate detection of early asymptomatic BLB is very necessary. The purpose of this study was to test the feasibility of detecting early asymptomatic infection of the rice BLB disease based on hyperspectral imaging and Spectral Dilated Convolution 3-Dimensional Convolutional Neural Network (SDC-3DCNN). First, hyperspectral images were obtained from rice leaves infected with the BLB disease at the tillering stage. The spectrum was smoothed by the Savitzky-Golay (SG) method, and the wavelength between 450 and 950 nm was intercepted for analysis. Then Principal Component Analysis (PCA) and Random Forest (RF) were used to extract the feature information from the original spectra as inputs. The overall performance of the SDC-3DCNN model with different numbers of input features and different spectral dilated ratios was evaluated. Lastly, the saliency map visualization was used to explain the sensitivity of individual wavelengths. The results showed that the performance of the SDC-3DCNN model reached an accuracy of 95.4427% when the number of inputs is 50 characteristic wavelengths (extracted by RF) and the dilated ratio is set at 5. The saliency-sensitive wavelengths were identified in the range from 530 to 570 nm, which overlaps with the important wavelengths extracted by RF. According to our findings, combining hyperspectral imaging and deep learning can be a reliable approach for identifying early asymptomatic infection of the rice BLB disease, providing sufficient support for early warning and rice disease prevention.

Hyperspectral Imaging for Clinical Applications

Measuring morphological and biochemical features of tissue is crucial for disease diagnosis and surgical guidance, providing clinically significant information related to pathophysiology. Hyperspectral imaging (HSI) techniques obtain both spatial and spectral features of tissue without labeling molecules such as fluorescent dyes, which provides rich information for improved disease diagnosis and treatment. Recent advances in HSI systems have demonstrated its potential for clinical applications, especially in disease diagnosis and image-guided surgery. This review summarizes the basic principle of HSI and optical systems, deep-learning-based image analysis, and clinical applications of HSI to provide insight into this rapidly growing field of research. In addition, the challenges facing the clinical implementation of HSI techniques are discussed.