Development of End-to-End Artificial Intelligence Models for Surgical Planning in Transforaminal Lumbar Interbody Fusion

Transforaminal lumbar interbody fusion (TLIF) is a commonly used technique for treating lumbar degenerative diseases. In this study, we developed a fully computer-supported pipeline to predict both the cage height and the degree of lumbar lordosis subtraction from the pelvic incidence (PI-LL) after TLIF surgery, utilizing preoperative X-ray images. The automated pipeline comprised two primary stages. First, the pretrained BiLuNet deep learning model was employed to extract essential features from X-ray images. Subsequently, five machine learning algorithms were trained using a five-fold cross-validation technique on a dataset of 311 patients to identify the optimal models to predict interbody cage height and postoperative PI-LL. LASSO regression and support vector regression demonstrated superior performance in predicting interbody cage height and postoperative PI-LL, respectively. For cage height prediction, the root mean square error (RMSE) was calculated as 1.01, and the model achieved the highest accuracy at a height of 12 mm, with exact prediction achieved in 54.43% (43/79) of cases. In most of the remaining cases, the prediction error of the model was within 1 mm. Additionally, the model demonstrated satisfactory performance in predicting PI-LL, with an RMSE of 5.19 and an accuracy of 0.81 for PI-LL stratification. In conclusion, our results indicate that machine learning models can reliably predict interbody cage height and postoperative PI-LL.

Retina-TransNet: A Gradient-Guided Few-Shot Retinal Vessel Segmentation Net

Due to the high labor cost of physicians, it is difficult to collect a rich amount of manually-labeled medical images for developing learning-based computer-aided diagnosis (CADx) systems or segmentation algorithms. To tackle this issue, we reshape the image segmentation task as an image-to-image (I2I) translation problem and propose a retinal vascular segmentation network, which can achieve good cross-domain generalizability even with a small amount of training data. We devise primarily two components to facilitate this I2I-based segmentation method. The first is the constraints provided by the proposed gradient-vector-flow (GVF) loss, and, the second is a two-stage Unet (2Unet) generator with a skip connection. This configuration makes 2Unet's first-stage play a role similar to conventional Unet, but forces 2Unet's second stage to learn to be a refinement module. Extensive experiments show that by re-casting retinal vessel segmentation as an image-to-image translation problem, our I2I translator-based segmentation subnetwork achieves better cross-domain generalizability than existing segmentation methods. Our model, trained on one dataset, e.g., DRIVE, can produce segmentation results stably on datasets of other domains, e.g., CHASE-DB1, STARE, HRF, and DIARETDB1, even in low-shot circumstances.

Real-Time 3D Liver Tumor Localization via Combined Optical Surface Imaging and an X-Ray Projection from Arbitrary Imaging Angles

PURPOSE/OBJECTIVE(S)

A deep learning (DL)-based, deformable registration-driven liver tumor localization technique was developed for onboard deformable motion tracking. The technique (Surf-X360-Bio) uses in-room optical surface imaging and an x-ray projection at an arbitrary scan angle to solve volumetric liver and liver tumor motion in real-time.

MATERIALS/METHODS

Surf-X360-Bio solves the volumetric motion of the liver and localizes the liver tumor, through deforming liver and liver tumor meshes segmented on prior 4D-CTs/CBCTs. It uses real-time onboard information from an optical surface image and a simultaneously-acquired x-ray projection (from an arbitrary scan angle). Surf-X360-Bio localizes tumors via two steps: liver boundary motion estimation and intra-liver motion derivation. Surf-X360-Bio first estimates liver boundary motion by a patient-specific surface imaging model (Surf), utilizing the correlation between the external body surface motion and the internal liver boundary motion. As the correlation can be imperfect, the residual motion estimation errors were corrected by a patient-specific, angle-agnostic x-ray imaging model (X360). X360 deformed the liver boundary to match motion-related imaging features encoded in an arbitrarily-angled x-ray projection, using the Surf output for initialization. X360 adopted a geometry-aware learning module to extract and adapt to angle-varying features of arbitrarily-angled x-ray images, by calculating the projection system matrix of each x-ray image on-the-fly during model training and inference. After the liver boundary motion estimation by Surf and X360, intra-liver deformation was solved by a biomechanical model (Bio) to propagate the liver boundary motion inside to localize the tumors. The DL-based Bio model used domain knowledge of tissue biomechanics and finite element analysis (FEA) to solve intra-liver motion, with a much faster speed than conventional FEA methods to meet the real-time constraint. Surf-X360-Bio was evaluated using a dataset of 34 liver patients. Liver tumor localization accuracy was evaluated by the Dice similarity coefficient (DSC), the 95-percentile Hausdorff distance (HD95), and the center-of-mass error (COME).

RESULTS

Using 3,306 motion scenarios spanning the 360 degree x-ray scan angles for each testing patient, Surf-X360-Bio localized the liver tumors to 0.81 (mean) ± 0.16 (s.d.) in DSC, 2.5 ± 1.7 mm in HD95, and 2.1 ± 1.8 mm in COME. In comparison, the prior reference image without deformable registration-driven localization yielded 0.42 ± 0.29 in DSC, 8.1 ± 5.2 mm in HD95, and 8.5 ± 5.2 mm in COME. Via Surf-X360-Bio, the overall inference time was below 230 ms for each case.

CONCLUSION

Combining optical surface imaging and x-ray imaging, Surf-X360-Bio achieved fast (<250 ms inference time), accurate (mean error < 2.1 mm), and robust liver tumor localizations at arbitrary x-ray scan angles for real-time, marker-less 3D deformable motion tracking.

Detection of Lumbar Spondylolisthesis from X-ray Images Using Deep Learning Network

Spondylolisthesis refers to the displacement of a vertebral body relative to the vertrabra below it, which can cause radicular symptoms, back pain or leg pain. It usually occurs in the lower lumbar spine, especially in women over the age of 60. The prevalence of spondylolisthesis is expected to rise as the global population ages, requiring prudent action to promptly identify it in clinical settings. The goal of this study was to develop a computer-aided diagnostic (CADx) algorithm, LumbarNet, and to evaluate the efficiency of this model in automatically detecting spondylolisthesis from lumbar X-ray images. Built upon U-Net, feature fusion module (FFM) and collaborating with (i) a P-grade, (ii) a piecewise slope detection (PSD) scheme, and (iii) a dynamic shift (DS), LumbarNet was able to analyze complex structural patterns on lumbar X-ray images, including true lateral, flexion, and extension lateral views. Our results showed that the model achieved a mean intersection over union (mIOU) value of 0.88 in vertebral region segmentation and an accuracy of 88.83% in vertebral slip detection. We conclude that LumbarNet outperformed U-Net, a commonly used method in medical image segmentation, and could serve as a reliable method to identify spondylolisthesis.

DotFAN: A Domain-Transferred Face Augmentation Net

The performance of a convolutional neural network (CNN) based face recognition model largely relies on the richness of labeled training data. However, it is expensive to collect a training set with large variations of a face identity under different poses and illumination changes, so the diversity of within-class face images becomes a critical issue in practice. In this paper, we propose a 3D model-assisted domain-transferred face augmentation network (DotFAN) that can generate a series of variants of an input face based on the knowledge distilled from existing rich face datasets of other domains. Extending from StarGAN's architecture, DotFAN integrates with two additional subnetworks, i.e., face expert model (FEM) and face shape regressor (FSR), for latent facial code control. While FSR aims to extract face attributes, FEM is designed to capture a face identity. With their aid, DotFAN can separately learn facial feature codes and effectively generate face images of various facial attributes while keeping the identity of augmented faces unaltered. Experiments show that DotFAN is beneficial for augmenting small face datasets to improve their within-class diversity so that a better face recognition model can be learned from the augmented dataset.

Retention of features on a mapped Drosophila brain surface using a Bézier-tube-based surface model averaging technique

Model averaging is a widely used technique in biomedical applications. Two established model averaging methods, iterative shape averaging (ISA) method and virtual insect brain (VIB) method, have been applied to several organisms to generate average representations of their brain surfaces. However, without sufficient samples, some features of the average Drosophila brain surface obtained using the above methods may disappear or become distorted. To overcome this problem, we propose a Bézier-tube-based surface model averaging strategy. The proposed method first compensates for disparities in position, orientation, and dimension of input surfaces, and then evaluates the average surface by performing shape-based interpolation. Structural features with larger individual disparities are simplified with half-ellipse-shaped Bézier tubes, and are unified according to these tubes to avoid distortion during the averaging process. Experimental results show that the average model yielded by our method could preserve fine features and avoid structural distortions even if only a limit amount of input samples are used. Finally, we qualitatively compare our results with those obtained by ISA and VIB methods by measuring the surface-to-surface distances between input surfaces and the averaged ones. The comparisons show that the proposed method could generate a more representative average surface than both ISA and VIB methods.

Optimal multiresolution blending of confocal microscope images

Typical mosaicing schemes assume that to-be-combined images are equally informative; thus, the images are processed in a similar manner. However, the new imaging technique for confocal fluorescence images has revealed a problem when two asymmetrically informative biological images are stitched during microscope image mosaicing. The latter process is widely used in biological studies to generate a higher resolution image by combining multiple images taken at different times and angles. To resolve the earlier problem, we propose a multiresolution optimization approach that evaluates the blending coefficients based on the relative importance of the overlapping regions of the to-be-combined image pair. The blending coefficients are the optimal solution obtained by a quadratic programming algorithm with constraints that are enforced by the biological requirements. We demonstrate the efficacy of the proposed approach on several confocal microscope fluorescence images and compare the results with those derived by other methods.