Federated learning with knowledge distillation for multi-organ segmentation with partially labeled datasets

The state-of-the-art multi-organ CT segmentation relies on deep learning models, which only generalize when trained on large samples of carefully curated data. However, it is challenging to train a single model that can segment all organs and types of tumors since most large datasets are partially labeled or are acquired across multiple institutes that may differ in their acquisitions. A possible solution is Federated learning, which is often used to train models on multi-institutional datasets where the data is not shared across sites. However, predictions of federated learning can be unreliable after the model is locally updated at sites due to 'catastrophic forgetting'. Here, we address this issue by using knowledge distillation (KD) so that the local training is regularized with the knowledge of a global model and pre-trained organ-specific segmentation models. We implement the models in a multi-head U-Net architecture that learns a shared embedding space for different organ segmentation, thereby obtaining multi-organ predictions without repeated processes. We evaluate the proposed method using 8 publicly available abdominal CT datasets of 7 different organs. Of those datasets, 889 CTs were used for training, 233 for internal testing, and 30 volumes for external testing. Experimental results verified that our proposed method substantially outperforms other state-of-the-art methods in terms of accuracy, inference time, and the number of parameters.

Machine learning-based high-frequency neuronal spike reconstruction from low-frequency and low-sampling-rate recordings

Recording neuronal activity using multiple electrodes has been widely used to understand the functional mechanisms of the brain. Increasing the number of electrodes allows us to decode more variety of functionalities. However, handling massive amounts of multichannel electrophysiological data is still challenging due to the limited hardware resources and unavoidable thermal tissue damage. Here, we present machine learning (ML)-based reconstruction of high-frequency neuronal spikes from subsampled low-frequency band signals. Inspired by the equivalence between high-frequency restoration and super-resolution in image processing, we applied a transformer ML model to neuronal data recorded from both in vitro cultures and in vivo male mouse brains. Even with the x8 downsampled datasets, our trained model reasonably estimated high-frequency information of spiking activity, including spike timing, waveform, and network connectivity. With our ML-based data reduction applicable to existing multichannel recording hardware while achieving neuronal signals of broad bandwidths, we expect to enable more comprehensive analysis and control of brain functions.

DeepFold: enhancing protein structure prediction through optimized loss functions, improved template features, and re-optimized energy function

MOTIVATION

Predicting protein structures with high accuracy is a critical challenge for the broad community of life sciences and industry. Despite progress made by deep neural networks like AlphaFold2, there is a need for further improvements in the quality of detailed structures, such as side-chains, along with protein backbone structures.

RESULTS

Building upon the successes of AlphaFold2, the modifications we made include changing the losses of side-chain torsion angles and frame aligned point error, adding loss functions for side chain confidence and secondary structure prediction, and replacing template feature generation with a new alignment method based on conditional random fields. We also performed re-optimization by conformational space annealing using a molecular mechanics energy function which integrates the potential energies obtained from distogram and side-chain prediction. In the CASP15 blind test for single protein and domain modeling (109 domains), DeepFold ranked fourth among 132 groups with improvements in the details of the structure in terms of backbone, side-chain, and Molprobity. In terms of protein backbone accuracy, DeepFold achieved a median GDT-TS score of 88.64 compared with 85.88 of AlphaFold2. For TBM-easy/hard targets, DeepFold ranked at the top based on Z-scores for GDT-TS. This shows its practical value to the structural biology community, which demands highly accurate structures. In addition, a thorough analysis of 55 domains from 39 targets with publicly available structures indicates that DeepFold shows superior side-chain accuracy and Molprobity scores among the top-performing groups.

AVAILABILITY AND IMPLEMENTATION

DeepFold tools are open-source software available at https://github.com/newtonjoo/deepfold.

One-shot Federated Learning on Medical Data using Knowledge Distillation with Image Synthesis and Client Model Adaptation

One-shot federated learning (FL) has emerged as a promising solution in scenarios where multiple communication rounds are not practical. Notably, as feature distributions in medical data are less discriminative than those of natural images, robust global model training with FL is non-trivial and can lead to overfitting. To address this issue, we propose a novel one-shot FL framework leveraging Image Synthesis and Client model Adaptation (FedISCA) with knowledge distillation (KD). To prevent overfitting, we generate diverse synthetic images ranging from random noise to realistic images. This approach (i) alleviates data privacy concerns and (ii) facilitates robust global model training using KD with decentralized client models. To mitigate domain disparity in the early stages of synthesis, we design noise-adapted client models where batch normalization statistics on random noise (synthetic images) are updated to enhance KD. Lastly, the global model is trained with both the original and noise-adapted client models via KD and synthetic images. This process is repeated till global model convergence. Extensive evaluation of this design on five small- and three large-scale medical image classification datasets reveals superior accuracy over prior methods. Code is available at https://github.com/myeongkyunkang/FedISCA.

Time-Dependent Deep Image Prior for Dynamic MRI

We propose a novel unsupervised deep-learning-based algorithm for dynamic magnetic resonance imaging (MRI) reconstruction. Dynamic MRI requires rapid data acquisition for the study of moving organs such as the heart. We introduce a generalized version of the deep-image-prior approach, which optimizes the weights of a reconstruction network to fit a sequence of sparsely acquired dynamic MRI measurements. Our method needs neither prior training nor additional data. In particular, for cardiac images, it does not require the marking of heartbeats or the reordering of spokes. The key ingredients of our method are threefold: 1) a fixed low-dimensional manifold that encodes the temporal variations of images; 2) a network that maps the manifold into a more expressive latent space; and 3) a convolutional neural network that generates a dynamic series of MRI images from the latent variables and that favors their consistency with the measurements in k -space. Our method outperforms the state-of-the-art methods quantitatively and qualitatively in both retrospective and real fetal cardiac datasets. To the best of our knowledge, this is the first unsupervised deep-learning-based method that can reconstruct the continuous variation of dynamic MRI sequences with high spatial resolution.

Grid-Free Localization Algorithm Using Low-rank Hankel Matrix for Super-Resolution Microscopy

Localization microscopy, such as STORM / PALM, can reconstruct super-resolution images with a nanometer resolution through the iterative localization of fluorescence molecules. Recent studies in this area have focused mainly on the localization of densely activated molecules to improve temporal resolutions. However, higher density imaging requires an advanced algorithm that can resolve closely spaced molecules. Accordingly, sparsitydriven methods have been studied extensively. One of the major limitations of existing sparsity-driven approaches is the need for a fine sampling grid or for Taylor series approximation which may result in some degree of localization bias toward the grid. In addition, prior knowledge of the point-spread function (PSF) is required. To address these drawbacks, here we propose a true grid-free localization algorithm with adaptive PSF estimation. Specifically, based on the observation that sparsity in the spatial domain implies a low rank in the Fourier domain, the proposed method converts source localization problems into Fourier-domain signal processing problems so that a truly gridfree localization is possible. We verify the performance of the newly proposed method with several numerical simulations and a live-cell imaging experiment.

CNN-Based Projected Gradient Descent for Consistent CT Image Reconstruction

We present a new image reconstruction method that replaces the projector in a projected gradient descent (PGD) with a convolutional neural network (CNN). Recently, CNNs trained as image-to-image regressors have been successfully used to solve inverse problems in imaging. However, unlike existing iterative image reconstruction algorithms, these CNN-based approaches usually lack a feedback mechanism to enforce that the reconstructed image is consistent with the measurements. We propose a relaxed version of PGD wherein gradient descent enforces measurement consistency, while a CNN recursively projects the solution closer to the space of desired reconstruction images. We show that this algorithm is guaranteed to converge and, under certain conditions, converges to a local minimum of a non-convex inverse problem. Finally, we propose a simple scheme to train the CNN to act like a projector. Our experiments on sparse-view computed-tomography reconstruction show an improvement over total variation-based regularization, dictionary learning, and a state-of-the-art deep learning-based direct reconstruction technique.

Sparse and Low-Rank Decomposition of a Hankel Structured Matrix for Impulse Noise Removal

Recently, the annihilating filter-based low-rank Hankel matrix (ALOHA) approach was proposed as a powerful image inpainting method. Based on the observation that smoothness or textures within an image patch correspond to sparse spectral components in the frequency domain, ALOHA exploits the existence of annihilating filters and the associated rank-deficient Hankel matrices in an image domain to estimate any missing pixels. By extending this idea, we propose a novel impulse-noise removal algorithm that uses the sparse and low-rank decomposition of a Hankel structured matrix. This method, referred to as the robust ALOHA, is based on the observation that an image corrupted with the impulse noise has intact pixels; consequently, the impulse noise can be modeled as sparse components, whereas the underlying image can still be modeled using a low-rank Hankel structured matrix. To solve the sparse and low-rank matrix decomposition problem, we propose an alternating direction method of multiplier approach, with initial factorized matrices coming from a low-rank matrix-fitting algorithm. To adapt local image statistics that have distinct spectral distributions, the robust ALOHA is applied in a patch-by-patch manner. Experimental results from impulse noise for both single-channel and multichannel color images demonstrate that the robust ALOHA is superior to existing approaches, especially during the reconstruction of complex texture patterns.

Predicting cognitive decline with deep learning of brain metabolism and amyloid imaging

For effective treatment of Alzheimer's disease (AD), it is important to identify subjects who are most likely to exhibit rapid cognitive decline. We aimed to develop an automatic image interpretation system based on a deep convolutional neural network (CNN) which can accurately predict future cognitive decline in mild cognitive impairment (MCI) patients using flurodeoxyglucose and florbetapir positron emission tomography (PET). PET images of 139 patients with AD, 171 patients with MCI and 182 normal subjects obtained from Alzheimer's Disease Neuroimaging Initiative database were used. Deep CNN was trained using 3-dimensional PET volumes of AD and normal controls as inputs. Manually defined image feature extraction such as quantification using predefined region-of-interests was unnecessary for our approach. Furthermore, it used minimally processed images without spatial normalization which has been commonly used in conventional quantitative analyses. Cognitive outcome of MCI subjects was predicted using this network. The prediction accuracy of the conversion of mild cognitive impairment to AD was compared with the conventional feature-based quantification approach. Accuracy of prediction (84.2%) for conversion to AD in MCI patients outperformed conventional feature-based quantification approaches. ROC analyses revealed that performance of CNN-based approach was significantly higher than that of the conventional quantification methods (p < 0.05). Output scores of the network were strongly correlated with the longitudinal change in cognitive measurements (p < 0.05). These results show the feasibility of deep learning as a practical tool for developing predictive neuroimaging biomarker.

MRI artifact correction using sparse + low-rank decomposition of annihilating filter-based hankel matrix

PURPOSE

Magnetic resonance imaging (MRI) artifacts are originated from various sources including instability of an magnetic resonance (MR) system, patient motion, inhomogeneities of gradient fields, and so on. Such MRI artifacts are usually considered as irreversible, so additional artifact-free scan or navigator scan is necessary. To overcome these limitations, this article proposes a novel compressed sensing-based approach for removal of various MRI artifacts.

THEORY

Recently, the annihilating filter based low-rank Hankel matrix approach was proposed. The annihilating filter based low-rank Hankel matrix exploits the duality between the low-rankness of weighted Hankel structured matrix and the sparsity of signal in a transform domain. Because MR artifacts usually appeared as sparse k-space components, the low-rank Hankel matrix from underlying artifact-free k-space data can be exploited to decompose the sparse outliers.

METHODS

The sparse + low-rank decomposition framework using Hankel matrix was proposed for removal of MRI artifacts. Alternating direction method of multipliers algorithm was employed for the minimization of associated cost function with the initialized matrices from a factorization-based matrix completion.

RESULTS

Experimental results demonstrated that the proposed algorithm can correct MR artifacts including herringbone (crisscross), motion, and zipper artifacts without image distortion.

CONCLUSION

The proposed method may be a robust correction solution for various MRI artifacts that can be represented as sparse outliers. Magn Reson Med 78:327-340, 2017. © 2016 International Society for Magnetic Resonance in Medicine.

Reference-free single-pass EPI Nyquist ghost correction using annihilating filter-based low rank Hankel matrix (ALOHA)

PURPOSE

MR measurements from an echo-planar imaging (EPI) sequence produce Nyquist ghost artifacts that originate from inconsistencies between odd and even echoes. Several reconstruction algorithms have been proposed to reduce such artifacts, but most of these methods require either additional reference scans or multipass EPI acquisition. This article proposes a novel and accurate single-pass EPI ghost artifact correction method that does not require any additional reference data.

THEORY AND METHODS

After converting a ghost correction problem into separate k-space data interpolation problems for even and odd phase encoding, our algorithm exploits an observation that the differential k-space data between the even and odd echoes is a Fourier transform of an underlying sparse image. Accordingly, we can construct a rank-deficient Hankel structured matrix, whose missing data can be recovered using an annihilating filter-based low rank Hankel structured matrix completion approach.

RESULTS

The proposed method was applied to EPI data for both single and multicoil acquisitions. Experimental results using in vivo data confirmed that the proposed method can completely remove ghost artifacts successfully without prescan echoes.

CONCLUSION

Owing to the discovery of the annihilating filter relationship from the intrinsic EPI image property, the proposed method successfully suppresses ghost artifacts without a prescan step. Magn Reson Med 76:1775-1789, 2016. © 2016 International Society for Magnetic Resonance in Medicine.

Acceleration of MR parameter mapping using annihilating filter-based low rank hankel matrix (ALOHA)

PURPOSE

MR parameter mapping is one of clinically valuable MR imaging techniques. However, increased scan time makes it difficult for routine clinical use. This article aims at developing an accelerated MR parameter mapping technique using annihilating filter based low-rank Hankel matrix approach (ALOHA).

THEORY

When a dynamic sequence can be sparsified using spatial wavelet and temporal Fourier transform, this results in a rank-deficient Hankel structured matrix that is constructed using weighted k-t measurements. ALOHA then utilizes the low rank matrix completion algorithm combined with a multiscale pyramidal decomposition to estimate the missing k-space data.

METHODS

Spin-echo inversion recovery and multiecho spin echo pulse sequences for T₁ and T₂ mapping, respectively, were redesigned to perform undersampling along the phase encoding direction according to Gaussian distribution. The missing k-space is reconstructed using ALOHA. Then, the parameter maps were constructed using nonlinear regression.

RESULTS

Experimental results confirmed that ALOHA outperformed the existing compressed sensing algorithms. Compared with the existing methods, the reconstruction errors appeared scattered throughout the entire images rather than exhibiting systematic distortion along edges and the parameter maps.

CONCLUSION

Given that many diagnostic errors are caused by the systematic distortion of images, ALOHA may have a great potential for clinical applications. Magn Reson Med 76:1848-1864, 2016. © 2016 International Society for Magnetic Resonance in Medicine.

Annihilating Filter-Based Low-Rank Hankel Matrix Approach for Image Inpainting

In this paper, we propose a patch-based image inpainting method using a low-rank Hankel structured matrix completion approach. The proposed method exploits the annihilation property between a shift-invariant filter and image data observed in many existing inpainting algorithms. In particular, by exploiting the commutative property of the convolution, the annihilation property results in a low-rank block Hankel structure data matrix, and the image inpainting problem becomes a low-rank structured matrix completion problem. The block Hankel structured matrices are obtained patch-by-patch to adapt to the local changes in the image statistics. To solve the structured low-rank matrix completion problem, we employ an alternating direction method of multipliers with factorization matrix initialization using the low-rank matrix fitting algorithm. As a side product of the matrix factorization, locally adaptive dictionaries can be also easily constructed. Despite the simplicity of the algorithm, the experimental results using irregularly subsampled images as well as various images with globally missing patterns showed that the proposed method outperforms existing state-of-the-art image inpainting methods.

Comparative study of iterative reconstruction algorithms for missing cone problems in optical diffraction tomography

In optical tomography, there exist certain spatial frequency components that cannot be measured due to the limited projection angles imposed by the numerical aperture of objective lenses. This limitation, often called as the missing cone problem, causes the under-estimation of refractive index (RI) values in tomograms and results in severe elongations of RI distributions along the optical axis. To address this missing cone problem, several iterative reconstruction algorithms have been introduced exploiting prior knowledge such as positivity in RI differences or edges of samples. In this paper, various existing iterative reconstruction algorithms are systematically compared for mitigating the missing cone problem in optical diffraction tomography. In particular, three representative regularization schemes, edge preserving, total variation regularization, and the Gerchberg-Papoulis algorithm, were numerically and experimentally evaluated using spherical beads as well as real biological samples; human red blood cells and hepatocyte cells. Our work will provide important guidelines for choosing the appropriate regularization in ODT.

High-speed terahertz reflection three-dimensional imaging using beam steering

High-speed terahertz (THz) reflection three-dimensional (3D) imaging is demonstrated using electronically-controlled optical sampling (ECOPS) and beam steering. ECOPS measurement is used for scanning an axial range of 7.8 mm in free space at 1 kHz scan rate while a transverse range of 100 × 100 mm(2) is scanned using beam steering instead of moving an imaging target. Telecentric f-θ lenses with axial and non-axial symmetry have been developed for beam steering. It is experimentally demonstrated that the non-axially symmetric lens has better characteristics than the axially symmetric lens. The total scan time depends on the number of points in a transverse range. For example, it takes 40 s for 200 × 200 points and 10 s for 100 × 100 points. To demonstrate the application of the imaging technique to nondestructive testing, THz 3D tomographic images of a glass fiber reinforced polymer sample with artificial internal defects have been acquired using the lenses for comparison.

Mid-long term effect of non-ablative high radiofrequency therapy on the rabbit dermal extracellular matrix

This study quantitatively investigated the postoperative effects of radiofrequency (RF) application on the normal dermal extracellular matrix (ECM) of in vivo rabbits. Postoperative effects were evaluated by histology and atomic force microscopy analysis of dermal tissues treated using three RF energy levels (10 ~ 30 W) and either a single- or multiple-pass procedure. Progressive changes in the morphology of rabbit dermal ECMs were investigated over a 30-day postoperartive period. All RF-treated groups, except for the low energy group (10 W), displayed more prominent inflammatory responses compared to the control. This inflammatory reaction was more prominent a day after application. Dermal tissues 30 days after RF application exhibited prominent myofibroblast activity associated with ECM contractile activity during wound healing in addition to chronic inflammation. A decrease in the morphology of dermal ECMs after RF application continued until seven days postoperatively. The ECM diameter increased to near baseline at 30 days postoperatively. Low energy and multi-pass applications resulted in greater collagen fibril contraction and recovery at the ultra-structural level at 30 days postoperatively than did a single high energy application.

Terahertz substance imaging by waveform shaping

Terahertz pulse shaping technique is used to adaptively design terahertz waveforms of enhanced spectral correlation to particular materials among a given set of materials. In a proof-of-principle experiment performed with a two-dimensional image target consisted of meta-materials of distinctive resonance frequencies, the as-designed waveforms are used to demonstrate terahertz substance imaging. It is hoped that this material-specific terahertz waveforms may enable single- or few-shot terahertz material classification when being used in conjunction with terahertz power measurement.

Enhancement of terahertz pulse emission by optical nanoantenna

Bridging the gap between ultrashort pulsed optical waves and terahertz (THz) waves, the THz photoconductive antenna (PCA) is a major constituent for the emission or detection of THz waves by diverse optical and electrical methods. However, THz PCA still lacks employment of advanced breakthrough technologies for high-power THz emission. Here, we report the enhancement of THz emission power by incorporating optical nanoantennas with a THz photoconductive antenna. The confinement and concentration of an optical pump beam on a photoconductive substrate can be efficiently achieved with optical nanoantennas over a high-index photoconductive substrate. Both numerical and experimental results clearly demonstrate the enhancement of THz wave emission due to high photocarrier generation at the plasmon resonance of nanoantennas. This work opens up many opportunities for diverse integrated photonic elements on a single PCA at THz and optical frequencies.

Telomerase activity in pterygeal and normal conjunctival epithelium

All pterygia have similar histologic features of solar degeneration seen in the skin such as acanthosis, keratosis, or hyperkeratosis. Although the pathogenesis of pterygia is still unclear, an association with solar exposure, in particular with UV radiation, has been reported. Telomerase activity has been found to be higher in some degenerative, precancerous, and cancerous skin lesions. We investigated telomerase activity in the epithelium and the stromal tissues of the pterygium. Pterygeal tissues were obtained from 30 patients. Telomerase activity was measured with TRAPeze-ELISA kit. Three of the 28 (10.7%) pterygeal stromal tissues demonstrated positive telomerase activity. Fourteen of the 27 (51.9%) epithelial tissues were positive in telomerase activity, whereas telomerase activity was positive in only 3 of 9 normal epithelia (33.3%). Telomerase activity in the pterygium-covered epithelium was increased as compared with that seen in the normal epithelium, but the increase was not statistically significant. In conclusion, telomerase activity was somewhat increased in pterygeal tissues. Telomerase activity may be involved in the pathogenesis of pterygium.

Non-traumatic middle meningeal aneurysm and angioma in a child

A 9-year-old boy having bronchial asthma showed fluctuation of his mental state for 14 days. EEG showed multiple spikes or irregular spike and wave complex bursts, the focus being in the left occipital region, and he was diagnosed as prolonged partial complex status epilepticus. Skull radiograms revealed the presence of radiolucent areas in the left parietal region and cerebral angiography showed a pear-shaped aneurysm and angioma in the distal part of the middle meningeal artery. Nontraumatic aneurysm of the middle meningeal artery has been rarely reported; only the reports on 2 old women with Paget disease are available now. The authors' case is the first child case.