Weight preserving image registration for monitoring disease progression in lung CT

A two-step deep learning method for 3DCT-2DUS kidney registration during breathing

This work proposed KidneyRegNet, a novel deep registration pipeline for 3D CT and 2D U/S kidney scans of free breathing, which comprises a feature network, and a 3D-2D CNN-based registration network. The feature network has handcrafted texture feature layers to reduce the semantic gap. The registration network is an encoder-decoder structure with loss of feature-image-motion (FIM), which enables hierarchical regression at decoder layers and avoids multiple network concatenation. It was first pretrained with a retrospective dataset cum training data generation strategy and then adapted to specific patient data under unsupervised one-cycle transfer learning in onsite applications. The experiment was performed on 132 U/S sequences, 39 multiple-phase CT and 210 public single-phase CT images, and 25 pairs of CT and U/S sequences. This resulted in a mean contour distance (MCD) of 0.94 mm between kidneys on CT and U/S images and MCD of 1.15 mm on CT and reference CT images. Datasets with small transformations resulted in MCDs of 0.82 and 1.02 mm, respectively. Large transformations resulted in MCDs of 1.10 and 1.28 mm, respectively. This work addressed difficulties in 3DCT-2DUS kidney registration during free breathing via novel network structures and training strategies.

Effects of Lung Injury on Regional Aeration and Expiratory Time Constants: Insights From Four-Dimensional Computed Tomography Image Registration

Rationale: Intratidal changes in regional lung aeration, as assessed with dynamic four-dimensional computed tomography (CT; 4DCT), may indicate the processes of recruitment and derecruitment, thus portending atelectrauma during mechanical ventilation. In this study, we characterized the time constants associated with deaeration during the expiratory phase of pressure-controlled ventilation in pigs before and after acute lung injury using respiratory-gated 4DCT and image registration.

Methods: Eleven pigs were mechanically ventilated in pressure-controlled mode under baseline conditions and following an oleic acid model of acute lung injury. Dynamic 4DCT scans were acquired without interrupting ventilation. Automated segmentation of lung parenchyma was obtained by a convolutional neural network. Respiratory structures were aligned using 4D image registration. Exponential regression was performed on the time-varying CT density in each aligned voxel during exhalation, resulting in regional estimates of intratidal aeration change and deaeration time constants. Regressions were also performed for regional and total exhaled gas volume changes.

Results: Normally and poorly aerated lung regions demonstrated the largest median intratidal aeration changes during exhalation, compared to minimal changes within hyper- and non-aerated regions. Following lung injury, median time constants throughout normally aerated regions within each subject were greater than respective values for poorly aerated regions. However, parametric response mapping revealed an association between larger intratidal aeration changes and slower time constants. Lower aeration and faster time constants were observed for the dependent lung regions in the supine position. Regional gas volume changes exhibited faster time constants compared to regional density time constants, as well as better correspondence to total exhaled volume time constants.

Conclusion: Mechanical time constants based on exhaled gas volume underestimate regional aeration time constants. After lung injury, poorly aerated regions experience larger intratidal changes in aeration over shorter time scales compared to normally aerated regions. However, the largest intratidal aeration changes occur over the longest time scales within poorly aerated regions. These dynamic 4DCT imaging data provide supporting evidence for the susceptibility of poorly aerated regions to ventilator-induced lung injury, and for the functional benefits of short exhalation times during mechanical ventilation of injured lungs.

A 3D-CNN model with CT-based parametric response mapping for classifying COPD subjects

Chronic obstructive pulmonary disease (COPD) is a respiratory disorder involving abnormalities of lung parenchymal morphology with different severities. COPD is assessed by pulmonary-function tests and computed tomography-based approaches. We introduce a new classification method for COPD grouping based on deep learning and a parametric-response mapping (PRM) method. We extracted parenchymal functional variables of functional small airway disease percentage (fSAD%) and emphysema percentage (Emph%) with an image registration technique, being provided as input parameters of 3D convolutional neural network (CNN). The integrated 3D-CNN and PRM (3D-cPRM) achieved a classification accuracy of 89.3% and a sensitivity of 88.3% in five-fold cross-validation. The prediction accuracy of the proposed 3D-cPRM exceeded those of the 2D model and traditional 3D CNNs with the same neural network, and was comparable to that of 2D pretrained PRM models. We then applied a gradient-weighted class activation mapping (Grad-CAM) that highlights the key features in the CNN learning process. Most of the class-discriminative regions appeared in the upper and middle lobes of the lung, consistent with the regions of elevated fSAD% and Emph% in COPD subjects. The 3D-cPRM successfully represented the parenchymal abnormalities in COPD and matched the CT-based diagnosis of COPD.

N-Phase Local Expansion Ratio for Characterizing Out-of-Phase Lung Ventilation

Out-of-phase ventilation occurs when local regions of the lung reach their maximum or minimum volumes at breathing phases other than the global end inhalation or exhalation phases. This paper presents the N-phase local expansion ratio (LER _N ) as a surrogate for lung ventilation. A common approach to estimate lung ventilation is to use image registration to align the end exhalation and inhalation 3DCT images and then analyze the resulting correspondence map. This 2-phase local expansion ratio (LER₂) is limited because it ignores out-of-phase ventilation and thus may underestimate local lung ventilation. To overcome this limitation, LER _N measures the maximum ratio of local expansion and contraction over the entire breathing cycle. Comparing LER₂ to LER _N provides a means for detecting and characterizing locations of the lung that experience out-of-phase ventilation. We present a novel in-phase/out-of-phase ventilation (IOV) function plot to visualize and measure the amount of high-function IOV that occurs during a breathing cycle. Treatment planning 4DCT scans collected during coached breathing from 32 human subjects with lung cancer were analyzed in this study. Results show that out-of-phase breathing occurred in all subjects and that the spatial distribution of out-of-phase ventilation varied from subject to subject. For the 32 subjects analyzed, 50% of the out-of-phase regions on average were mislabeled as low-function by LER₂ (high-function threshold of 1.1, IOV threshold of 1.05). 4DCT and Xenon-enhanced CT of four sheep showed that LER₈ is more accurate than LER₂ for measuring lung ventilation.

Quantifying Regional Lung Deformation Using Four-Dimensional Computed Tomography: A Comparison of Conventional and Oscillatory Ventilation

Mechanical ventilation strategies that reduce the heterogeneity of regional lung stress and strain may reduce the risk of ventilator-induced lung injury (VILI). In this study, we used registration of four-dimensional computed tomographic (4DCT) images to assess regional lung aeration and deformation in 10 pigs under baseline conditions and following acute lung injury induced with oleic acid. CT images were obtained via dynamic axial imaging (Siemens SOMATOM Force) during conventional pressure-controlled mechanical ventilation (CMV), as well as high-frequency and multi-frequency oscillatory ventilation modalities (HFOV and MFOV, respectively). Our results demonstrate that oscillatory modalities reduce intratidal strain throughout the lung in comparison to conventional ventilation, as well as the spatial gradients of dynamic strain along the dorsal-ventral axis. Harmonic distortion of parenchymal deformation was observed during HFOV with a single discrete sinusoid delivered at the airway opening, suggesting inherent mechanical nonlinearity of the lung tissues. MFOV may therefore provide improved lung-protective ventilation by reducing strain magnitudes and spatial gradients of strain compared to either CMV or HFOV.

Recent Advances in Computed Tomography Imaging in Chronic Obstructive Pulmonary Disease

Lung imaging is increasingly being used to diagnose, quantify, and phenotype chronic obstructive pulmonary disease (COPD). Although spirometry is the gold standard for the diagnosis of COPD and for severity staging, the role of computed tomography (CT) imaging has expanded in both clinical practice and research. COPD is a heterogeneous disease with considerable variability in clinical features, radiographic disease, progression, and outcomes. Recent studies have examined the utility of CT imaging in enhancing diagnostic certainty, improving phenotyping, predicting disease progression and prognostication, selecting patients for intervention, and also in furthering our understanding of the complex pathophysiology of this disease. Multiple CT metrics show promise for use as imaging biomarkers in COPD.

Topology preserving non-rigid image registration using time-varying elasticity model for MRI brain volumes

In this paper, we present a new non-rigid image registration method that imposes a topology preservation constraint on the deformation. We propose to incorporate the time varying elasticity model into the deformable image matching procedure and constrain the Jacobian determinant of the transformation over the entire image domain. The motion of elastic bodies is governed by a hyperbolic partial differential equation, generally termed as elastodynamics wave equation, which we propose to use as a deformation model. We carried out clinical image registration experiments on 3D magnetic resonance brain scans from IBSR database. The results of the proposed registration approach in terms of Kappa index and relative overlap computed over the subcortical structures were compared against the existing topology preserving non-rigid image registration methods and non topology preserving variant of our proposed registration scheme. The Jacobian determinant maps obtained with our proposed registration method were qualitatively and quantitatively analyzed. The results demonstrated that the proposed scheme provides good registration accuracy with smooth transformations, thereby guaranteeing the preservation of topology.

Registration of lung CT images acquired in different respiratory ranges with 4DCT and HRCT

Pulmonary image registration is challenging because of the unique structure of the lung, its high deformability and its non-uniform intensity change with breathing. In the present work we propose a new method for pulmonary image registration, based on the reconstruction and the combination of the main pulmonary structures to modify parenchyma intensity prior to the application of the registration algorithm. The algorithm has been applied to both four dimensional CT and multi-volume high resolution CT demonstrating an increased accuracy of the results with the application of the pulmonary structure enhancement, evaluated both on landmarks distance in 4DCT and structures' surface distance in HRCT.

Computing global minimizers to a constrained B-spline image registration problem from optimal l1 perturbations to block match data

PURPOSE

Block matching is a well-known strategy for estimating corresponding voxel locations between a pair of images according to an image similarity metric. Though robust to issues such as image noise and large magnitude voxel displacements, the estimated point matches are not guaranteed to be spatially accurate. However, the underlying optimization problem solved by the block matching procedure is similar in structure to the class of optimization problem associated with B-spline based registration methods. By exploiting this relationship, the authors derive a numerical method for computing a global minimizer to a constrained B-spline registration problem that incorporates the robustness of block matching with the global smoothness properties inherent to B-spline parameterization.

METHODS

The method reformulates the traditional B-spline registration problem as a basis pursuit problem describing the minimall1-perturbation to block match pairs required to produce a B-spline fitting error within a given tolerance. The sparsity pattern of the optimal perturbation then defines a voxel point cloud subset on which the B-spline fit is a global minimizer to a constrained variant of the B-spline registration problem. As opposed to traditional B-spline algorithms, the optimization step involving the actual image data is addressed by block matching.

RESULTS

The performance of the method is measured in terms of spatial accuracy using ten inhale/exhale thoracic CT image pairs (available for download atwww.dir-lab.com) obtained from the COPDgene dataset and corresponding sets of expert-determined landmark point pairs. The results of the validation procedure demonstrate that the method can achieve a high spatial accuracy on a significantly complex image set.

CONCLUSIONS

The proposed methodology is demonstrated to achieve a high spatial accuracy and is generalizable in that in can employ any displacement field parameterization described as a least squares fit to block match generated estimates. Thus, the framework allows for a wide range of image similarity block match metric and physical modeling combinations.

Towards local progression estimation of pulmonary emphysema using CT

PURPOSE

Whole lung densitometry on chest CT images is an accepted method for measuring tissue destruction in patients with pulmonary emphysema in clinical trials. Progression measurement is required for evaluation of change in health condition and the effect of drug treatment. Information about the location of emphysema progression within the lung may be important for the correct interpretation of drug efficacy, or for determining a treatment plan. The purpose of this study is therefore to develop and validate methods that enable the local measurement of lung density changes, which requires proper modeling of the effect of respiration on density.

METHODS

Four methods, all based on registration of baseline and follow-up chest CT scans, are compared. The first naïve method subtracts registered images. The second employs the so-called dry sponge model, where volume correction is performed using the determinant of the Jacobian of the transformation. The third and the fourth introduce a novel adaptation of the dry sponge model that circumvents its constant-mass assumption, which is shown to be invalid. The latter two methods require a third CT scan at a different inspiration level to estimate the patient-specific density-volume slope, where one method employs a global and the other a local slope. The methods were validated on CT scans of a phantom mimicking the lung, where mass and volume could be controlled. In addition, validation was performed on data of 21 patients with pulmonary emphysema.

RESULTS

The image registration method was optimized leaving a registration error below half the slice increment (median 1.0 mm). The phantom study showed that the locally adapted slope model most accurately measured local progression. The systematic error in estimating progression, as measured on the phantom data, was below 2 gr/l for a 70 ml (6%) volume difference, and 5 gr/l for a 210 ml (19%) difference, if volume correction was applied. On the patient data an underlying linearity assumption relating lung volume change with density change was shown to hold (fitR(2) = 0.94), and globalized versions of the local models are consistent with global results (R(2) of 0.865 and 0.882 for the two adapted slope models, respectively).

CONCLUSIONS

In conclusion, image matching and subsequent analysis of differences according to the proposed lung models (i) has good local registration accuracy on patient data, (ii) effectively eliminates a dependency on inspiration level at acquisition time, (iii) accurately predicts progression in phantom data, and (iv) is reasonably consistent with global results in patient data. It is therefore a potential future tool for assessing local emphysema progression in drug evaluation trials and in clinical practice.

Parametric response mapping as an indicator of bronchiolitis obliterans syndrome after hematopoietic stem cell transplantation

The management of bronchiolitis obliterans syndrome (BOS) after hematopoietic cell transplantation presents many challenges, both diagnostically and therapeutically. We developed a computed tomography (CT) voxel-wise methodology termed parametric response mapping (PRM) that quantifies normal parenchyma, functional small airway disease (PRM(fSAD)), emphysema, and parenchymal disease as relative lung volumes. We now investigate the use of PRM as an imaging biomarker in the diagnosis of BOS. PRM was applied to CT data from 4 patient cohorts: acute infection (n = 11), BOS at onset (n = 34), BOS plus infection (n = 9), and age-matched, nontransplant control subjects (n = 23). Pulmonary function tests and bronchoalveolar lavage were used for group classification. Mean values for PRM(fSAD) were significantly greater in patients with BOS (38% ± 2%) when compared with those with infection alone (17% ± 4%, P < .0001) and age-matched control subjects (8.4% ± 1%, P < .0001). Patients with BOS had similar PRM(fSAD) profiles, whether a concurrent infection was present or not. An optimal cut-point for PRM(fSAD) of 28% of the total lung volume was identified, with values >28% highly indicative of BOS occurrence. PRM may provide a major advance in our ability to identify the small airway obstruction that characterizes BOS, even in the presence of concurrent infection.

Detection of time-varying structures by large deformation diffeomorphic metric mapping to aid reading of high-resolution CT images of the lung

Objectives

To evaluate the accuracy of advanced non-linear registration of serial lung Computed Tomography (CT) images using Large Deformation Diffeomorphic Metric Mapping (LDDMM).

Methods

Fifteen cases of lung cancer with serial lung CT images (interval: 62.2±26.9 days) were used. After affine transformation, three dimensional, non-linear volume registration was conducted using LDDMM with or without cascading elasticity control. Registration accuracy was evaluated by measuring the displacement of landmarks placed on vessel bifurcations for each lung segment. Subtraction images and Jacobian color maps, calculated from the transformation matrix derived from image warping, were generated, which were used to evaluate time-course changes of the tumors.

Results

The average displacement of landmarks was 0.02±0.16 mm and 0.12±0.60 mm for proximal and distal landmarks after LDDMM transformation with cascading elasticity control, which was significantly smaller than 3.11±2.47 mm and 3.99±3.05 mm, respectively, after affine transformation. Emerged or vanished nodules were visualized on subtraction images, and enlarging or shrinking nodules were displayed on Jacobian maps enabled by highly accurate registration of the nodules using LDDMM. However, some residual misalignments were observed, even with non-linear transformation when substantial changes existed between the image pairs.

Conclusions

LDDMM provides accurate registration of serial lung CT images, and temporal subtraction images with Jacobian maps help radiologists to find changes in pulmonary nodules.

Improving Intensity-Based Lung CT Registration Accuracy Utilizing Vascular Information

Accurate pulmonary image registration is a challenging problem when the lungs have a deformation with large distance. In this work, we present a nonrigid volumetric registration algorithm to track lung motion between a pair of intrasubject CT images acquired at different inflation levels and introduce a new vesselness similarity cost that improves intensity-only registration. Volumetric CT datasets from six human subjects were used in this study. The performance of four intensity-only registration algorithms was compared with and without adding the vesselness similarity cost function. Matching accuracy was evaluated using landmarks, vessel tree, and fissure planes. The Jacobian determinant of the transformation was used to reveal the deformation pattern of local parenchymal tissue. The average matching error for intensity-only registration methods was on the order of 1 mm at landmarks and 1.5 mm on fissure planes. After adding the vesselness preserving cost function, the landmark and fissure positioning errors decreased approximately by 25% and 30%, respectively. The vesselness cost function effectively helped improve the registration accuracy in regions near thoracic cage and near the diaphragm for all the intensity-only registration algorithms tested and also helped produce more consistent and more reliable patterns of regional tissue deformation.

Computed tomography-based biomarker provides unique signature for diagnosis of COPD phenotypes and disease progression

Chronic obstructive pulmonary disease (COPD) is increasingly being recognized as a highly heterogeneous disorder, composed of varying pathobiology. Accurate detection of COPD subtypes by image biomarkers are urgently needed to enable individualized treatment thus improving patient outcome. We adapted the Parametric Response Map (PRM), a voxel-wise image analysis technique, for assessing COPD phenotype. We analyzed whole lung CT scans of 194 COPD individuals acquired at inspiration and expiration from the COPDGene Study. PRM identified the extent of functional small airways disease (fSAD) and emphysema as well as provided CT-based evidence that supports the concept that fSAD precedes emphysema with increasing COPD severity. PRM is a versatile imaging biomarker capable of diagnosing disease extent and phenotype, while providing detailed spatial information of disease distribution and location. PRMs ability to differentiate between specific COPD phenotypes will allow for more accurate diagnosis of individual patients complementing standard clinical techniques.

Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge

EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.

Probabilistic elastography: estimating lung elasticity

We formulate registration-based elastography in a probabilistic framework and apply it to study lung elasticity in the presence of emphysematous and fibrotic tissue. The elasticity calculations are based on a Finite Element discretization of a linear elastic biomechanical model. We marginalize over the boundary conditions (deformation) of the biomechanical model to determine the posterior distribution over elasticity parameters. Image similarity is included in the likelihood, an elastic prior is included to constrain the boundary conditions, while a Markov model is used to spatially smooth the inhomogeneous elasticity. We use a Markov Chain Monte Carlo (MCMC) technique to characterize the posterior distribution over elasticity from which we extract the most probable elasticity as well as the uncertainty of this estimate. Even though registration-based lung elastography with inhomogeneous elasticity is challenging due the problem's highly underdetermined nature and the sparse image information available in lung CT, we show promising preliminary results on estimating lung elasticity contrast in the presence of emphysematous and fibrotic tissue.

Early detection of emphysema progression

Emphysema is one of the most widespread diseases in subjects with smoking history. The gold standard method for estimating the severity of emphysema is a lung function test, such as forced expiratory volume in first second (FEV1). However, several clinical studies showed that chest CT scans offer more sensitive estimates of emphysema progression. The standard CT densitometric score of emphysema is the relative area of voxels below a threshold (RA). The RA score is a global measurement and reflects the overall emphysema progression. In this work, we propose a framework for estimation of local emphysema progression from longitudinal chest CT scans. First, images are registered to a common system of coordinates and then local image dissimilarities are computed in corresponding anatomical locations. Finally, the obtained dissimilarity representation is converted into a single emphysema progression score. We applied the proposed algorithm on 27 patients with severe emphysema with CT scans acquired five time points, at baseline, after 3, after 12, after 21 and after 24 or 30 months. The results showed consistent emphysema progression with time and the overall progression score correlates significantly with the increase in RA score.

Location registration and recognition (LRR) for serial analysis of nodules in lung CT scans

In the clinical workflow for lung cancer management, the comparison of nodules between CT scans from subsequent visits by a patient is necessary for timely classification of pulmonary nodules into benign and malignant and for analyzing nodule growth and response to therapy. The algorithm described in this paper takes (a) two temporally-separated CT scans, I(1) and I(2), and (b) a series of nodule locations in I(1), and for each location it produces an affine transformation that maps the locations and their immediate neighborhoods from I(1) to I(2). It does this without deformable registration and without initialization by global affine registration. Requiring the nodule locations to be specified in only one volume provides the clinician more flexibility in investigating the condition of the lung. The algorithm uses a combination of feature extraction, indexing, refinement, and decision processes. Together, these processes essentially "recognize" the neighborhoods. We show on lung CT scans that our technique works at near interactive speed and that the median alignment error of 134 nodules is 1.70mm compared to the error 2.14mm of the Diffeomorphic Demons algorithm, and to the error 3.57mm of the global nodule registration with local refinement. We demonstrate on the alignment of 250 nodules, that the algorithm is robust to changes caused by cancer progression and differences in breathing states, scanning procedures, and patient positioning. Our algorithm may be used both for diagnosis and treatment monitoring of lung cancer. Because of the generic design of the algorithm, it might also be used in other applications that require fast and accurate mapping of regions.

Mass preserving nonrigid registration of CT lung images using cubic B-spline

The authors propose a nonrigid image registration approach to align two computed-tomography (CT)-derived lung datasets acquired during breath-holds at two inspiratory levels when the image distortion between the two volumes is large. The goal is to derive a three-dimensional warping function that can be used in association with computational fluid dynamics studies. In contrast to the sum of squared intensity difference (SSD), a new similarity criterion, the sum of squared tissue volume difference (SSTVD), is introduced to take into account changes in reconstructed Hounsfield units (scaled attenuation coefficient, HU) with inflation. This new criterion aims to minimize the local tissue volume difference within the lungs between matched regions, thus preserving the tissue mass of the lungs if the tissue density is assumed to be relatively constant. The local tissue volume difference is contributed by two factors: Change in the regional volume due to the deformation and change in the fractional tissue content in a region due to inflation. The change in the regional volume is calculated from the Jacobian value derived from the warping function and the change in the fractional tissue content is estimated from reconstructed HU based on quantitative CT measures. A composite of multilevel B-spline is adopted to deform images and a sufficient condition is imposed to ensure a one-to-one mapping even for a registration pair with large volume difference. Parameters of the transformation model are optimized by a limited-memory quasi-Newton minimization approach in a multiresolution framework. To evaluate the effectiveness of the new similarity measure, the authors performed registrations for six lung volume pairs. Over 100 annotated landmarks located at vessel bifurcations were generated using a semiautomatic system. The results show that the SSTVD method yields smaller average landmark errors than the SSD method across all six registration pairs.