Deep Inception-Residual Laplacian Pyramid Networks for Accurate Single-Image Super-Resolution

With exploiting contextual information over large image regions in an efficient way, the deep convolutional neural network has shown an impressive performance for single-image super-resolution (SR). In this paper, we propose a new deep convolutional network by cascading multiple well-designed inception-residual blocks within the deep Laplacian pyramid framework to progressively restore the missing high-frequency details in the low-resolution images. By optimizing our network structure, the trainable depth of our proposed network gains a significant improvement, which in turn improves super-resolving accuracy. However, the saturation and degradation of training accuracy remains a critical problem. With regard to this, we propose an effective two-stage training strategy, in which we first use the images downsampled from the ground-truth high-resolution (HR) images to pretrain the inception-residual blocks on each pyramid level with an extremely high learning rate enabled by gradient clipping, and then the original ground-truth HR images are used to fine-tune all the pretrained inception-residual blocks for obtaining our final SR models. Furthermore, we present a new loss function operating in both image space and local rank space to optimize our network for exploiting the contextual information among different output components. Extensive experiments on benchmark data sets validate that the proposed method outperforms the existing state-of-the-art SR methods in terms of the objective evaluation as well as the visual quality.

Prostate localization on daily cone-beam computed tomography images: accuracy assessment of similarity metrics

PURPOSE

To evaluate different similarity metrics (SM) using natural calcifications and observation-based measures to determine the most accurate prostate and seminal vesicle localization on daily cone-beam CT (CBCT) images.

METHODS AND MATERIALS

CBCT images of 29 patients were retrospectively analyzed; 14 patients with prostate calcifications (calcification data set) and 15 patients without calcifications (no-calcification data set). Three groups of test registrations were performed. Test 1: 70 CT/CBCT pairs from calcification dataset were registered using 17 SMs (6,580 registrations) and compared using the calcification mismatch error as an endpoint. Test 2: Using the four best SMs from Test 1, 75 CT/CBCT pairs in the no-calcification data set were registered (300 registrations). Accuracy of contour overlays was ranked visually. Test 3: For the best SM from Tests 1 and 2, accuracy was estimated using 356 CT/CBCT registrations. Additionally, target expansion margins were investigated for generating registration regions of interest.

RESULTS

Test 1-Incremental sign correlation (ISC), gradient correlation (GC), gradient difference (GD), and normalized cross correlation (NCC) showed the smallest errors (mu +/- sigma: 1.6 +/- 0.9 approximately 2.9 +/- 2.1 mm). Test 2-Two of the three reviewers ranked GC higher. Test 3-Using GC, 96% of registrations showed <3-mm error when calcifications were filtered. Errors were left/right: 0.1 +/- 0.5mm, anterior/posterior: 0.8 +/- 1.0mm, and superior/inferior: 0.5 +/- 1.1 mm. The existence of calcifications increased the success rate to 97%. Expansion margins of 4-10 mm were equally successful.

CONCLUSION

Gradient-based SMs were most accurate. Estimated error was found to be <3 mm (1.1 mm SD) in 96% of the registrations. Results suggest that the contour expansion margin should be no less than 4 mm.

A convex optimization approach for depth estimation under illumination variation

Illumination changes cause serious problems in many computer vision applications. We present a new method for addressing robust depth estimation from a stereo pair under varying illumination conditions. First, a spatially varying multiplicative model is developed to account for brightness changes induced between left and right views. The depth estimation problem, based on this model, is then formulated as a constrained optimization problem in which an appropriate convex objective function is minimized under various convex constraints modelling prior knowledge and observed information. The resulting multiconstrained optimization problem is finally solved via a parallel block iterative algorithm which offers great flexibility in the incorporation of several constraints. Experimental results on both synthetic and real stereo pairs demonstrate the good performance of our method to efficiently recover depth and illumination variation fields, simultaneously.

Automated volume analysis of head and neck lesions on CT scans using 3D level set segmentation

The authors have developed a semiautomatic system for segmentation of a diverse set of lesions in head and neck CT scans. The system takes as input an approximate bounding box, and uses a multistage level set to perform the final segmentation. A data set consisting of 69 lesions marked on 33 scans from 23 patients was used to evaluate the performance of the system. The contours from automatic segmentation were compared to both 2D and 3D gold standard contours manually drawn by three experienced radiologists. Three performance metric measures were used for the comparison. In addition, a radiologist provided quality ratings on a 1 to 10 scale for all of the automatic segmentations. For this pilot study, the authors observed that the differences between the automatic and gold standard contours were larger than the interobserver differences. However, the system performed comparably to the radiologists, achieving an average area intersection ratio of 85.4% compared to an average of 91.2% between two radiologists. The average absolute area error was 21.1% compared to 10.8%, and the average 2D distance was 1.38 mm compared to 0.84 mm between the radiologists. In addition, the quality rating data showed that, despite the very lax assumptions made on the lesion characteristics in designing the system, the automatic contours approximated many of the lesions very well.

Comparison of similarity measures for the task of template matching of masses on serial mammograms

We conducted a study to evaluate the effectiveness of twelve different similarity measures in matching the corresponding masses on temporal pairs of current and prior mammograms. To perform this comparison we implemented each of the twelve similarity measures in the final stage of our multistage registration technique for automated registration of breast lesions in serial mammograms. The multistage technique consists of three stages. In the first stage an initial fan-shape search region was estimated on the prior mammogram based on the geometrical position of the mass on the current mammogram. In the second stage, the location of the fan-shape region was refined by warping, based on an affine transformation and simplex optimization. A new refined search region was defined on the prior mammogram. In the third stage, a search for the best match between the lesion template from the current mammogram and a structure on the prior mammogram was carried out within the search region. Our data set consisted of 318 temporal pairs. We performed three experiments, using a different subset of the 318 temporal pairs for each experiment. In each experiment we further tested how the performance of the similarity measures varied as the size of the search region increased or decreased. We evaluated the twelve similarity measures based on four criteria. The first criterion was the mean Euclidean distance, which was the average distance of the true location of the mass to the location detected by the similarity measure. The second criterion was the percentage of temporal pairs that were aligned so that 50% or more of the lesion area overlapped. The third criterion was the percentage of pairs that were aligned so that 75% or more of the lesion area overlapped. The fourth and final criterion was the robustness of the similarity measure. Our results showed that three of the similarity measures, Pearson's correlation, the cosine coefficient, and Goodman and Kruskal's Gamma coefficient, provide significantly higher accuracy (p < 0.05) in the task of matching the corresponding masses on serial mammograms than the other nine similarity measures.