ProRefiner: an entropy-based refining strategy for inverse protein folding with global graph attention

Inverse Protein Folding (IPF) is an important task of protein design, which aims to design sequences compatible with a given backbone structure. Despite the prosperous development of algorithms for this task, existing methods tend to rely on noisy predicted residues located in the local neighborhood when generating sequences. To address this limitation, we propose an entropy-based residue selection method to remove noise in the input residue context. Additionally, we introduce ProRefiner, a memory-efficient global graph attention model to fully utilize the denoised context. Our proposed method achieves state-of-the-art performance on multiple sequence design benchmarks in different design settings. Furthermore, we demonstrate the applicability of ProRefiner in redesigning Transposon-associated transposase B, where six out of the 20 variants we propose exhibit improved gene editing activity.

Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark

PURPOSE

Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill.

METHODS

To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment.

RESULTS

F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team).

CONCLUSION

Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.

Federated learning enables big data for rare cancer boundary detection

Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing.

DLTTA: Dynamic Learning Rate for Test-Time Adaptation on Cross-Domain Medical Images

Test-time adaptation (TTA) has increasingly been an important topic to efficiently tackle the cross-domain distribution shift at test time for medical images from different institutions. Previous TTA methods have a common limitation of using a fixed learning rate for all the test samples. Such a practice would be sub-optimal for TTA, because test data may arrive sequentially therefore the scale of distribution shift would change frequently. To address this problem, we propose a novel dynamic learning rate adjustment method for test-time adaptation, called DLTTA, which dynamically modulates the amount of weights update for each test image to account for the differences in their distribution shift. Specifically, our DLTTA is equipped with a memory bank based estimation scheme to effectively measure the discrepancy of a given test sample. Based on this estimated discrepancy, a dynamic learning rate adjustment strategy is then developed to achieve a suitable degree of adaptation for each test sample. The effectiveness and general applicability of our DLTTA is extensively demonstrated on three tasks including retinal optical coherence tomography (OCT) segmentation, histopathological image classification, and prostate 3D MRI segmentation. Our method achieves effective and fast test-time adaptation with consistent performance improvement over current state-of-the-art test-time adaptation methods. Code is available at https://github.com/med-air/DLTTA.

Learning With Privileged Multimodal Knowledge for Unimodal Segmentation

Multimodal learning usually requires a complete set of modalities during inference to maintain performance. Although training data can be well-prepared with high-quality multiple modalities, in many cases of clinical practice, only one modality can be acquired and important clinical evaluations have to be made based on the limited single modality information. In this work, we propose a privileged knowledge learning framework with the 'Teacher-Student' architecture, in which the complete multimodal knowledge that is only available in the training data (called privileged information) is transferred from a multimodal teacher network to a unimodal student network, via both a pixel-level and an image-level distillation scheme. Specifically, for the pixel-level distillation, we introduce a regularized knowledge distillation loss which encourages the student to mimic the teacher's softened outputs in a pixel-wise manner and incorporates a regularization factor to reduce the effect of incorrect predictions from the teacher. For the image-level distillation, we propose a contrastive knowledge distillation loss which encodes image-level structured information to enrich the knowledge encoding in combination with the pixel-level distillation. We extensively evaluate our method on two different multi-class segmentation tasks, i.e., cardiac substructure segmentation and brain tumor segmentation. Experimental results on both tasks demonstrate that our privileged knowledge learning is effective in improving unimodal segmentation and outperforms previous methods.

Dual-Teacher++: Exploiting Intra-Domain and Inter-Domain Knowledge With Reliable Transfer for Cardiac Segmentation

Annotation scarcity is a long-standing problem in medical image analysis area. To efficiently leverage limited annotations, abundant unlabeled data are additionally exploited in semi-supervised learning, while well-established cross-modality data are investigated in domain adaptation. In this paper, we aim to explore the feasibility of concurrently leveraging both unlabeled data and cross-modality data for annotation-efficient cardiac segmentation. To this end, we propose a cutting-edge semi-supervised domain adaptation framework, namely Dual-Teacher++. Besides directly learning from limited labeled target domain data (e.g., CT) via a student model adopted by previous literature, we design novel dual teacher models, including an inter-domain teacher model to explore cross-modality priors from source domain (e.g., MR) and an intra-domain teacher model to investigate the knowledge beneath unlabeled target domain. In this way, the dual teacher models would transfer acquired inter- and intra-domain knowledge to the student model for further integration and exploitation. Moreover, to encourag reliable dual-domain knowledge transfer, we enhance the inter-domain knowledge transfer on the samples with higher similarity to target domain after appearance alignment, and also strengthen intra-domain knowledge transfer of unlabeled target data with higher prediction confidence. In this way, the student model can obtain reliable dual-domain knowledge and yield improved performance on target domain data. We extensively evaluated the feasibility of our method on the MM-WHS 2017 challenge dataset. The experiments have demonstrated the superiority of our framework over other semi-supervised learning and domain adaptation methods. Moreover, our performance gains could be yielded in bidirections, i.e., adapting from MR to CT, and from CT to MR. Our code will be available at https://github.com/kli-lalala/Dual-Teacher-.

Multitask Feature Learning Meets Robust Tensor Decomposition for EEG Classification

In this article, we study a tensor-based multitask learning (MTL) method for classification. Taking into account the fact that in many real-world applications, the given training samples are limited and can be inherently arranged into multidimensional arrays (tensors), we are motivated by the advantages of MTL, where the shared structural information among related tasks can be leveraged to produce better generalization performance. We propose a regularized tensor-based MTL method for joint feature selection and classification. For feature selection, we employ the Fisher discriminant criterion to both select discriminative features and control the within-class nonstationarity. For classification, we take both shared and task-specific structural information into consideration. We decompose the regression tensor for each task into a linear combination of a shared tensor and a task-specific tensor and propose a composite tensor norm. Specifically, we use the scaled latent trace norm for regularizing the shared tensor and the l₁ -norm for task-specific tensor. Further, we give a computationally efficient optimization algorithm based on the alternating direction method of multipliers (ADMMs) to tackle the joint learning of discriminative features and multitask classification. The experimental results on real electroencephalography (EEG) datasets demonstrate the superiority of our method over the state-of-the-art techniques.

Semi-Supervised Medical Image Classification With Relation-Driven Self-Ensembling Model

Training deep neural networks usually requires a large amount of labeled data to obtain good performance. However, in medical image analysis, obtaining high-quality labels for the data is laborious and expensive, as accurately annotating medical images demands expertise knowledge of the clinicians. In this paper, we present a novel relation-driven semi-supervised framework for medical image classification. It is a consistency-based method which exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations, and leverages a self-ensembling model to produce high-quality consistency targets for the unlabeled data. Considering that human diagnosis often refers to previous analogous cases to make reliable decisions, we introduce a novel sample relation consistency (SRC) paradigm to effectively exploit unlabeled data by modeling the relationship information among different samples. Superior to existing consistency-based methods which simply enforce consistency of individual predictions, our framework explicitly enforces the consistency of semantic relation among different samples under perturbations, encouraging the model to explore extra semantic information from unlabeled data. We have conducted extensive experiments to evaluate our method on two public benchmark medical image classification datasets, i.e., skin lesion diagnosis with ISIC 2018 challenge and thorax disease classification with ChestX-ray14. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.

MS-Net: Multi-Site Network for Improving Prostate Segmentation With Heterogeneous MRI Data

Automated prostate segmentation in MRI is highly demanded for computer-assisted diagnosis. Recently, a variety of deep learning methods have achieved remarkable progress in this task, usually relying on large amounts of training data. Due to the nature of scarcity for medical images, it is important to effectively aggregate data from multiple sites for robust model training, to alleviate the insufficiency of single-site samples. However, the prostate MRIs from different sites present heterogeneity due to the differences in scanners and imaging protocols, raising challenges for effective ways of aggregating multi-site data for network training. In this paper, we propose a novel multi-site network (MS-Net) for improving prostate segmentation by learning robust representations, leveraging multiple sources of data. To compensate for the inter-site heterogeneity of different MRI datasets, we develop Domain-Specific Batch Normalization layers in the network backbone, enabling the network to estimate statistics and perform feature normalization for each site separately. Considering the difficulty of capturing the shared knowledge from multiple datasets, a novel learning paradigm, i.e., Multi-site-guided Knowledge Transfer, is proposed to enhance the kernels to extract more generic representations from multi-site data. Extensive experiments on three heterogeneous prostate MRI datasets demonstrate that our MS-Net improves the performance across all datasets consistently, and outperforms state-of-the-art methods for multi-site learning.

Unpaired Multi-Modal Segmentation via Knowledge Distillation

Multi-modal learning is typically performed with network architectures containing modality-specific layers and shared layers, utilizing co-registered images of different modalities. We propose a novel learning scheme for unpaired cross-modality image segmentation, with a highly compact architecture achieving superior segmentation accuracy. In our method, we heavily reuse network parameters, by sharing all convolutional kernels across CT and MRI, and only employ modality-specific internal normalization layers which compute respective statistics. To effectively train such a highly compact model, we introduce a novel loss term inspired by knowledge distillation, by explicitly constraining the KL-divergence of our derived prediction distributions between modalities. We have extensively validated our approach on two multi-class segmentation problems: i) cardiac structure segmentation, and ii) abdominal organ segmentation. Different network settings, i.e., 2D dilated network and 3D U-net, are utilized to investigate our method's general efficacy. Experimental results on both tasks demonstrate that our novel multi-modal learning scheme consistently outperforms single-modal training and previous multi-modal approaches.

Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation

Unsupervised domain adaptation has increasingly gained interest in medical image computing, aiming to tackle the performance degradation of deep neural networks when being deployed to unseen data with heterogeneous characteristics. In this work, we present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA), to effectively adapt a segmentation network to an unlabeled target domain. Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives. In particular, we simultaneously transform the appearance of images across domains and enhance domain-invariance of the extracted features by leveraging adversarial learning in multiple aspects and with a deeply supervised mechanism. The feature encoder is shared between both adaptive perspectives to leverage their mutual benefits via end-to-end learning. We have extensively evaluated our method with cardiac substructure segmentation and abdominal multi-organ segmentation for bidirectional cross-modality adaptation between MRI and CT images. Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images, and outperforms the state-of-the-art domain adaptation approaches by a large margin.

Assessing performance of augmented reality-based neurosurgical training

This paper presents a novel augmented reality (AR)-based neurosurgical training simulator which provides a very natural way for surgeons to learn neurosurgical skills. Surgical simulation with bimanual haptic interaction is integrated in this work to provide a simulated environment for users to achieve holographic guidance for pre-operative training. To achieve the AR guidance, the simulator should precisely overlay the 3D anatomical information of the hidden target organs in the patients in real surgery. In this regard, the patient-specific anatomy structures are reconstructed from segmented brain magnetic resonance imaging. We propose a registration method for precise mapping of the virtual and real information. In addition, the simulator provides bimanual haptic interaction in a holographic environment to mimic real brain tumor resection. In this study, we conduct AR-based guidance validation and a user study on the developed simulator, which demonstrate the high accuracy of our AR-based neurosurgery simulator, as well as the AR guidance mode’s potential to improve neurosurgery by simplifying the operation, reducing the difficulty of the operation, shortening the operation time, and increasing the precision of the operation.

Online Subspace Learning from Gradient Orientations for Robust Image Alignment

Robust and efficient image alignment remains a challenging task, due to the massiveness of images, great illumination variations between images, partial occlusion, and corruption. To address these challenges, we propose an online image alignment method via subspace learning from image gradient orientations (IGOs). The proposed method integrates the subspace learning, transformed the IGO reconstruction and image alignment into a unified online framework, which is robust for aligning images with severe intensity distortions. Our method is motivated by a principal component analysis (PCA) from gradient orientations that provides more reliable low-dimensional subspace than that from pixel intensities. Instead of processing in the intensity-domain-like conventional methods, we seek alignment in the IGO domain, such that the aligned IGO of the newly arrived image can be decomposed as the sum of a sparse error and a linear composition of the IGO-PCA basis learned from previously well-aligned ones. The optimization problem is tackled by an iterative linearization that minimizes the l₁ -norm of the sparse error. Furthermore, the IGO-PCA basis is adaptively updated based on incremental thin singular value decomposition, which takes the shift of IGO mean into consideration. The efficacy of the proposed method is validated on the extensive challenging datasets through image alignment, medical atlas construction, and face recognition. The experimental results demonstrate that our algorithm provides more illumination- and occlusion-robust image alignment than the state-of-the-art methods.

Online Robust Projective Dictionary Learning: Shape Modeling for MR-TRUS Registration

Robust and effective shape prior modeling from a set of training data remains a challenging task, since the shape variation is complicated, and shape models should preserve local details as well as handle shape noises. To address these challenges, a novel robust projective dictionary learning (RPDL) scheme is proposed in this paper. Specifically, the RPDL method integrates the dimension reduction and dictionary learning into a unified framework for shape prior modeling, which can not only learn a robust and representative dictionary with the energy preservation of the training data, but also reduce the dimensionality and computational cost via the subspace learning. In addition, the proposed RPDL algorithm is regularized by using the norm to handle the outliers and noises, and is embedded in an online framework so that of memory and time efficiency. The proposed method is employed to model prostate shape prior for the application of magnetic resonance transrectal ultrasound registration. The experimental results demonstrate that our method provides more accurate and robust shape modeling than the state-of-the-art methods do. The proposed RPDL method is applicable for modeling other organs, and hence, a general solution for the problem of shape prior modeling.

Personalized heterogeneous deformable model for fast volumetric registration

BACKGROUND

Biomechanical deformable volumetric registration can help improve safety of surgical interventions by ensuring the operations are extremely precise. However, this technique has been limited by the accuracy and the computational efficiency of patient-specific modeling.

METHODS

This study presents a tissue-tissue coupling strategy based on penalty method to model the heterogeneous behavior of deformable body, and estimate the personalized tissue-tissue coupling parameters in a data-driven way. Moreover, considering that the computational efficiency of biomechanical model is highly dependent on the mechanical resolution, a practical coarse-to-fine scheme is proposed to increase runtime efficiency. Particularly, a detail enrichment database is established in an offline fashion to represent the mapping relationship between the deformation results of high-resolution hexahedral mesh extracted from the raw medical data and a newly constructed low-resolution hexahedral mesh. At runtime, the mechanical behavior of human organ under interactions is simulated with this low-resolution hexahedral mesh, then the microstructures are synthesized in virtue of the detail enrichment database.

RESULTS

The proposed method is validated by volumetric registration in an abdominal phantom compression experiments. Our personalized heterogeneous deformable model can well describe the coupling effects between different tissues of the phantom. Compared with high-resolution heterogeneous deformable model, the low-resolution deformable model with our detail enrichment database can achieve 9.4× faster, and the average target registration error is 3.42 mm, which demonstrates that the proposed method shows better volumetric registration performance than state-of-the-art.

CONCLUSIONS

Our framework can well balance the precision and efficiency, and has great potential to be adopted in the practical augmented reality image-guided robotic systems.

Towards Personalized Statistical Deformable Model and Hybrid Point Matching for Robust MR-TRUS Registration

Registration and fusion of magnetic resonance (MR) and 3D transrectal ultrasound (TRUS) images of the prostate gland can provide high-quality guidance for prostate interventions. However, accurate MR-TRUS registration remains a challenging task, due to the great intensity variation between two modalities, the lack of intrinsic fiducials within the prostate, the large gland deformation caused by the TRUS probe insertion, and distinctive biomechanical properties in patients and prostate zones. To address these challenges, a personalized model-to-surface registration approach is proposed in this study. The main contributions of this paper can be threefold. First, a new personalized statistical deformable model (PSDM) is proposed with the finite element analysis and the patient-specific tissue parameters measured from the ultrasound elastography. Second, a hybrid point matching method is developed by introducing the modality independent neighborhood descriptor (MIND) to weight the Euclidean distance between points to establish reliable surface point correspondence. Third, the hybrid point matching is further guided by the PSDM for more physically plausible deformation estimation. Eighteen sets of patient data are included to test the efficacy of the proposed method. The experimental results demonstrate that our approach provides more accurate and robust MR-TRUS registration than state-of-the-art methods do. The averaged target registration error is 1.44 mm, which meets the clinical requirement of 1.9 mm for the accurate tumor volume detection. It can be concluded that the presented method can effectively fuse the heterogeneous image information in the elastography, MR, and TRUS to attain satisfactory image alignment performance.

Fully automatic and nonparametric quantification of adipose tissue in fat-water separation MR imaging

Despite increasing demand and research efforts, currently there is no consensus on the protocol for automated and reliable quantification of adipose tissue (AT) and visceral adipose tissue (VAT) using MRI. The purpose of this study was to propose a novel computational method with enhanced objectiveness for the quantification of AT and VAT in fat-water separation MRI. 3T data from IDEAL were acquired for the fat-water separation. Fat tissues were separated from nonfat regions (background air, bone, water, and other nonfat tissues) using K-means clustering (K = 2). From the binary fat mask, arm regions were separated from body based on the relative size of connected component. AT was obtained from the binary body fat mask. With the initial contour as the outer boundary of body fat, the subcutaneous adipose tissue (SAT) and VAT were separated using deformable model driven by a specifically generated deformation field pointing to the inner boundary of SAT. The proposed method was tested on 16 patients with dyslipidemia and evaluated by comparing the correlation with semi-automatic segmentation results. Good robustness was also observed in the proposed method from the Bland-Altman plots. Compared to other established fat segmentation methods, the proposed method is highly objective for fat-water separation MRI with minimal variability induced by subjective parameter settings.

Standard plane localization in ultrasound by radial component model and selective search

Acquisition of the standard plane is crucial for medical ultrasound diagnosis. However, this process requires substantial experience and a thorough knowledge of human anatomy. Therefore it is very challenging for novices and even time consuming for experienced examiners. We proposed a hierarchical, supervised learning framework for automatically detecting the standard plane from consecutive 2-D ultrasound images. We tested this technique by developing a system that localizes the fetal abdominal standard plane from ultrasound video by detecting three key anatomical structures: the stomach bubble, umbilical vein and spine. We first proposed a novel radial component-based model to describe the geometric constraints of these key anatomical structures. We then introduced a novel selective search method which exploits the vessel probability algorithm to produce probable locations for the spine and umbilical vein. Next, using component classifiers trained by random forests, we detected the key anatomical structures at their probable locations within the regions constrained by the radial component-based model. Finally, a second-level classifier combined the results from the component detection to identify an ultrasound image as either a "fetal abdominal standard plane" or a "non- fetal abdominal standard plane." Experimental results on 223 fetal abdomen videos showed that the detection accuracy of our method was as high as 85.6% and significantly outperformed both the full abdomen and the separate anatomy detection methods without geometric constraints. The experimental results demonstrated that our system shows great promise for application to clinical practice.

Automatic detection of arterial input function in dynamic contrast enhanced MRI based on affinity propagation clustering

PURPOSE

To automatically and robustly detect the arterial input function (AIF) with high detection accuracy and low computational cost in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI).

MATERIALS AND METHODS

In this study, we developed an automatic AIF detection method using an accelerated version (Fast-AP) of affinity propagation (AP) clustering. The validity of this Fast-AP-based method was proved on two DCE-MRI datasets, i.e., rat kidney and human head and neck. The detailed AIF detection performance of this proposed method was assessed in comparison with other clustering-based methods, namely original AP and K-means, as well as the manual AIF detection method.

RESULTS

Both the automatic AP- and Fast-AP-based methods achieved satisfactory AIF detection accuracy, but the computational cost of Fast-AP could be reduced by 64.37-92.10% on rat dataset and 73.18-90.18% on human dataset compared with the cost of AP. The K-means yielded the lowest computational cost, but resulted in the lowest AIF detection accuracy. The experimental results demonstrated that both the AP- and Fast-AP-based methods were insensitive to the initialization of cluster centers, and had superior robustness compared with K-means method.

CONCLUSION

The Fast-AP-based method enables automatic AIF detection with high accuracy and efficiency.

Mesh quality oriented 3D geometric vascular modeling based on parallel transport frame

While a number of methods have been proposed to reconstruct geometrically and topologically accurate 3D vascular models from medical images, little attention has been paid to constantly maintain high mesh quality of these models during the reconstruction procedure, which is essential for many subsequent applications such as simulation-based surgical training and planning. We propose a set of methods to bridge this gap based on parallel transport frame. An improved bifurcation modeling method and two novel trifurcation modeling methods are developed based on 3D Bézier curve segments in order to ensure the continuous surface transition at furcations. In addition, a frame blending scheme is implemented to solve the twisting problem caused by frame mismatch of two successive furcations. A curvature based adaptive sampling scheme combined with a mesh quality guided frame tilting algorithm is developed to construct an evenly distributed, non-concave and self-intersection free surface mesh for vessels with distinct radius and high curvature. Extensive experiments demonstrate that our methodology can generate vascular models with better mesh quality than previous methods in terms of surface mesh quality criteria.

Impulse-Based Rendering Methods for Haptic Simulation of Bone-Burring

Bone-burring is a common procedure in orthopedic, dental, and otologic surgeries. Virtual reality (VR)-based surgical simulations with both visual and haptic feedbacks provide novice surgeons with a feasible and safe way to practice their burring skill. However, creating realistic haptic interactions between a high-speed rotary burr and stiff bone is a challenging task. In this paper, we propose a novel interactive haptic bone-burring model based on impulse-based dynamics to simulate the contact forces, including resistant and frictional forces. In order to mimic the lateral and axial burring vibration forces, a 3D vibration model has been developed. A prototype haptic simulation system for the bone-burring procedure has been implemented to evaluate the proposed haptic rendering methods. Several experiments of force evaluations and task-oriented tests were conducted on the prototype system. The results demonstrate the validity and feasibility of the proposed methods.

Application of L1-norm regularization to epicardial potential reconstruction based on gradient projection

The epicardial potential (EP)-targeted inverse problem of electrocardiography (ECG) has been widely investigated as it is demonstrated that EPs reflect underlying myocardial activity. It is a well-known ill-posed problem as small noises in input data may yield a highly unstable solution. Traditionally, L2-norm regularization methods have been proposed to solve this ill-posed problem. But the L2-norm penalty function inherently leads to considerable smoothing of the solution, which reduces the accuracy of distinguishing abnormalities and locating diseased regions. Directly using the L1-norm penalty function, however, may greatly increase computational complexity due to its non-differentiability. We propose an L1-norm regularization method in order to reduce the computational complexity and make rapid convergence possible. Variable splitting is employed to make the L1-norm penalty function differentiable based on the observation that both positive and negative potentials exist on the epicardial surface. Then, the inverse problem of ECG is further formulated as a bound-constrained quadratic problem, which can be efficiently solved by gradient projection in an iterative manner. Extensive experiments conducted on both synthetic data and real data demonstrate that the proposed method can handle both measurement noise and geometry noise and obtain more accurate results than previous L2- and L1-norm regularization methods, especially when the noises are large.

Color quantification for evaluation of stained tissues

The objective evaluation of the color and shade in stained images remains unsolved and is frequently and extensively encountered in biomedical studies. Most of the evaluations on the color and shade in the stained images are currently performed by subjective grading, which is prone to be affected by inter-reader variation. This paper introduces a novel approach to automatically quantify the color and shade in the stained histological image based on its similarity map in the CIELAB color space with respect to a user specified reference color. The proposed algorithm was applied on three datasets, i.e., a phantom image, the Prussian blue staining of human osteosarcoma cell culture, and histological sections of the Prussian blue stained rat kidney, liver and spleen. The result shows that our method is able to represent the color and shade as a numerical value that correlated well with human perception.

Chinese Acupuncture Expert System (CAES)-a useful tool to practice and learn medical acupuncture

This paper describes the development of a Chinese Acupuncture Expert System (CAES) that will assist the learning and practice of medical acupuncture. This was the development of a Chinese Acupuncture Expert System which incorporated eight functional modules. These modules were 1) Add Patient Record subsystem; 2) Diagnosis subsystem ; 3) Acupuncture Prescription subsystem ; 4) Needle Insertion Position Animation subsystem ; 5) Acupuncture Points Usage Statistic subsystem ; 6) History Query subsystem; 7) Acupuncture Points Query subsystem and 8) Diagnosis Remarks and Diagnosis Record Save subsystem. Two databases were built-Patient Record database and Diagnosis (Acupuncture) Knowledge database. All the Traditional Chinese Medicine (TCM) knowledge and acupuncture treatment prescriptions were extracted from officially used TCM textbooks and received guidance and expert advice from two acupuncturists working in this project. A Chinese Acupuncture Expert System (CAES) was built, which after the input from users of any Chinese disease symptoms and signs, it can provide a list of related TCM syndrome diagnoses based on the patients' disease symptoms and signs, and at the same time it can offer advice of the appropriate Chinese acupuncture treatment to the users. CAES also provided text descriptions and acupuncture animations showing the acupoint locations and the direction and depth of the needle insertion technique. Therefore users can easily learn acupuncture and practice it. This new acupuncture expert system will hopefully provide an easy way for users to learn and practice Chinese Acupuncture and establish its usefulness after it was fully evaluated.

CvhSlicer: an interactive cross-sectional anatomy navigation system based on high-resolution Chinese visible human data

We introduce the design and implementation of an interactive system for the navigation of cross-sectional anatomy based on Chinese Visible Human (CVH) data, named CvhSlicer. This system is featured in real-time computation and rendering of high-resolution anatomical images on standard personal computers (PCs) equipped with commodity Graphics Processing Units (GPUs). In order to load the whole-body dataset into the memory of a common PC, several processing steps are first applied to compress the huge CVH data. Thereafter, an adaptive CPU-GPU balancing scheme is performed to dynamically distribute rendering tasks among CPU and GPU based on parameters of computing resources. Experimental results demonstrate that our system can achieve real-time performance and has great potential to be used in anatomy education.

Fast and accurate 3-D registration of HR-pQCT images

High-resolution peripheral quantitative computed tomography (HR-pQCT) is a new noninvasive bone imaging technology that generates high-resolution 3-D images for quantitatively analysis of the bone microarchitecture in human. To enable quantitative evaluation of bone changes, either bone gain or loss, accurate alignment between the baseline and follow-up scans of the same individual is necessary. The major difficulties in achieving efficient and automatic registration of the HR-pQCT data are the large data size, deformations in the nonskeletal structures, and the complexity of the trabecular bone geometry. In this paper, we propose an automatic surface-based approach for fast and accurate registration of the HR-pQCT data, where the rigid registration is applied on the surfaces of the bony structures extracted from the grayscale HR-pQCT. The bony structure segmentation is performed via an automatic method that can adaptively determine the thresholds for separating the bony structure from the background and nonskeletal tissues. Experimental results performed on ten pairs of baseline and follow-up wrist scans of five adolescents and five elderly patients with osteoporosis showed the advantage of the proposed method in the high degree of automation, while the resultant parameters describing bone mineral density and trabecular architecture after registration were comparable with the outputs of the scanner's software. This automatic and accurate matching procedure may contribute to the clinical application and research of HR-pQCT.

Concatenated and parallel optimization for the estimation of T1 map in FLASH MRI with multiple flip angles

Most traditional methods for T(1) map estimation in MRI with fast low-angle-shot sequences are aimed at high efficiency by compromising the fitting accuracy. In this paper, the fundamental problem of parameter estimation in fast low-angle-shot MRI was re-examined, and an accurate and fast optimization approach, named concatenated optimization for parameter estimation, was proposed for the regression of data points acquired with multiple flip angles. The initial estimation of T(1) was obtained from the linear regression, followed by the constrained nonlinear regression based on the initial estimates. This heterogeneous initialization strategy improves the fitting accuracy and reduces the computational time. A computationally efficient implementation of concatenated optimization for parameter estimation was achieved based on the graphic processing unit, named as concatenated optimization for parameter estimation graphic processing unit. In experimental comparison with Fram's method and the Fitter Tool in Jim, the proposed methods are capable of achieving significantly higher efficiency and more accurate estimations.

Reconstruction of volumetric ultrasound panorama based on improved 3D SIFT

Registration of ultrasound volumes is a key issue for the reconstruction of volumetric ultrasound panorama. In this paper, we propose an improved three-dimensional (3D) scale invariant feature transform (SIFT) algorithm to globally register ultrasound volumes acquired from dedicated ultrasound probe, where local deformations are corrected by block-based warping algorithm. Original SIFT algorithm is extended to 3D and improved by combining the SIFT detector with Rohr3D detector to extract complementary features and applying the diffusion distance algorithm for robust feature comparison. Extensive experiments have been performed on both phantom and clinical data sets to demonstrate the effectiveness and robustness of our approach.

Volume-based morphometry of brain MR images in adolescent idiopathic scoliosis and healthy control subjects

BACKGROUND AND PURPOSE

Adolescent idiopathic scoliosis (AIS) is a spinal deformity with unknown cause. Previous studies have suggested that subclinical neurologic abnormalities are associated with AIS. The objective of this prospective study was to characterize systematically neuroanatomic changes in patients with left thoracic AIS vs right thoracic AIS and healthy control subjects by using volume-based morphometry.

MATERIALS AND METHODS

Our current study involved 9 girls with left thoracic AIS and 20 girls with right thoracic AIS vs 11 and 17 matched female control subjects, respectively. Voxel-based morphometry (VBM), deformation-based morphometry (DBM), and tensor-based morphometry (TBM) were used to analyze the MR images aligned with a specific brain template of local adolescent girls. The statistical t test was used in VBM and TBM, and the Hotelling T(2) test was applied in DBM.

RESULTS

Using VBM, we found statistically significant differences (P < .05) in the white matter attenuation of the genu of the corpus callosum and left internal capsule (left thoracic AIS < control subjects). In contrast, no significant differences were observed between patients with right thoracic AIS and control subjects.

CONCLUSIONS

White matter attenuation in the corpus callosum and left internal capsule, responsible for interhemispheric communication and conduit of the corticothalamic projectional fibers, respectively, were found to be significantly lower in left thoracic AIS compared with control subjects; however, this was not the case in right thoracic AIS. Confirmation of the findings is required in future research, which needs to evaluate the relationship of white matter abnormality to curve laterality, pathogenesis, and prognosis in patients with AIS, with biologic significance and possible therapeutic correction.

Volumetric ultrasound panorama based on 3D SIFT

The reconstruction of three-dimensional (3D) ultrasound panorama from multiple ultrasound volumes can provide a wide field of view for better clinical diagnosis. Registration of ultrasound volumes has been a key issue for the success of this panoramic process. In this paper, we propose a method to register and stitch ultrasound volumes, which are scanned by dedicated ultrasound probe, based on an improved 3D Scale Invariant Feature Transform (SIFT) algorithm. We propose methods to exclude artifacts from ultrasound images in order to improve the overall performance in 3D feature point extraction and matching. Our method has been validated on both phantom and clinical data sets of human liver. Experimental results show the effectiveness and stability of our approach, and the precision of our method is comparable to that of the position tracker based registration.

An adaptive transmission protocol for managing dynamic shared states in collaborative surgical simulation

A force prediction algorithm is proposed to facilitate virtual-reality (VR) based collaborative surgical simulation by reducing the effect of network latencies. State regeneration is used to correct the estimated prediction. This algorithm is incorporated into an adaptive transmission protocol in which auxiliary features such as view synchronization and coupling control are equipped to ensure the system consistency. We implemented this protocol using multi-threaded technique on a cluster-based network architecture.

Morphometric analysis for pathological abnormality detection in the skull vaults of adolescent idiopathic scoliosis girls

In this paper, we present a comprehensive framework to detect morphological changes in skull vaults of adolescent idiopathic scoliosis girls. To our knowledge, this is the first attempt to use a combination of medical knowledge, image analysis techniques, statistical learning tools, and scientific visualization methods to detect skull morphological changes. The shape analysis starts from a reliable 3-D segmentation of the skull using thresholding and math-morphological operations. The gradient vector flow is used to model the skull vault surface, which is followed by a spherically uniform sampling. The scale-normalized distances from the shape centroid to sample points are defined as the features. The most discriminative features are selected using recursive feature elimination for support vector machine. The results of this study specify the skull vault surface changes and shed light on building the evidence of bone formation abnormality in AIS girls.

Research and applications of virtual medicine

In this paper, we would like to share our experience in developing various virtual medicine applications, including virtual arthroscopy, virtual acupuncture and virtual anatomy. We have deployed the latest graphics processor to develop these virtual reality based learning and training systems. Having utilized various visible human datasets, the visual quality of the applications has been greatly improved.

A spectral clustering approach to fMRI activation detection

Conventional clustering methods for fMRI activation detection implicitly assume that data scatter in clusters with certain shapes. But this assumption is inconsistent with the general reality in fMRI data, and will consequently achieve detection results with higher false alarm rate. To solve this problem, we propose an alternative clustering method, namely spectral cluster analysis (SCA), which uses eigenvectors of a matrix derived from the dataset to cluster the wavelet coefficients extracted from the fMRI time series. Experimental results demonstrate reliability and flexibility of this new fMRI clustering approach.

Color image diffusion using adaptive bilateral filter

In this paper, we propose an approach to diffuse color images based on the bilateral filter. Real image data has a level of uncertainty that is manifested in the variability of measures assigned to pixels. This uncertainty is usually interpreted as noise and considered an undesirable component of the image data. Image diffusion can smooth away small-scale structures and noise while retaining important features, thus improving the performances for many image processing algorithms such as image compression, segmentation and recognition. The bilateral filter is noniterative, simple and fast. It has been shown to give similar and possibly better filtering results than iterative approaches. However, the performance of this filter is greatly affected by the choose of the parameters of filtering kernels. In order to remove noise and maintain the significant features on images, we extend the bilateral filter by introducing an adaptive domain spread into the nonlinear diffusion scheme. For color images, we employ the CIE-Lab color system to describe input images and the filtering process is operated using three channels together. Our analysis shows that the proposed method is more suitable for preserving strong edges on noisy images than the original bilateral filter. Empirical results on both nature images and color medical images confirm the novel method's advantages, and show it can diffuse various kinds of color images correctly and efficiently.

Landmark correspondence optimization for coupled surfaces

Volumetric layers are often encountered in medical images. Unlike solid structures, volumetric layers are characterized by double and nested bounding surfaces. It is expected that better statistical models can be built by utilizing the surface coupleness rather than simply applying the landmarking method on each of them separately. We propose an approach to optimizing the landmark correspondence on the coupled surfaces by minimizing the description length that incorporates local thickness gradient. The evaluations are performed on a set of 2-D synthetic close coupled contours and a set of real-world open surfaces, the skull vaults. Compared with performing landmarking separately on the coupled surfaces, the proposed method constructs models that have better generalization ability and specificity.

An adaptive framework using cluster-based hybrid architecture for enhancing collaboration in surgical simulation

Research on collaborative virtual surgery opens the opportunity for simulating the cooperative work during surgical operations. It is however a challenging task to design and implement a high performance collaborative surgical simulation system because of the difficulty in maintaining a high level of state consistency under limited network transmission capacity. In this paper, we present an adaptive framework using cluster-based hybrid architecture to support real-time collaboration in surgical simulation. In addition to the TCP communication protocol, the framework is also equipped with UDP for multicasting, allowing for a flexible strategy to reduce network latency. A set of techniques was proposed to assure reliable transmission on top of standard yet unreliable multicast protocols. Experimental results demonstrate that this framework can support collaborative surgical simulation with lower network latencies than traditional client-server architecture.

Shape Statistics Variational Approach for the Outer Contour Segmentation of Left Ventricle MR Images

Segmentation of left ventricles is one of the important research topics in cardiac magnetic resonance (MR) imaging. The segmentation precision influences the authenticity of ventricular motion reconstruction. In left ventricle MR images, the weak and broken boundary increases the difficulty of segmenting the outer contour precisely. In this paper, we present an improved shape statistics variational approach for the outer contour segmentation of left ventricle MR images. We use the Mumford-Shah model in an object feature space and incorporate the shape statistics and an edge image to the variational framework. The introduction of shape statistics can improve the segmentation with broken boundaries. The edge image can enhance the weak boundary and thus improve the segmentation precision. The generation of the object feature image, which has homogenous "intensities" in the left ventricle, facilitates the application of the Mumford-Shah model. A comparison of mean absolute distance analysis between different contours generated with our algorithm and that generated by hand demonstrated that our method can achieve a higher segmentation precision and a better stability than various approaches. It is a semiautomatic way for the segmentation of the outer contour of the left ventricle in clinical applications.

Semi-automatic segmentation and tracking of CVH data

Construction of speed function is crucial in applying level set method for medical image segmentation. We present a unified approach for segmenting and tracking of the high-resolution Chinese Visible Human (CVH) data. The underlying link of these two parts relies on the proposed variational framework for the speed function. Our proposed method can be applied to segmenting the first slice of the volume data, in the first step; It can also be adapted to track the boundaries of the homogeneous organs in the following serial images. In addition to promising segmentation results, the tracking procedure shows the advantage of less amount of user intervention.

Photorealistic virtual anatomy based on Chinese Visible Human data

Virtual reality based learning of human anatomy is feasible when a database of 3D organ models is available for the learner to explore, visualize, and dissect in virtual space interactively. In this article, we present our latest work on photorealistic virtual anatomy applications based on the Chinese Visible Human (CVH) data. We have focused on the development of state-of-the-art virtual environments that feature interactive photo-realistic visualization and dissection of virtual anatomical models constructed from ultra-high resolution CVH datasets. We also outline our latest progress in applying these highly accurate virtual and functional organ models to generate realistic look and feel to advanced surgical simulators.