A New Deformation Model of Biological Tissue for Surgery Simulation

A data-driven approach for real-time soft tissue deformation prediction using nonlinear presurgical simulations

A method that allows a fast and accurate registration of digital tissue models obtained during preoperative, diagnostic imaging with those captured intraoperatively using lower-fidelity ultrasound imaging techniques is presented. Minimally invasive surgeries are often planned using preoperative, high-fidelity medical imaging techniques such as MRI and CT imaging. While these techniques allow clinicians to obtain detailed 3D models of the surgical region of interest (ROI), various factors such as physical changes to the tissue, changes in the body's configuration, or apparatus used during the surgery may cause large, non-linear deformations of the ROI. Such deformations of the tissue can result in a severe mismatch between the preoperatively obtained 3D model and the real-time image data acquired during surgery, potentially compromising surgical success. To overcome this challenge, this work presents a new approach for predicting intraoperative soft tissue deformations. The approach works by simply tracking the displacements of a handful of fiducial markers or analogous biological features embedded in the tissue, and produces a 3D deformed version of the high-fidelity ROI model that registers accurately with the intraoperative data. In an offline setting, we use the finite element method to generate deformation fields given various boundary conditions that mimic the realistic environment of soft tissues during a surgery. To reduce the dimensionality of the 3D deformation field involving thousands of degrees of freedom, we use an autoencoder neural network to encode each computed deformation field into a short latent space representation, such that a neural network can accurately map the fiducial marker displacements to the latent space. Our computational tests on a head and neck tumor, a kidney, and an aorta model show prediction errors as small as 0.5 mm. Considering that the typical resolution of interventional ultrasound is around 1 mm and each prediction takes less than 0.5 s, the proposed approach has the potential to be clinically relevant for an accurate tracking of soft tissue deformations during image-guided surgeries.

Advances of surgical robotics: image-guided classification and application

Surgical robotics application in the field of minimally invasive surgery has developed rapidly and has been attracting increasingly more research attention in recent years. A common consensus has been reached that surgical procedures are to become less traumatic and with the implementation of more intelligence and higher autonomy, which is a serious challenge faced by the environmental sensing capabilities of robotic systems. One of the main sources of environmental information for robots are images, which are the basis of robot vision. In this review article, we divide clinical image into direct and indirect based on the object of information acquisition, and into continuous, intermittent continuous, and discontinuous according to the target-tracking frequency. The characteristics and applications of the existing surgical robots in each category are introduced based on these two dimensions. Our purpose in conducting this review was to analyze, summarize, and discuss the current evidence on the general rules on the application of image technologies for medical purposes. Our analysis gives insight and provides guidance conducive to the development of more advanced surgical robotics systems in the future.

A high-fidelity virtual liver model incorporating biological characteristics

Flexible tissue modeling plays an important role in the field of telemedicine. It is related to whether the soft tissue deformation process can be accurately, real-time and vividly simulated during surgery. However, most existing models lack the unique biological characteristics. To solve this problem, we proposed a high-fidelity virtual liver model incorporating biological characteristics, such as the viscoelastic, anisotropic and nonlinear biological characteristics. Besides, to the best of our knowledge, our study is the first to introduce the viscoplasticity of biological tissues to improve the fidelity of the liver model. This mothod was proposed to describe the viscoplastic characteristics of the diseased liver resection process, when the liver is in a state of excessive deformation and loss of elasticity, however, there are few works focusing on this problem. The 3DMax2020 and OpenGL4.6 were used to build a liver surgery simulation platform, and the PHANTOM OMNI manual controller was used to sense the feedback force during the operation. The proposed model was verified from three aspects of accuracy, fidelity and real-time performance. The experimental results show that the proposed virtual liver model can enhance visual perception ability, improve deformation accuracy and fidelity.

Modeling fibrous soft tissue dissection with elastic-plastic deformation for simulation of brain tumor removal

BACKGROUND AND OBJECTIVE

Realistic modeling the dissection of brain tissue is of key importance for simulation of brain tumor removal in virtual neurosurgery systems. However, existing methods are unable to characterize inelastic behaviors of brain tissue, such as plastic deformation and dissection evolution, making it ineffective in simulating brain tumor removal procedures.

METHODS

In this paper, a model of fibrous soft tissue dissection for the simulation of brain tumor removal is proposed. A dissection variable of representative volume element is used to characterize the dissection state of the fibrous soft tissue. The evolution of dissection with elastic-plastic deformation under the effects of external loads is presented.

RESULTS

Simulation results show that the proposed model provides realistic, stable and intuitive results in the simulation of fracture in fibrous soft tissues. As the external load increases, the fibrous soft tissue begins to crack, with the cracks growing and multiplying until they eventually merge to form a fracture. The proposed model is incorporated into the simulation of brain tumor removal.

CONCLUSIONS

The experimental results demonstrate the feasibility of modeling fibrous soft tissue dissection with elastic-plastic deformation. A relative high degree of realistic visual feedback is achieved.

A new soft tissue deformation model based on Runge-Kutta: Application in lung

Flexible body deformation model is the most critical research in the field of telemedicine, which decides whether the deformation process of tissues and organs can be simulated in real time and realistically. Basically, the improvement of model accuracy often means the loss of real-time performance. In order to effectively balance between accuracy and real-time performance, this paper proposes a new model, which uses the step-variable fourth-order Runge-Kutta method for the first time to solve the relationship problem between the external force and displacement of the nodes in the finite element deformation of the lung. To improve the real-time performance of the model, a one-stage solution optimization algorithm is proposed to optimize the step size selection problem. Finally, an accelerated two-level node update algorithm is proposed to further improve the real-time performance. A lung surgery simulation platform with 3DMax2020 and OpenGL4.5 is built to verify the accuracy, efficiency, realism and applicability of the model. Experimental results show that the proposed lung model can achieve real-world visual reproduction during remote surgery, and exceeds other 13 reference models in real-time performance, with natural deformation effect, high simulation accuracy, which is suitable for modeling normal lung and four types of lungs suffering from diseases.

Biological activity of multicomponent bio-hydrogels loaded with tragacanth gum

Producing hydrogels capable of mimicking the biomechanics of soft tissue remains a challenge. We explore the potential of plant-based hydrogels as polysaccharide tragacanth gum and antioxidant lignin nanoparticles in bioactive multicomponent hydrogels for tissue engineering. These natural components are combined with TEMPO-oxidized cellulose nanofibrils, a material with known shear thinning behavior. Hydrogels presented tragacanth gum (TG) concentration-dependent rheological properties suitable for extrusion 3D printing. TG enhanced the swelling capacity up to 645 % and the degradation rate up to 1.3 %/day for hydrogels containing 75 % of TG. Young's moduli of the hydrogels varied from 5.0 to 11.6 kPa and were comparable to soft tissues like skin and muscle. In vitro cell viability assays revealed that the scaffolds were non-toxic and promoted proliferation of hepatocellular carcinoma HepG2 cells. Therefore, the plant-based hydrogels designed in this work have a significant potential for tissue engineering.

A deformation model of pulsating brain tissue for neurosurgery simulation

BACKGROUND AND OBJECTIVES

For neurological simulation, an accurate deformation model of brain tissue is of key importance for faithful visual feedback. Existing models, however, do not take into account intracranial pulsation, which degrades significantly the realism of visual feedback.

METHODS

In this paper, a finite element model incorporating intracranial pressure is proposed for simulating brain tissue deformation with pulsation. An implicit Euler method is developed to calculate the deformation of brain tissue. A circuit model of intracranial pressure dynamics is established based on cerebral blood and cerebrospinal fluid circulations. The intracranial pulsation of pressure is introduced into the deformation model, so that the simulated brain tissues pulsate with a rhythm in accord with the changes of intracranial pressure, which resembles real-life neurosurgery.

RESULTS AND CONCLUSIONS

The experimental implementation of the proposed deformation model and the calculation method shows that it provides realistic simulation of brain tissue pulsation and real-time performance is achieved on an ordinary computer for certain procedures of neurosurgery.

A new geometric combination of cutting and bleeding modules for surgical simulation systems

BACKGROUND AND OBJECTIVES

Cutting and bleeding are often independent of each other in the traditional virtual surgery system because of the differences in the calculation of physical models and the lack of internal structure. In order to improve the fidelity of virtual surgery scene and the training value for surgeons, a new geometric combination of cutting and bleeding modules is introduced.

METHODS

In this paper, we introduce a cutting model based on volume rendering and meshless method. The multidimensional parameters derived from the gray values are presented to participate in the calculation of both physical and geometric models, which distinguishes between different adjacent soft tissues. The bleeding simulation with improved physical properties and rendering algorithms of geometric model is proposed to meet several different bleeding states. After cutting procedures, the tearing parts can be judged through the vision and the tactile sensation. The initial velocity and rendering algorithm of bleeding particles are determined by the multidimensional parameters of the cutting position, which realizes the geometric combination of cutting and bleeding modules.

RESULTS AND CONCLUSIONS

Simulation results show that tearing different tissue structures will produce corresponding bleeding states. When the skin and flesh are torn, the blood is slowly generated at the incision, and then diffuses to the surface of soft tissue. When the important blood vessels are ruptured, the blood gushes from the laceration. Compared with the conventional virtual surgery system, both visual effect and interactivity of the cutting and bleeding modules are improved in the proposed geometric combination.

A study of real-time simulation of liver of virtual surgical robot based on biologically position-based dynamic model

Virtual surgery robot can accurately modeling of surgical instruments and human organs, and realistic simulation of various surgical phenomena such as deformation of organic tissues, surgery simulation system can provide operators with reusable virtual training and simulation environment. To meet the requirement of virtual surgery robot for the authenticity and real-time of soft tissue deformation and surgical simulation in liver surgery, a new method is proposed to simulate the deformation of soft tissue. This method combines the spring force, the external force of the system, and the constraint force produced by the constraint function of the position-based dynamics. Based on the position-based dynamics, an improved three-parameter mass-spring model is added. In the calculation of the elastic force, the nonlinearity and viscoelasticity of the soft tissue are introduced, and the joint force of the constraint projection process and the constraint force of the position-based dynamics is used to modify mass points movement. The method of position-based dynamics based on biological characteristics, not only considers the biomechanical properties of biological soft tissue as an organic polymer such as viscoelasticity, nonlinearity, and incompressibility but also retains the rapidity and stability of the position based dynamic method. Through the simulation data, the optimal side length of tetrahedral mesh in the improved three-parameter model is obtained, and the physical properties of the model are proved. The real-time simulation of the liver and other organs is completed by using the Geomagic touch force feedback device, which proves the practicability and effectiveness of this method.

Fast and accurate nonlinear hyper-elastic deformation with a posteriori numerical verification of the convergence of solution: Application to the simulation of liver deformation

In this paper, we propose a new method to reduce the computational complexity of calculating the tangential stiffness matrix in a nonlinear finite element formulation. Our approach consists in partially updating the tangential stiffness matrix during a classic Newton-Raphson iterative process. The complexity of such an update process has the order of the number of mesh vertices to the power of two. With our approach, this complexity is reduced to the power of two of only the number of updated vertices. We numerically study the convergence of the solution with our modified algorithm. We describe the deformation through a strain energy density function which is defined with respect to the Lagrangian strain. We derive the conditions of convergence for a given tangential stiffness matrix and a given set of updated vertices. We use nonlinear geometric deformation and the nonlinear Mooney-Rivilin model with both tetrahedron and hexahedron element meshing. We provide extensive results using a cube with small and large number of elements. We provide results on nonlinearly deformed liver with multiple deformation ranges of updated vertices. We compare the proposed method to state-of-the-art work and we prove its efficiency at three levels: accuracy, speed of convergence and small radius of convergence.

Automatic Construction of Chinese Herbal Prescriptions From Tongue Images Using CNNs and Auxiliary Latent Therapy Topics

The tongue image provides important physical information of humans. It is of great importance for diagnoses and treatments in clinical medicine. Herbal prescriptions are simple, noninvasive, and have low side effects. Thus, they are widely applied in China. Studies on the automatic construction technology of herbal prescriptions based on tongue images have great significance for deep learning to explore the relevance of tongue images for herbal prescriptions, it can be applied to healthcare services in mobile medical systems. In order to adapt to the tongue image in a variety of photographic environments and construct herbal prescriptions, a neural network framework for prescription construction is designed. It includes single/double convolution channels and fully connected layers. Furthermore, it proposes the auxiliary therapy topic loss mechanism to model the therapy of Chinese doctors and alleviate the interference of sparse output labels on the diversity of results. The experiment use the real-world tongue images and the corresponding prescriptions and the results can generate prescriptions that are close to the real samples, which verifies the feasibility of the proposed method for the automatic construction of herbal prescriptions from tongue images. Also, it provides a reference for automatic herbal prescription construction from more physical information.

Modeling of connective tissue damage for blunt dissection of brain tumor in neurosurgery simulation

We introduce a new model for connective tissue damage in blunt dissection, which is a very important process in neurosurgery simulation. Specifically, the tool-tissue interaction between the instrument and connective tissue is incorporated into the model of connective tissue damage. This damage develops with the evolution criterion due to the effect of the external load. The tetrahedral mesh in the soft tissue model is removed for the representation of rupture as the damage accumulates to the threshold value. Analysis and experiments show that the connective tissue damage model provides stable, visually realistic results for the simulation of the connective tissue rupture process. The stiffness of the connective tissue decreases as the damage accumulates. The proposed model for connective tissue damage was incorporated into the development of a neurosurgery simulator, in which blunt dissection of a brain tumor was simulated.

Active Collision Avoidance for Human-Robot Interaction With UKF, Expert System, and Artificial Potential Field Method

With the development of Industry 4.0, the cooperation between robots and people is increasing. Therefore, man-machine security is the first problem that must be solved. In this paper, we proposed a novel methodology of active collision avoidance to safeguard the human who enters the robot's workspace. In the conventional approaches of obstacle avoidance, it is not easy for robots and humans to work safely in the common unstructured environment due to the lack of the intelligence. In this system, one Kinect is employed to monitor the workspace of the robot and detect anyone who enters the workspace of the robot. Once someone enters the working space, the human will be detected, and the skeleton of the human can be calculated in real time by the Kinect. The measurement errors increase over time, owing to the tracking error and the noise of the device. Therefore we use an Unscented Kalman Filter (UKF) to estimate the positions of the skeleton points. We employ an expert system to estimate the behavior of the human. Then let the robot avoid the human by taking different measures, such as stopping, bypassing the human or getting away. Finally, when the robot needs to execute bypassing the human in real time, to achieve this, we adopt a method called artificial potential field method to generate a new path for the robot. By using this active collision avoidance, the system can achieve the purpose that the robot is unable to touch on the human. This proposed system highlights the advantage that during the process, it can first detect the human, then analyze the motion of the human and finally safeguard the human. We experimentally tested the active collision avoidance system in real-world applications. The results of the test indicate that it can effectively ensure human security.

Virtual reality simulation of robotic transsphenoidal brain tumor resection: Evaluating dynamic motion scaling in a master-slave system

Background

Integrating simulators with robotic surgical procedures could assist in designing and testing of novel robotic control algorithms and further enhance patient‐specific pre‐operative planning and training for robotic surgeries.

Methods

A virtual reality simulator, developed to perform the transsphenoidal resection of pituitary gland tumours, tested the usability of robotic interfaces and control algorithms. It used position‐based dynamics to allow soft‐tissue deformation and resection with haptic feedback; dynamic motion scaling control was also incorporated into the simulator.

Results

Neurosurgeons and residents performed the surgery under constant and dynamic motion scaling conditions (CMS vs DMS). DMS increased dexterity and reduced the risk of damage to healthy brain tissue. Post‐experimental questionnaires indicated that the system was well‐evaluated by experts.

Conclusion

The simulator was intuitively and realistically operated. It increased the safety and accuracy of the procedure without affecting intervention time. Future research can investigate incorporating this simulation into a real micro‐surgical robotic system.

A review of virtual cutting methods and technology in deformable objects

BACKGROUND

Virtual cutting of deformable objects has been a research topic for more than a decade and has been used in many areas, especially in surgery simulation.

METHODS

We refer to the relevant literature and briefly describe the related research. The virtual cutting method is introduced, and we discuss the benefits and limitations of these methods and explore possible research directions.

RESULTS

Virtual cutting is a category of object deformation. It needs to represent the deformation of models in real time as accurately, robustly and efficiently as possible. To accurately represent models, the method must be able to: (1) model objects with different material properties; (2) handle collision detection and collision response; and (3) update the geometry and topology of the deformable model that is caused by cutting.

CONCLUSION

Virtual cutting is widely used in surgery simulation, and research of the cutting method is important to the development of surgery simulation.

A Novel Haptic Interactive Approach to Simulation of Surgery Cutting Based on Mesh and Meshless Models

In the present work, the majority of implemented virtual surgery simulation systems have been based on either a mesh or meshless strategy with regard to soft tissue modelling. To take full advantage of the mesh and meshless models, a novel coupled soft tissue cutting model is proposed. Specifically, the reconstructed virtual soft tissue consists of two essential components. One is associated with surface mesh that is convenient for surface rendering and the other with internal meshless point elements that is used to calculate the force feedback during cutting. To combine two components in a seamless way, virtual points are introduced. During the simulation of cutting, the Bezier curve is used to characterize smooth and vivid incision on the surface mesh. At the same time, the deformation of internal soft tissue caused by cutting operation can be treated as displacements of the internal point elements. Furthermore, we discussed and proved the stability and convergence of the proposed approach theoretically. The real biomechanical tests verified the validity of the introduced model. And the simulation experiments show that the proposed approach offers high computational efficiency and good visual effect, enabling cutting of soft tissue with high stability.

A Novel Nonlinear Parameter Estimation Method of Soft Tissues

The elastic parameters of soft tissues are important for medical diagnosis and virtual surgery simulation. In this study, we propose a novel nonlinear parameter estimation method for soft tissues. Firstly, an in-house data acquisition platform was used to obtain external forces and their corresponding deformation values. To provide highly precise data for estimating nonlinear parameters, the measured forces were corrected using the constructed weighted combination forecasting model based on a support vector machine (WCFM_SVM). Secondly, a tetrahedral finite element parameter estimation model was established to describe the physical characteristics of soft tissues, using the substitution parameters of Young's modulus and Poisson's ratio to avoid solving complicated nonlinear problems. To improve the robustness of our model and avoid poor local minima, the initial parameters solved by a linear finite element model were introduced into the parameter estimation model. Finally, a self-adapting Levenberg-Marquardt (LM) algorithm was presented, which is capable of adaptively adjusting iterative parameters to solve the established parameter estimation model. The maximum absolute error of our WCFM_SVM model was less than 0.03 Newton, resulting in more accurate forces in comparison with other correction models tested. The maximum absolute error between the calculated and measured nodal displacements was less than 1.5 mm, demonstrating that our nonlinear parameters are precise.

A high-resolution model for soft tissue deformation based on point primitives

BACKGROUND AND OBJECTIVES

In order to achieve a high degree of visual realism in surgery simulation, we propose a new model, which is based on point primitives and continuous elastic mechanics theory, for soft tissue deformation, tearing and/or cutting.

METHODS

The model can be described as a two-step local high-resolution strategy. First, appropriate volumetric data are sampled and assigned with proper physical properties. Second, sparsely sampled points in non-deformed regions and densely-sampled points in the deformed zone are selected and evaluated. By using a meshless deformation model based on point primitives for all volumetric data, the affine transform matrix of collision points can be computed. The new positions of neighboring points in the collided surface can be then calculated, and more details in the local deformed zone can be obtained for rendering. Technical details about the derivations of the proposed model as well as its implementation are given.

RESULTS

The visual effects and computation cost of the proposed model are evaluated and compared with conventional primitives-based methods. Experimental results show that the proposed model provides users (trainees) with improved visual feedback while the computational cost is at the same magnitude of other similar methods.

CONCLUSIONS

The proposed method is especially suitable for the simulation of soft tissue deformation and tearing because no grid information needs to be maintained. It can simulate soft tissue deformation in a high degree of authenticity with real-time performance. It could be considered implemented in the development of a mixed reality application of neurosurgery simulators in the future.