Neural network modelling of soft tissue deformation for surgical simulation

Invited Review for 20th Anniversary Special Issue of PLRev "AI for Mechanomedicine"

Mechanomedicine is an interdisciplinary field that combines different areas including biomechanics, mechanobiology, and clinical applications like mechanodiagnosis and mechanotherapy. The emergence of artificial intelligence (AI) has revolutionized mechanomedicine, providing advanced tools to analyze the complex interactions between mechanics and biology. This review explores how AI impacts mechanomedicine across four key aspects, i.e., biomechanics, mechanobiology, mechanodiagnosis, and mechanotherapy. AI improves the accuracy of biomechanical characterizations and models, deepens the understanding of cellular mechanotransduction pathways, and enables early disease detection through mechanodiagnosis. In addition, AI optimizes mechanotherapy that targets biomechanical features and mechanobiological markers by personalizing treatment strategies based on real-time patient data. Even with these advancements, challenges still exist, particularly in data quality and the ethical integration into AI in clinical practice. The integration of AI with mechanomedicine offers transformative potential, enabling more accurate diagnostics and personalized treatments, and discovering novel mechanobiological pathways.

Real-time haptic characterisation of Hunt-Crossley model based on radial basis function neural network for contact environment

Dynamic soft tissue characterisation is an important element in robotic minimally invasive surgery. This paper presents a novel method by combining neural network with recursive least square (RLS) estimation for dynamic soft tissue characterisation based on the nonlinear Hunt-Crossley (HC) model. It develops a radial basis function neural network (RBFNN) to compensate for the error caused by natural logarithmic factorisation (NLF) of the HC model for dynamic RLS estimation of soft tissue properties. The RBFNN weights are estimated according to the maximum likelihood principle to evaluate the probability distribution of the neural network modelling residual. Further, by using the linearisation error modelled by RBFNN to compensate for the linearised HC model, an RBFNN-based RLS algorithm is developed for dynamic soft tissue characterisation. Simulation and experimental results demonstrate that the proposed method can effectively model the natural logarithmic linearisation error, leading to improved accuracy for RLS estimation of the HC model parameters.

Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields

Virtual surgical training is crucial for enhancing minimally invasive surgical skills. Traditional geometric reconstruction methods based on medical CT/MRI images often fall short in providing color information, which is typically generated through pseudo-coloring or artistic rendering. To simultaneously reconstruct both the geometric shape and appearance information of organs, we propose a novel organ model reconstruction network called Endoscope-NeSRF. This network jointly leverages neural radiance fields and Signed Distance Function (SDF) to reconstruct a textured geometric model of the organ of interest from multi-view photometric images acquired by an endoscope. The prior knowledge of the inverse correlation between the distance from the light source to the object and the radiance improves the real physical properties of the organ. The dilated mask further refines the appearance and geometry at the organ's edges. We also proposed a highlight adaptive optimization strategy to remove highlights caused by the light source during the acquisition process, thereby preventing the reconstruction results in areas previously affected by highlights from turning white. Finally, the real-time realistic rendering of the organ model is achieved by combining the inverse rendering and Bidirectional Reflectance Distribution Function (BRDF) rendering methods. Experimental results show that our method closely matches the Instant-NGP method in appearance reconstruction, outperforming other state-of-the-art methods, and stands as the superior method in terms of geometric reconstruction. Our method obtained a detailed geometric model and realistic appearance, providing a realistic visual sense for virtual surgical simulation, which is important for medical training.

Maximum likelihood-based extended Kalman filter for soft tissue modelling

Realistic modelling of human soft tissue is very important in medical applications. This paper proposes a novel method by dynamically incorporating soft tissue characterisation in the process of soft tissue modelling to increase the modelling fidelity. This method defines nonlinear tissue deformation with unknown mechanical properties as a problem of nonlinear filtering identification to dynamically identify mechanical properties and further estimate nonlinear deformation behaviour of soft tissue. It combines maximum likelihood theory, nonlinear filtering and nonlinear finite element method (NFEM) for modelling of nonlinear tissue deformation behaviour based on dynamic identification of homogeneous tissue properties. On the basis of hyperelasticity, a nonlinear state-space equation is established by discretizing tissue deformation through NFEM for dynamic filtering. A maximum likelihood algorithm is also established to dynamically identify tissue mechanical properties during the deformation process. Upon above, a maximum likelihood-based extended Kalman filter is further developed for dynamically estimating tissue nonlinear deformation based on dynamic identification of tissue mechanical properties. Simulation and experimental analyses reveal that the proposed method not only overcomes the NFEM limitation of expensive computations, but also absorbs the NFEM merit of high accuracy for modelling of homogeneous tissue deformation. Further, the proposed method also effectively identifies tissue mechanical properties during the deformation modelling process.

Finite-element kalman filter with state constraint for dynamic soft tissue modelling

This research work proposes a novel method for realistic and real-time modelling of deformable biological tissues by the combination of the traditional finite element method (FEM) with constrained Kalman filtering. This methodology transforms the problem of deformation modelling into a problem of constrained filtering to estimate physical tissue deformation online. It discretises the deformation of biological tissues in 3D space according to linear elasticity using FEM. On the basis of this, a constrained Kalman filter is derived to dynamically compute mechanical deformation of biological tissues by minimizing the error between estimated reaction forces and applied mechanical load. The proposed method solves the disadvantage of costly computation in FEM while inheriting the superiority of physical fidelity.

Extended Kalman Filter Nonlinear Finite Element Method for Nonlinear Soft Tissue Deformation

BACKGROUND AND OBJECTIVE

Soft tissue modelling is crucial to surgery simulation. This paper introduces an innovative approach to realistic simulation of nonlinear deformation behaviours of biological soft tissues in real time.

METHODS

This approach combines the traditional nonlinear finite-element method (NFEM) and nonlinear Kalman filtering to address both physical fidelity and real-time performance for soft tissue modelling. It defines tissue mechanical deformation as a nonlinear filtering process for dynamic estimation of nonlinear deformation behaviours of biological tissues. Tissue mechanical deformation is discretized in space using NFEM in accordance with nonlinear elastic theory and in time using the central difference scheme to establish the nonlinear state-space models for dynamic filtering.

RESULTS

An extended Kalman filter is established to dynamically estimate nonlinear mechanical deformation of biological tissues. Interactive deformation of biological soft tissues with haptic feedback is accomplished as well for surgery simulation.

CONCLUSIONS

The proposed approach conquers the NFEM limitation of step computation but without trading off the modelling accuracy. It not only has a similar level of accuracy as NFEM, but also meets the real-time requirement for soft tissue modelling.

Domain Heterogeneity in Radiofrequency Therapies for Pain Relief: A Computational Study with Coupled Models

The objective of the current research work is to study the differences between the predicted ablation volume in homogeneous and heterogeneous models of typical radiofrequency (RF) procedures for pain relief. A three-dimensional computational domain comprising of the realistic anatomy of the target tissue was considered in the present study. A comparative analysis was conducted for three different scenarios: (a) a completely homogeneous domain comprising of only muscle tissue, (b) a heterogeneous domain comprising of nerve and muscle tissues, and (c) a heterogeneous domain comprising of bone, nerve and muscle tissues. Finite-element-based simulations were performed to compute the temperature and electrical field distribution during conventional RF procedures for treating pain, and exemplified here for the continuous case. The predicted results reveal that the consideration of heterogeneity within the computational domain results in distorted electric field distribution and leads to a significant reduction in the attained ablation volume during the continuous RF application for pain relief. The findings of this study could provide first-hand quantitative information to clinical practitioners about the impact of such heterogeneities on the efficacy of RF procedures, thereby assisting them in developing standardized optimal protocols for different cases of interest.

Thermal ablation of biological tissues in disease treatment: A review of computational models and future directions

Percutaneous thermal ablation has proven to be an effective modality for treating both benign and malignant tumours in various tissues. Among these modalities, radiofrequency ablation (RFA) is the most promising and widely adopted approach that has been extensively studied in the past decades. Microwave ablation (MWA) is a newly emerging modality that is gaining rapid momentum due to its capability of inducing rapid heating and attaining larger ablation volumes, and its lesser susceptibility to the heat sink effects as compared to RFA. Although the goal of both these therapies is to attain cell death in the target tissue by virtue of heating above 50°C, their underlying mechanism of action and principles greatly differs. Computational modelling is a powerful tool for studying the effect of electromagnetic interactions within the biological tissues and predicting the treatment outcomes during thermal ablative therapies. Such a priori estimation can assist the clinical practitioners during treatment planning with the goal of attaining successful tumour destruction and preservation of the surrounding healthy tissue and critical structures. This review provides current state-of-the-art developments and associated challenges in the computational modelling of thermal ablative techniques, viz., RFA and MWA, as well as touch upon several promising avenues in the modelling of laser ablation, nanoparticles assisted magnetic hyperthermia and non-invasive RFA. The application of RFA in pain relief has been extensively reviewed from modelling point of view. Additionally, future directions have also been provided to improve these models for their successful translation and integration into the hospital work flow.

Fast computation of soft tissue thermal response under deformation based on fast explicit dynamics finite element algorithm for surgical simulation

BACKGROUND AND OBJECTIVES

During thermal heating surgical procedures such as electrosurgery, thermal ablative treatment and hyperthermia, soft tissue deformation due to surgical tool-tissue interaction and patient movement can affect the distribution of thermal energy induced. Soft tissue temperature must be obtained from the deformed tissue for precise delivery of thermal energy. However, the classical Pennes bio-heat transfer model can handle only the static non-moving state of tissue. In addition, in order to enable a surgeon to visualise the simulated results immediately, the solution procedure must be suitable for real-time thermal applications.

METHODS

This paper presents a formulation of bio-heat transfer under the effect of soft tissue deformation for fast or near real-time tissue temperature prediction, based on fast explicit dynamics finite element algorithm (FED-FEM) for transient heat transfer. The proposed thermal analysis under deformation is achieved by transformation of the unknown deformed tissue state to the known initial static state via a mapping function. The appropriateness and effectiveness of the proposed formulation are evaluated on a realistic virtual human liver model with blood vessels to demonstrate a clinically relevant scenario of thermal ablation of hepatic cancer.

RESULTS

For numerical accuracy, the proposed formulation can achieve a typical 10^-3 level of normalised relative error at nodes and between 10^-4 and 10^-5 level of total errors for the simulation, by comparing solutions against the commercial finite element analysis package. For computation time, the proposed formulation under tissue deformation with anisotropic temperature-dependent properties consumes 2.518 × 10^-4 ms for one element thermal loads computation, compared to 2.237 × 10^-4 ms for the formulation without deformation which is 0.89 times of the former. Comparisons with three other formulations for isotropic and temperature-independent properties are also presented.

CONCLUSIONS

Compared to conventional methods focusing on numerical accuracy, convergence and stability, the proposed formulation focuses on computational performance for fast tissue thermal analysis. Compared to the classical Pennes model that handles only the static state of tissue, the proposed formulation can achieve fast thermal analysis on deformed states of tissue and can be applied in addition to tissue deformable models for non-linear heating analysis at even large deformation of soft tissue, leading to great translational potential in dynamic tissue temperature analysis and thermal dosimetry computation for computer-integrated medical education and personalised treatment.

[Artificial intelligence in general and visceral surgery]

Artificial intelligence procedures will find special fields of application also in general and visceral surgery. These will not only be limited to intraoperative surgical applications but also extend to perioperative processes, education and training as well as to future scientific developments. Major impulses are to be expected in decision support systems, cognitive collaborative interventional environments and in evidence-based knowledge acquisition models; however, the implementation into the daily practice not only requires profound insights into the field of informatics and computer science but also a comprehensive knowledge of the surgical domain. Accordingly, the future implementation of artificial intelligence in surgery requires a new culture of collaboration between surgeons and researchers/computer scientists.

Data-Driven GENERIC Modeling of Poroviscoelastic Materials

Biphasic soft materials are challenging to model by nature. Ongoing efforts are targeting their effective modeling and simulation. This work uses experimental atomic force nanoindentation of thick hydrogels to identify the indentation forces are a function of the indentation depth. Later on, the atomic force microscopy results are used in a GENERIC general equation for non-equilibrium reversible–irreversible coupling (GENERIC) formalism to identify the best model conserving basic thermodynamic laws. The data-driven GENERIC analysis identifies the material behavior with high fidelity for both data fitting and prediction.