Parallel Multigrid Preconditioner for the Cardiac Bidomain Model

A smoothed boundary bidomain model for cardiac simulations in anatomically detailed geometries

This manuscript presents a novel finite difference method to solve cardiac bidomain equations in anatomical models of the heart. The proposed method employs a smoothed boundary approach that represents the boundaries between the heart and the surrounding medium as a spatially diffuse interface of finite thickness. The bidomain boundary conditions are implicitly implemented in the smoothed boundary bidomain equations presented in the manuscript without the need of a structured mesh that explicitly tracks the heart-torso boundaries. We reported some significant examples assessing the method's accuracy using nontrivial test geometries and demonstrating the applicability of the method to complex anatomically detailed human cardiac geometries. In particular, we showed that our approach could be employed to simulate cardiac defibrillation in a human left ventricle comprising fiber architecture. The main advantage of the proposed method is the possibility of implementing bidomain boundary conditions directly on voxel structures, which makes it attractive for three dimensional, patient specific simulations based on medical images. Moreover, given the ease of implementation, we believe that the proposed method could provide an interesting and feasible alternative to finite element methods, and could find application in future cardiac research guiding electrotherapy with computational models.

An automated near-real time computational method for induction and treatment of scar-related ventricular tachycardias

Catheter ablation is currently the only curative treatment for scar-related ventricular tachycardias (VTs). However, not only are ablation procedures long, with relatively high risk, but success rates are punitively low, with frequent VT recurrence. Personalized in-silico approaches have the opportunity to address these limitations. However, state-of-the-art reaction diffusion (R-D) simulations of VT induction and subsequent circuits used for in-silico ablation target identification require long execution times, along with vast computational resources, which are incompatible with the clinical workflow. Here, we present the Virtual Induction and Treatment of Arrhythmias (VITA), a novel, rapid and fully automated computational approach that uses reaction-Eikonal methodology to induce VT and identify subsequent ablation targets. The rationale for VITA is based on finding isosurfaces associated with an activation wavefront that splits in the ventricles due to the presence of an isolated isthmus of conduction within the scar; once identified, each isthmus may be assessed for their vulnerability to sustain a reentrant circuit, and the corresponding exit site automatically identified for potential ablation targeting. VITA was tested on a virtual cohort of 7 post-infarcted porcine hearts and the results compared to R-D simulations. Using only a standard desktop machine, VITA could detect all scar-related VTs, simulating activation time maps and ECGs (for clinical comparison) as well as computing ablation targets in 48 minutes. The comparable VTs probed by the R-D simulations took 68.5 hours on 256 cores of high-performance computing infrastructure. The set of lesions computed by VITA was shown to render the ventricular model VT-free. VITA could be used in near real-time as a complementary modality aiding in clinical decision-making in the treatment of post-infarction VTs.

The openCARP simulation environment for cardiac electrophysiology

BACKGROUND AND OBJECTIVE

Cardiac electrophysiology is a medical specialty with a long and rich tradition of computational modeling. Nevertheless, no community standard for cardiac electrophysiology simulation software has evolved yet. Here, we present the openCARP simulation environment as one solution that could foster the needs of large parts of this community.

METHODS AND RESULTS

openCARP and the Python-based carputils framework allow developing and sharing simulation pipelines which automate in silico experiments including all modeling and simulation steps to increase reproducibility and productivity. The continuously expanding openCARP user community is supported by tailored infrastructure. Documentation and training material facilitate access to this complementary research tool for new users. After a brief historic review, this paper summarizes requirements for a high-usability electrophysiology simulator and describes how openCARP fulfills them. We introduce the openCARP modeling workflow in a multi-scale example of atrial fibrillation simulations on single cell, tissue, organ and body level and finally outline future development potential.

CONCLUSION

As an open simulator, openCARP can advance the computational cardiac electrophysiology field by making state-of-the-art simulations accessible. In combination with the carputils framework, it offers a tailored software solution for the scientific community and contributes towards increasing use, transparency, standardization and reproducibility of in silico experiments.

Composite Backward Differentiation Formula for the Bidomain Equations

The bidomain equations have been widely used to model the electrical activity of cardiac tissue. While it is well-known that implicit methods have much better stability than explicit methods, implicit methods usually require the solution of a very large nonlinear system of equations at each timestep which is computationally prohibitive. In this work, we present two fully implicit time integration methods for the bidomain equations: the backward Euler method and a second-order one-step two-stage composite backward differentiation formula (CBDF2) which is an L-stable time integration method. Using the backward Euler method as fundamental building blocks, the CBDF2 scheme is easily implementable. After solving the nonlinear system resulting from application of the above two fully implicit schemes by a nonlinear elimination method, the obtained nonlinear global system has a much smaller size, whose Jacobian is symmetric and possibly positive definite. Thus, the residual equation of the approximate Newton approach for the global system can be efficiently solved by standard optimal solvers. As an alternative, we point out that the above two implicit methods combined with operator splittings can also efficiently solve the bidomain equations. Numerical results show that the CBDF2 scheme is an efficient time integration method while achieving high stability and accuracy.

Performance of preconditioned iterative linear solvers for cardiovascular simulations in rigid and deformable vessels

Computing the solution of linear systems of equations is invariably the most time consuming task in the numerical solutions of PDEs in many fields of computational science. In this study, we focus on the numerical simulation of cardiovascular hemodynamics with rigid and deformable walls, discretized in space and time through the variational multiscale finite element method. We focus on three approaches: the problem agnostic generalized minimum residual (GMRES) and stabilized bi-conjugate gradient (BICGS) methods, and a recently proposed, problem specific, bi-partitioned (BIPN) method. We also perform a comparative analysis of several preconditioners, including diagonal, block-diagonal, incomplete factorization, multigrid, and resistance based methods. Solver performance and matrix characteristics (diagonal dominance, symmetry, sparsity, bandwidth and spectral properties) are first examined for an idealized cylindrical geometry with physiologic boundary conditions and then successively tested on several patient-specific anatomies representative of realistic cardiovascular simulation problems. Incomplete factorization preconditioners provide the best performance and results in terms of both strong and weak scalability. The BIPN method was found to outperform other methods in patient-specific models with rigid walls. In models with deformable walls, BIPN was outperformed by BICG with diagonal and Incomplete LU preconditioners.

Scalable and Accurate ECG Simulation for Reaction-Diffusion Models of the Human Heart

Realistic electrocardiogram (ECG) simulation with numerical models is important for research linking cellular and molecular physiology to clinically observable signals, and crucial for patient tailoring of numerical heart models. However, ECG simulation with a realistic torso model is computationally much harder than simulation of cardiac activity itself, so that many studies with sophisticated heart models have resorted to crude approximations of the ECG. This paper shows how the classical concept of electrocardiographic lead fields can be used for an ECG simulation method that matches the realism of modern heart models. The accuracy and resource requirements were compared to those of a full-torso solution for the potential and scaling was tested up to 14,336 cores with a heart model consisting of 11 million nodes. Reference ECGs were computed on a 3.3 billion-node heart-torso mesh at 0.2 mm resolution. The results show that the lead-field method is more efficient than a full-torso solution when the number of simulated samples is larger than the number of computed ECG leads. While the initial computation of the lead fields remains a hard and poorly scalable problem, the ECG computation itself scales almost perfectly and, even for several hundreds of ECG leads, takes much less time than the underlying simulation of cardiac activity.

A Numerical Study of Scalable Cardiac Electro-Mechanical Solvers on HPC Architectures

We introduce and study some scalable domain decomposition preconditioners for cardiac electro-mechanical 3D simulations on parallel HPC (High Performance Computing) architectures. The electro-mechanical model of the cardiac tissue is composed of four coupled sub-models: (1) the static finite elasticity equations for the transversely isotropic deformation of the cardiac tissue; (2) the active tension model describing the dynamics of the intracellular calcium, cross-bridge binding and myofilament tension; (3) the anisotropic Bidomain model describing the evolution of the intra- and extra-cellular potentials in the deforming cardiac tissue; and (4) the ionic membrane model describing the dynamics of ionic currents, gating variables, ionic concentrations and stretch-activated channels. This strongly coupled electro-mechanical model is discretized in time with a splitting semi-implicit technique and in space with isoparametric finite elements. The resulting scalable parallel solver is based on Multilevel Additive Schwarz preconditioners for the solution of the Bidomain system and on BDDC preconditioned Newton-Krylov solvers for the non-linear finite elasticity system. The results of several 3D parallel simulations show the scalability of both linear and non-linear solvers and their application to the study of both physiological excitation-contraction cardiac dynamics and re-entrant waves in the presence of different mechano-electrical feedbacks.

Preconditioned augmented Lagrangian formulation for nearly incompressible cardiac mechanics

Computational modeling of the heart is a subject of substantial medical and scientific interest, which may contribute to increase the understanding of several phenomena associated with cardiac physiological and pathological states. Modeling the mechanics of the heart have led to considerable insights, but it still represents a complex and a demanding computational problem, especially in a strongly coupled electromechanical setting. Passive cardiac tissue is commonly modeled as hyperelastic and is characterized by quasi-incompressible, orthotropic, and nonlinear material behavior. These factors are known to be very challenging for the numerical solution of the model. The near-incompressibility is known to cause numerical issues such as the well-known locking phenomenon and ill-conditioning of the stiffness matrix. In this work, the augmented Lagrangian method is used to handle the nearly incompressible condition. This approach can potentially improve computational performance by reducing the condition number of the stiffness matrix and thereby improving the convergence of iterative solvers. We also improve the performance of iterative solvers by the use of an algebraic multigrid preconditioner. Numerical results of the augmented Lagrangian method combined with a preconditioned iterative solver for a cardiac mechanics benchmark suite are presented to show its improved performance.

Performance evaluation of GPU parallelization, space-time adaptive algorithms, and their combination for simulating cardiac electrophysiology

The use of computer models as a tool for the study and understanding of the complex phenomena of cardiac electrophysiology has attained increased importance nowadays. At the same time, the increased complexity of the biophysical processes translates into complex computational and mathematical models. To speed up cardiac simulations and to allow more precise and realistic uses, 2 different techniques have been traditionally exploited: parallel computing and sophisticated numerical methods. In this work, we combine a modern parallel computing technique based on multicore and graphics processing units (GPUs) and a sophisticated numerical method based on a new space-time adaptive algorithm. We evaluate each technique alone and in different combinations: multicore and GPU, multicore and GPU and space adaptivity, multicore and GPU and space adaptivity and time adaptivity. All the techniques and combinations were evaluated under different scenarios: 3D simulations on slabs, 3D simulations on a ventricular mouse mesh, ie, complex geometry, sinus-rhythm, and arrhythmic conditions. Our results suggest that multicore and GPU accelerate the simulations by an approximate factor of 33×, whereas the speedups attained by the space-time adaptive algorithms were approximately 48. Nevertheless, by combining all the techniques, we obtained speedups that ranged between 165 and 498. The tested methods were able to reduce the execution time of a simulation by more than 498× for a complex cellular model in a slab geometry and by 165× in a realistic heart geometry simulating spiral waves. The proposed methods will allow faster and more realistic simulations in a feasible time with no significant loss of accuracy.

Numerical sensitivity analysis of a variational data assimilation procedure for cardiac conductivities

An accurate estimation of cardiac conductivities is critical in computational electro-cardiology, yet experimental results in the literature significantly disagree on the values and ratios between longitudinal and tangential coefficients. These are known to have a strong impact on the propagation of potential particularly during defibrillation shocks. Data assimilation is a procedure for merging experimental data and numerical simulations in a rigorous way. In particular, variational data assimilation relies on the least-square minimization of the misfit between simulations and experiments, constrained by the underlying mathematical model, which in this study is represented by the classical Bidomain system, or its common simplification given by the Monodomain problem. Operating on the conductivity tensors as control variables of the minimization, we obtain a parameter estimation procedure. As the theory of this approach currently provides only an existence proof and it is not informative for practical experiments, we present here an extensive numerical simulation campaign to assess practical critical issues such as the size and the location of the measurement sites needed for in silico test cases of potential experimental and realistic settings. This will be finalized with a real validation of the variational data assimilation procedure. Results indicate the presence of lower and upper bounds for the number of sites which guarantee an accurate and minimally redundant parameter estimation, the location of sites being generally non critical for properly designed experiments. An effective combination of parameter estimation based on the Monodomain and Bidomain models is tested for the sake of computational efficiency. Parameter estimation based on the Monodomain equation potentially leads to the accurate computation of the transmembrane potential in real settings.

A comparison of solver performance for complex gastric electrophysiology models

Computational techniques for solving systems of equations arising in gastric electrophysiology have not been studied for efficient solution process. We present a computationally challenging problem of simulating gastric electrophysiology in anatomically realistic stomach geometries with multiple intracellular and extracellular domains. The multiscale nature of the problem and mesh resolution required to capture geometric and functional features necessitates efficient solution methods if the problem is to be tractable. In this study, we investigated and compared several parallel preconditioners for the linear systems arising from tetrahedral discretisation of electrically isotropic and anisotropic problems, with and without stimuli. The results showed that the isotropic problem was computationally less challenging than the anisotropic problem and that the application of extracellular stimuli increased workload considerably. Preconditioning based on block Jacobi and algebraic multigrid solvers were found to have the best overall solution times and least iteration counts, respectively. The algebraic multigrid preconditioner would be expected to perform better on large problems.

An electromechanical left ventricular wedge model to study the effects of deformation on repolarization during heart failure

Heart failure is a major and costly problem in public health, which, in certain cases, may lead to death. The failing heart undergo a series of electrical and structural changes that provide the underlying basis for disturbances like arrhythmias. Computer models of coupled electrical and mechanical activities of the heart can be used to advance our understanding of the complex feedback mechanisms involved. In this context, there is a lack of studies that consider heart failure remodeling using strongly coupled electromechanics. We present a strongly coupled electromechanical model to study the effects of deformation on a human left ventricle wedge considering normal and hypertrophic heart failure conditions. We demonstrate through a series of simulations that when a strongly coupled electromechanical model is used, deformation results in the thickening of the ventricular wall that in turn increases transmural dispersion of repolarization. These effects were analyzed in both normal and failing heart conditions. We also present transmural electrograms obtained from these simulations. Our results suggest that the waveform of electrograms, particularly the T-wave, is influenced by cardiac contraction on both normal and pathological conditions.

Effects of deformation on transmural dispersion of repolarization using in silico models of human left ventricular wedge

Mechanical deformation affects the electrical activity of the heart through multiple feedback loops. The purpose of this work is to study the effect of deformation on transmural dispersion of repolarization and on surface electrograms using an in silico human ventricular wedge. To achieve this purpose, we developed a strongly coupled electromechanical cell model by coupling a human left ventricle electrophysiology model and an active contraction model reparameterized for human cells. This model was then embedded in tissue simulations on the basis of bidomain equations and nonlinear solid mechanics. The coupled model was used to evaluate effects of mechanical deformation on important features of repolarization and electrograms. Our results indicate an increase in the T-wave amplitude of the surface electrograms in simulations that account for the effects of cardiac deformation. This increased T-wave amplitude can be explained by changes to the coupling between neighboring myocytes, also known as electrotonic effect. The thickening of the ventricular wall during repolarization contributes to the decoupling of cells in the transmural direction, enhancing action potential heterogeneity and increasing both transmural repolarization dispersion and T-wave amplitude of surface electrograms. The simulations suggest that a considerable percentage of the T-wave amplitude (15%) may be related to cardiac deformation.

A computer model of lens structure and function predicts experimental changes to steady state properties and circulating currents

BACKGROUND

In a previous study (Vaghefi et al. 2012) we described a 3D computer model that used finite element modeling to capture the structure and function of the ocular lens. This model accurately predicted the steady state properties of the lens including the circulating ionic and fluid fluxes that are believed to underpin the lens internal microcirculation system. In the absence of a blood supply, this system brings nutrients to the core of the lens and removes waste products faster than would be achieved by passive diffusion alone. Here we test the predictive properties of our model by investigating whether it can accurately mimic the experimentally measured changes to lens steady-state properties induced by either depolarising the lens potential or reducing Na+ pump rate.

METHODS

To mimic experimental manipulations reported in the literature, the boundary conditions of the model were progressively altered and the model resolved for each new set of conditions. Depolarisation of lens potential was implemented by increasing the extracellular [K+], while inhibition of the Na+ pump was stimulated by utilising the inherent temperature sensitivity of the pump and changing the temperature at which the model was solved.

RESULTS

Our model correctly predicted that increasing extracellular [K+] depolarizes the lens potential, reducing and then reversing the magnitude of net current densities around the lens. While lowering the temperature reduced Na+ pump activity and caused a reduction in circulating current, it had a minimal effect on the lens potential, a result consistent with published experimental data.

CONCLUSION

We have shown that our model is capable of accurately simulating the effects of two known experimental manipulations on lens steady-state properties. Our results suggest that the model will be a valuable predictive tool to support ongoing studies of lens structure and function.

A fully implicit finite element method for bidomain models of cardiac electromechanics

We propose a novel, monolithic, and unconditionally stable finite element algorithm for the bidomain-based approach to cardiac electromechanics. We introduce the transmembrane potential, the extracellular potential, and the displacement field as independent variables, and extend the common two-field bidomain formulation of electrophysiology to a three-field formulation of electromechanics. The intrinsic coupling arises from both excitation-induced contraction of cardiac cells and the deformation-induced generation of intra-cellular currents. The coupled reaction-diffusion equations of the electrical problem and the momentum balance of the mechanical problem are recast into their weak forms through a conventional isoparametric Galerkin approach. As a novel aspect, we propose a monolithic approach to solve the governing equations of excitation-contraction coupling in a fully coupled, implicit sense. We demonstrate the consistent linearization of the resulting set of non-linear residual equations. To assess the algorithmic performance, we illustrate characteristic features by means of representative three-dimensional initial-boundary value problems. The proposed algorithm may open new avenues to patient specific therapy design by circumventing stability and convergence issues inherent to conventional staggered solution schemes.

Simulations of complex and microscopic models of cardiac electrophysiology powered by multi-GPU platforms

Key aspects of cardiac electrophysiology, such as slow conduction, conduction block, and saltatory effects have been the research topic of many studies since they are strongly related to cardiac arrhythmia, reentry, fibrillation, or defibrillation. However, to reproduce these phenomena the numerical models need to use subcellular discretization for the solution of the PDEs and nonuniform, heterogeneous tissue electric conductivity. Due to the high computational costs of simulations that reproduce the fine microstructure of cardiac tissue, previous studies have considered tissue experiments of small or moderate sizes and used simple cardiac cell models. In this paper, we develop a cardiac electrophysiology model that captures the microstructure of cardiac tissue by using a very fine spatial discretization (8 μm) and uses a very modern and complex cell model based on Markov chains for the characterization of ion channel's structure and dynamics. To cope with the computational challenges, the model was parallelized using a hybrid approach: cluster computing and GPGPUs (general-purpose computing on graphics processing units). Our parallel implementation of this model using a multi-GPU platform was able to reduce the execution times of the simulations from more than 6 days (on a single processor) to 21 minutes (on a small 8-node cluster equipped with 16 GPUs, i.e., 2 GPUs per node).

Modeling defibrillation of the heart: approaches and insights

Cardiac defibrillation, as accomplished nowadays by automatic, implantable devices (ICDs), constitutes the most important means of combating sudden cardiac death. While ICD therapy has proved to be efficient and reliable, defibrillation is a traumatic experience. Thus, research on defibrillation mechanisms, particularly aimed at lowering defibrillation voltage, remains an important topic. Advancing our understanding towards a full appreciation of the mechanisms by which a shock interacts with the heart is the most promising approach to achieve this goal. The aim of this paper is to assess the current state-of-the-art in ventricular defibrillation modeling, focusing on both numerical modeling approaches and major insights that have been obtained using defibrillation models, primarily those of realistic ventricular geometry. The paper showcases the contributions that modeling and simulation have made to our understanding of the defibrillation process. The review thus provides an example of biophysically based computational modeling of the heart (i.e., cardiac defibrillation) that has advanced the understanding of cardiac electrophysiological interaction at the organ level and has the potential to contribute to the betterment of the clinical practice of defibrillation.

Computational modelling of cardiac electrophysiology: explanation of the variability of results from different numerical solvers

A recent verification study compared 11 large-scale cardiac electrophysiology solvers on an unambiguously defined common problem. An unexpected amount of variation was observed between the codes, including significant error in conduction velocity in the majority of the codes at certain spatial resolutions. In particular, the results of the six finite element codes varied considerably despite each using the same order of interpolation. In this present study, we compare various algorithms for cardiac electrophysiological simulation, which allows us to fully explain the differences between the solvers. We identify the use of mass lumping as the fundamental cause of the largest variations, specifically the combination of the commonly used techniques of mass lumping and operator splitting, which results in a slightly different form of mass lumping to that supported by theory and leads to increased numerical error. Other variations are explained through the manner in which the ionic current is interpolated. We also investigate the effect of different forms of mass lumping in various types of simulation.

Accelerating cardiac bidomain simulations using graphics processing units

Anatomically realistic and biophysically detailed multiscale computer models of the heart are playing an increasingly important role in advancing our understanding of integrated cardiac function in health and disease. Such detailed simulations, however, are computationally vastly demanding, which is a limiting factor for a wider adoption of in-silico modeling. While current trends in high-performance computing (HPC) hardware promise to alleviate this problem, exploiting the potential of such architectures remains challenging since strongly scalable algorithms are necessitated to reduce execution times. Alternatively, acceleration technologies such as graphics processing units (GPUs) are being considered. While the potential of GPUs has been demonstrated in various applications, benefits in the context of bidomain simulations where large sparse linear systems have to be solved in parallel with advanced numerical techniques are less clear. In this study, the feasibility of multi-GPU bidomain simulations is demonstrated by running strong scalability benchmarks using a state-of-the-art model of rabbit ventricles. The model is spatially discretized using the finite element methods (FEM) on fully unstructured grids. The GPU code is directly derived from a large pre-existing code, the Cardiac Arrhythmia Research Package (CARP), with very minor perturbation of the code base. Overall, bidomain simulations were sped up by a factor of 11.8 to 16.3 in benchmarks running on 6-20 GPUs compared to the same number of CPU cores. To match the fastest GPU simulation which engaged 20 GPUs, 476 CPU cores were required on a national supercomputing facility.

Bidomain ECG simulations using an augmented monodomain model for the cardiac source

The electrocardiogram (ECG) is an essential clinical tool for the non-invasive assessment of cardiac function. Computational simulations of ECGs using bidomain models are considered the biophysically most detailed approach, but computational costs are significant. Alternatively, pseudo-bidomain formulations can be used, combining a monodomain model with an infrequent bidomain solve to obtain full extracellular potential (φ(e)) distributions and traces. However, previous attempts at such approaches did not see the expected significant decrease in compute time and did not include important effects of bath-loading on activation wavefront morphology (present in full bidomain models), representing a less accurate source term for φ(e) solution. ECG traces can also be derived from computationally cheaper φ(e) recovery techniques, whereby the time-course of φ(e) is approximated at a particular point using the monodomain transmembrane potential as source term. However, φ(e) recovery methods also assume tissue to be immersed in an unbounded conductive medium; not the case in most practical scenarios. We recently demonstrated how bath-loading effects in bidomain simulations could be replicated using an augmented monodomain model, faithfully reproducing bidomain wavefront shapes and activation patterns. Here, a computationally-efficient pseudobidomain formulation is suggested which combines the advantages of an augmented monodomain method with an infrequent bidomain solve, providing activation sequences, ECG traces and φ(e) distributions in a bounded medium surrounding the heart which closely match those of the full bidomain, but at ≈ 10% the computational cost. We demonstrate the important impact of both bath-loading and a finite surrounding bath on spatiotemporal φ(e) distributions, thus demonstrating the utility of our novel pseudo-bidomain model in ECG computation with respect to previous pseudo-bidomain and φ(e) recovery approaches.

Orthogonal recursive bisection as data decomposition strategy for massively parallel cardiac simulations

We present the orthogonal recursive bisection algorithm that hierarchically segments the anatomical model structure into subvolumes that are distributed to cores. The anatomy is derived from the Visible Human Project, with electrophysiology based on the FitzHugh-Nagumo (FHN) and ten Tusscher (TT04) models with monodomain diffusion. Benchmark simulations with up to 16,384 and 32,768 cores on IBM Blue Gene/P and L supercomputers for both FHN and TT04 results show good load balancing with almost perfect speedup factors that are close to linear with the number of cores. Hence, strong scaling is demonstrated. With 32,768 cores, a 1000 ms simulation of full heart beat requires about 6.5 min of wall clock time for a simulation of the FHN model. For the largest machine partitions, the simulations execute at a rate of 0.548 s (BG/P) and 0.394 s (BG/L) of wall clock time per 1 ms of simulation time. To our knowledge, these simulations show strong scaling to substantially higher numbers of cores than reported previously for organ-level simulation of the heart, thus significantly reducing run times. The ability to reduce runtimes could play a critical role in enabling wider use of cardiac models in research and clinical applications.

A fully implicit finite element method for bidomain models of cardiac electrophysiology

This work introduces a novel, unconditionally stable and fully coupled finite element method for the bidomain system of equations of cardiac electrophysiology. The transmembrane potential Φ(i)-Φ(e) and the extracellular potential Φ(e) are treated as independent variables. To this end, the respective reaction-diffusion equations are recast into weak forms via a conventional isoparametric Galerkin approach. The resultant nonlinear set of residual equations is consistently linearised. The method results in a symmetric set of equations, which reduces the computational time significantly compared to the conventional solution algorithms. The proposed method is inherently modular and can be combined with phenomenological or ionic models across the cell membrane. The efficiency of the method and the comparison of its computational cost with respect to the simplified monodomain models are demonstrated through representative numerical examples.

Stimulus protocol determines the most computationally efficient preconditioner for the bidomain equations

The efficient solution of the bidomain equations is a fundamental tool in the field of cardiac electrophysiology. When choosing a finite element discretization of the coupled system, one has to deal with the solution of a large, highly sparse system of linear equations. The conjugate gradient algorithm, along with suitable preconditioning, is the natural choice in this scenario. In this study, we identify the optimal preconditioners with respect to both stimulus protocol and mesh geometry. The results are supported by a comprehensive study of the mesh-dependence properties of several preconditioning techniques found in the literature. Our results show that when only intracellular stimulus is considered, incomplete LU factorization remains a valid choice for current cardiac geometries. However, when extracellular shocks are delivered to tissue, preconditioners that take into account the structure of the system minimize execution time and ensure mesh-independent convergence.

Models of cardiac tissue electrophysiology: progress, challenges and open questions

Models of cardiac tissue electrophysiology are an important component of the Cardiac Physiome Project, which is an international effort to build biophysically based multi-scale mathematical models of the heart. Models of tissue electrophysiology can provide a bridge between electrophysiological cell models at smaller scales, and tissue mechanics, metabolism and blood flow at larger scales. This paper is a critical review of cardiac tissue electrophysiology models, focussing on the micro-structure of cardiac tissue, generic behaviours of action potential propagation, different models of cardiac tissue electrophysiology, the choice of parameter values and tissue geometry, emergent properties in tissue models, numerical techniques and computational issues. We propose a tentative list of information that could be included in published descriptions of tissue electrophysiology models, and used to support interpretation and evaluation of simulation results. We conclude with a discussion of challenges and open questions.

Near-real-time simulations of biolelectric activity in small mammalian hearts using graphical processing units

Simulations of cardiac bioelectric phenomena remain a significant challenge despite continual advancements in computational machinery. Spanning large temporal and spatial ranges demands millions of nodes to accurately depict geometry, and a comparable number of timesteps to capture dynamics. This study explores a new hardware computing paradigm, the graphics processing unit (GPU), to accelerate cardiac models, and analyzes results in the context of simulating a small mammalian heart in real time. The ODEs associated with membrane ionic flow were computed on traditional CPU and compared to GPU performance, for one to four parallel processing units. The scalability of solving the PDE responsible for tissue coupling was examined on a cluster using up to 128 cores. Results indicate that the GPU implementation was between 9 and 17 times faster than the CPU implementation and scaled similarly. Solving the PDE was still 160 times slower than real time.

Simulation of cardiac electrophysiology on next-generation high-performance computers

Models of cardiac electrophysiology consist of a system of partial differential equations (PDEs) coupled with a system of ordinary differential equations representing cell membrane dynamics. Current software to solve such models does not provide the required computational speed for practical applications. One reason for this is that little use is made of recent developments in adaptive numerical algorithms for solving systems of PDEs. Studies have suggested that a speedup of up to two orders of magnitude is possible by using adaptive methods. The challenge lies in the efficient implementation of adaptive algorithms on massively parallel computers. The finite-element (FE) method is often used in heart simulators as it can encapsulate the complex geometry and small-scale details of the human heart. An alternative is the spectral element (SE) method, a high-order technique that provides the flexibility and accuracy of FE, but with a reduced number of degrees of freedom. The feasibility of implementing a parallel SE algorithm based on fully unstructured all-hexahedra meshes is discussed. A major computational task is solution of the large algebraic system resulting from FE or SE discretization. Choice of linear solver and preconditioner has a substantial effect on efficiency. A fully parallel implementation based on dynamic partitioning that accounts for load balance, communication and data movement costs is required. Each of these methods must be implemented on next-generation supercomputers in order to realize the necessary speedup. The problems that this may cause, and some of the techniques that are beginning to be developed to overcome these issues, are described.

CHASTE: incorporating a novel multi-scale spatial and temporal algorithm into a large-scale open source library

Recent work has described the software engineering and computational infrastructure that has been set up as part of the Cancer, Heart and Soft Tissue Environment (CHASTE) project. CHASTE is an open source software package that currently has heart and cancer modelling functionality. This software has been written using a programming paradigm imported from the commercial sector and has resulted in a code that has been subject to a far more rigorous testing procedure than that is usual in this field. In this paper, we explain how new functionality may be incorporated into CHASTE. Whiteley has developed a numerical algorithm for solving the bidomain equations that uses the multi-scale (MS) nature of the physiology modelled to enhance computational efficiency. Using a simple geometry in two dimensions and a purpose-built code, this algorithm was reported to give an increase in computational efficiency of more than two orders of magnitude. In this paper, we begin by reviewing numerical methods currently in use for solving the bidomain equations, explaining how these methods may be developed to use the MS algorithm discussed above. We then demonstrate the use of this algorithm within the CHASTE framework for solving the monodomain and bidomain equations in a three-dimensional realistic heart geometry. Finally, we discuss how CHASTE may be developed to include new physiological functionality--such as modelling a beating heart and fluid flow in the heart--and how new algorithms aimed at increasing the efficiency of the code may be incorporated.

Numerical solution of the bidomain equations

Knowledge of cardiac electrophysiology is efficiently formulated in terms of mathematical models. However, most of these models are very complex and thus defeat direct mathematical reasoning founded on classical and analytical considerations. This is particularly so for the celebrated bidomain model that was developed almost 40 years ago for the concurrent analysis of extra- and intracellular electrical activity. Numerical simulations based on this model represent an indispensable tool for studying electrophysiology. However, complex mathematical models, steep gradients in the solutions and complicated geometries lead to extremely challenging computational problems. The greatest achievement in scientific computing over the past 50 years has been to enable the solving of linear systems of algebraic equations that arise from discretizations of partial differential equations in an optimal manner, i.e. such that the central processing unit (CPU) effort increases linearly with the number of computational nodes. Over the past decade, such optimal methods have been introduced in the simulation of electrophysiology. This development, together with the development of affordable parallel computers, has enabled the solution of the bidomain model combined with accurate cellular models, on geometries resembling a human heart. However, in spite of recent progress, the full potential of modern computational methods has yet to be exploited for the solution of the bidomain model. This paper reviews the development of numerical methods for solving the bidomain model. However, the field is huge and we thus restrict our focus to developments that have been made since the year 2000.

Efficient fully implicit time integration methods for modeling cardiac dynamics

Implicit methods are well known to have greater stability than explicit methods for stiff systems, but they often are not used in practice due to perceived computational complexity. This paper applies the backward Euler (BE) method and a second-order one-step two-stage composite backward differentiation formula (C-BDF2) for the monodomain equations arising from mathematically modeling the electrical activity of the heart. The C-BDF2 scheme is an L-stable implicit time integration method and easily implementable. It uses the simplest forward Euler and BE methods as fundamental building blocks. The nonlinear system resulting from application of the BE method for the monodomain equations is solved for the first time by a nonlinear elimination method, which eliminates local and nonsymmetric components by using a Jacobian-free Newton solver, called Newton--Krylov solver. Unlike other fully implicit methods proposed for the monodomain equations in the literature, the Jacobian of the global system after the nonlinear elimination has much smaller size, is symmetric and possibly positive definite, which can be solved efficiently by standard optimal solvers. Numerical results are presented demonstrating that the C-BDF2 scheme can yield accurate results with less CPU times than explicit methods for both a single patch and spatially extended domains.

From mitochondrial ion channels to arrhythmias in the heart: computational techniques to bridge the spatio-temporal scales

Computer simulations of electrical behaviour in the whole ventricles have become commonplace during the last few years. The goals of this article are (i) to review the techniques that are currently employed to model cardiac electrical activity in the heart, discussing the strengths and weaknesses of the various approaches, and (ii) to implement a novel modelling approach, based on physiological reasoning, that lifts some of the restrictions imposed by current state-of-the-art ionic models. To illustrate the latter approach, the present study uses a recently developed ionic model of the ventricular myocyte that incorporates an excitation-contraction coupling and mitochondrial energetics model. A paradigm to bridge the vastly disparate spatial and temporal scales, from subcellular processes to the entire organ, and from sub-microseconds to minutes, is presented. Achieving sufficient computational efficiency is the key to success in the quest to develop multiscale realistic models that are expected to lead to better understanding of the mechanisms of arrhythmia induction following failure at the organelle level, and ultimately to the development of novel therapeutic applications.

Chaste: using agile programming techniques to develop computational biology software

Cardiac modelling is the area of physiome modelling where the available simulation software is perhaps most mature, and it therefore provides an excellent starting point for considering the software requirements for the wider physiome community. In this paper, we will begin by introducing some of the most advanced existing software packages for simulating cardiac electrical activity. We consider the software development methods used in producing codes of this type, and discuss their use of numerical algorithms, relative computational efficiency, usability, robustness and extensibility. We then go on to describe a class of software development methodologies known as test-driven agile methods and argue that such methods are more suitable for scientific software development than the traditional academic approaches. A case study is a project of our own, Cancer, Heart and Soft Tissue Environment, which is a library of computational biology software that began as an experiment in the use of agile programming methods. We present our experiences with a review of our progress thus far, focusing on the advantages and disadvantages of this new approach compared with the development methods used in some existing packages. We conclude by considering whether the likely wider needs of the cardiac modelling community are currently being met and suggest that, in order to respond effectively to changing requirements, it is essential that these codes should be more malleable. Such codes will allow for reliable extensions to include both detailed mathematical models--of the heart and other organs--and more efficient numerical techniques that are currently being developed by many research groups worldwide.

An efficient computational method for simulation of the two-dimensional electrophysiological waves

This work presents an efficient simulation for the two-dimensional bidomain model, a non-linear system of partial differential equations which is widely used for simulation of the electrical activity of the heart. The accuracy of the solution is obtained by using an anisotropic time-dependent adaptive method. The method reduces greatly the number of element and therefore the computational time. Two-dimensional numerical results are presented to illustrate the performance of the proposed method.

Solvers for the cardiac bidomain equations

The bidomain equations are widely used for the simulation of electrical activity in cardiac tissue. They are especially important for accurately modeling extracellular stimulation, as evidenced by their prediction of virtual electrode polarization before experimental verification. However, solution of the equations is computationally expensive due to the fine spatial and temporal discretization needed. This limits the size and duration of the problem which can be modeled. Regardless of the specific form into which they are cast, the computational bottleneck becomes the repeated solution of a large, linear system. The purpose of this review is to give an overview of the equations and the methods by which they have been solved. Of particular note are recent developments in multigrid methods, which have proven to be the most efficient.

A guide to modelling cardiac electrical activity in anatomically detailed ventricles

One of the most recent trends in cardiac electrophysiology is the development of integrative anatomically accurate models of the heart, which include description of cardiac activity from sub-cellular and cellular level to the level of the whole organ. In order to construct this type of model, a researcher needs to collect a wide range of information from books and journal articles on various aspects of biology, physiology, electrophysiology, numerical mathematics and computer programming. The aim of this methodological article is to survey recent developments in integrative modelling of electrical activity in the ventricles of the heart, and to provide a practical guide to the resources and tools that are available for work in this exciting and challenging area.

Reduced-order preconditioning for bidomain simulations

Simulations of the bidomain equations involve solving large, sparse, linear systems of the form Ax = b. Being an initial value problems, it is solved at every time step. Therefore, efficient solvers are essential to keep simulations tractable. Iterative solvers, especially the preconditioned conjugate gradient (PCG) method, are attractive since memory demands are minimized compared to direct methods, albeit at the cost of solution speed. However, a proper preconditioner can drastically speed up the solution process by reducing the number of iterations. In this paper, a novel preconditioner for the PCG method based on system order reduction using the Arnoldi method (A-PCG) is proposed. Large order systems, generated during cardiac bidomain simulations employing a finite element method formulation, are solved with the A-PCG method. Its performance is compared with incomplete LU (ILU) preconditioning. Results indicate that the A-PCG estimates an approximate solution considerably faster than the ILU, often within a single iteration. To reduce the computational demands in terms of memory and run time, the use of a cascaded preconditioner was suggested. The A-PCG was applied to quickly obtain an approximate solution, and subsequently a cheap iterative method such as successive overrelaxation (SOR) is applied to further refine the solution to arrive at a desired accuracy. The memory requirements are less than those of direct LU but more than ILU method. The proposed scheme is shown to yield significant speedups when solving time evolving systems.

Physiology driven adaptivity for the numerical solution of the bidomain equations

Previous work [Whiteley, J. P. IEEE Trans. Biomed. Eng. 53:2139-2147, 2006] derived a stable, semi-implicit numerical scheme for solving the bidomain equations. This scheme allows the timestep used when solving the bidomain equations numerically to be chosen by accuracy considerations rather than stability considerations. In this study we modify this scheme to allow an adaptive numerical solution in both time and space. The spatial mesh size is determined by the gradient of the transmembrane and extracellular potentials while the timestep is determined by the values of: (i) the fast sodium current; and (ii) the calcium release from junctional sarcoplasmic reticulum to myoplasm current. For two-dimensional simulations presented here, combining the numerical algorithm in the paper cited above with the adaptive algorithm presented here leads to an increase in computational efficiency by a factor of around 250 over previous work, together with significantly less computational memory being required. The speedup for three-dimensional simulations is likely to be more impressive.

Forward Euler Stability of the Bidomain Model of Cardiac Tissue

The bidomain model describes the anisotropic electrical properties of cardiac tissue. One common numerical technique for solving the bidomain equations is the explicit forward Euler method. In this communication we derive a relationship between the time and space steps that ensures the stability of this numerical method.

Algebraic multigrid preconditioner for the cardiac bidomain model

The bidomain equations are considered to be one of the most complete descriptions of the electrical activity in cardiac tissue, but large scale simulations, as resulting from discretization of an entire heart, remain a computational challenge due to the elliptic portion of the problem, the part associated with solving the extracellular potential. In such cases, the use of iterative solvers and parallel computing environments are mandatory to make parameter studies feasible. The preconditioned conjugate gradient (PCG) method is a standard choice for this problem. Although robust, its efficiency greatly depends on the choice of preconditioner. On structured grids, it has been demonstrated that a geometric multigrid preconditioner performs significantly better than an incomplete LU (ILU) preconditioner. However, unstructured grids are often preferred to better represent organ boundaries and allow for coarser discretization in the bath far from cardiac surfaces. Under these circumstances, algebraic multigrid (AMG) methods are advantageous since they compute coarser levels directly from the system matrix itself, thus avoiding the complexity of explicitly generating coarser, geometric grids. In this paper, the performance of an AMG preconditioner (BoomerAMG) is compared with that of the standard ILU preconditioner and a direct solver. BoomerAMG is used in two different ways, as a preconditioner and as a standalone solver. Two 3-D simulation examples modeling the induction of arrhythmias in rabbit ventricles were used to measure performance in both sequential and parallel simulations. It is shown that the AMG preconditioner is very well suited for the solution of the bidomain equation, being clearly superior to ILU preconditioning in all regards, with speedups by factors in the range 5.9-7.7.

Alternans and the influence of ionic channel modifications: Cardiac three-dimensional simulations and one-dimensional numerical bifurcation analysis

Cardiac propagation is investigated by simulations using a realistic three-dimensional (3D) geometry including muscle fiber orientation of the ventricles of a rabbit heart and the modified Beeler-Reuter ionic model. Electrical excitation is introduced by a periodic pacing of the lower septum. Depending on the pacing frequency, qualitatively different dynamics are observed, namely, normal heart beat, T-wave alternans, and 2:1 conduction block at small, intermediate, and large pacing frequencies, respectively. In a second step, we performed a numerical stability and bifurcation analysis of a pulse propagating in a one-dimensional (1D) ring of cardiac tissue. The precise onset of the alternans instability is obtained from computer-assisted linear stability analysis of the pulse and computation of the associated spectrum. The critical frequency at the onset of alternans and the profiles of the membrane potential agree well with the ones obtained in the 3D simulations. Next, we computed changes in the wave profiles and in the onset of alternans for the Beeler-Reuter model with modifications of the sodium, calcium, and potassium channels, respectively. For this purpose, we employ the method of numerical bifurcation and stability analysis. While blocking of calcium channels has a stabilizing effect, blocked sodium or potassium channels lead to the occurrence of alternans at lower pacing frequencies. The findings regarding channel blocking are verified within three-dimensional simulations. Altogether, we have found T-wave alternans and conduction block in 3D simulations of a realistic rabbit heart geometry. The onset of alternans has been analyzed by numerical bifurcation and stability analysis of 1D wave trains. By comparing the results of the two approaches, we find that alternans is not strongly influenced by ingredients such as 3D geometry and propagation anisotropy, but depends mostly on the frequency of pacing (frequency of subsequent action potentials). In addition, we have introduced numerical bifurcation and stability analysis as a tool into heart modeling and demonstrated its efficiency in scanning a large set of parameters in the case of models with reduced conductivity. Bifurcation analysis also provides an accurate test for analytical theories of alternans as is demonstrated for the case of the restitution hypothesis.

A comparison of multilevel solvers for the cardiac bidomain equations

Computing the extracellular potentials in a bidomain cardiac activation model is a computationally significant step in the solution process. Thus, using a fast solver can drastically reduce the overall time of simulation. Solving for the extracellular potentials involves inverting the matrix coming from the elliptic equation describing the extracellular-intracellular potential coupling. Elliptic equations are known to yield matrices that become progressively more ill-conditioned as the spatial resolution is increased. However, optimal multilevel solution methods are known to exist for these equations given enough effort is placed into developing the correct solution components. Two multilevel solvers that automatically perform much of this work are black box multigrid (BOXMG) and algebraic multigrid (AMG). In this paper, we compare the performance of BOXMG and AMG as solvers for the elliptic component of the bidomain equations. Our investigation is with respect to simulations of reentry in two-dimensional cardiac tissue.

Arnoldi preconditioning for solving large linear biomedical systems

Simulations of biomedical systems often involve solving large, sparse, linear systems of the form Ax = b. In initial value problems, this system is solved at every time step, so a quick solution is essential for tractability. Iterative solvers, especially preconditioned conjugate gradient, are attractive since memory demands are minimized compared to direct methods, albeit at a cost of solution speed. A proper preconditioner can drastically reduce computation and remains an area of active research. In this paper, we propose a novel preconditioner based on system order reduction using the Arnoldi method. Systems of orders up to a million, generated from a finite element method formulation of the elliptic portion of the bidomain equations, are solved with the new preconditioner and performance is compared with that of other preconditioners. Results indicate that the new method converges considerably faster, often within a single iteration. It also uses less memory than an incomplete LU decomposition (ILU). For solving a system repeatedly, the Arnoldi transformation must be continually recomputed, unlike ILU, but this can be done quickly. In conclusion, for solving a system once, the Arnoldi preconditioner offers a greatly reduced solution time, and for repeated solves, will still be faster than an ILU preconditioner.

A comparison of monodomain and bidomain reaction-diffusion models for action potential propagation in the human heart

A bidomain reaction-diffusion model of the human heart was developed, and potentials resulting from normal depolarization and repolarization were compared with results from a compatible monodomain model. Comparisons were made for an empty isolated heart and for a heart with fluid-filled ventricles. Both sinus rhythm and ectopic activation were simulated. The bidomain model took 2 days on 32 processors to simulate a complete cardiac cycle. Differences between monodomain and bidomain results were extremely small, even for the extracellular potentials, which in case of the monodomain model were computed with a high-resolution forward model. Propagation of activation was 2% faster in the bidomain model than in the monodomain model. Electrograms computed with monodomain and bidomain models were visually indistinguishable. We conclude that, in the absence of applied currents, propagating action potentials on the scale of a human heart can be studied with a monodomain model.

An efficient numerical technique for the solution of the monodomain and bidomain equations

Most numerical schemes for solving the monodomain or bidomain equations use a forward approximation to some or all of the time derivatives. This approach, however, constrains the maximum timestep that may be used by stability considerations as well as accuracy considerations. Stability may be ensured by using a backward approximation to all time derivatives, although this approach requires the solution of a very large system of nonlinear equations at each timestep which is computationally prohibitive. In this paper we propose a semi-implicit algorithm that ensures stability. A linear system is solved on each timestep to update the transmembrane potential and, if the bidomain equations are being used, the extracellular potential. The remainder of the equations to be solved uncouple into small systems of ordinary differential equations. The backward Euler method may be used to solve these systems and guarantee numerical stability: as these systems are small, only the solution of small nonlinear systems are required. Simulations are carried out to show that the use of this algorithm allows much larger timesteps to be used with only a minimal loss of accuracy. As a result of using these longer timesteps the computation time may be reduced substantially.

Solving the cardiac bidomain equations for discontinuous conductivities

Fast simulations of cardiac electrical phenomena demand fast matrix solvers for both the elliptic and parabolic parts of the bidomain equations. It is well known that fast matrix solvers for the elliptic part must address multiple physical scales in order to show robust behavior. Recent research on finding the proper solution method for the bidomain equations has addressed this issue whereby multigrid preconditioned conjugate gradients has been used as a solver. In this paper, a more robust multigrid method, called Black Box Multigrid, is presented as an alternative to conventional geometric multigrid, and the effect of discontinuities on solver performance for the elliptic and parabolic part is investigated. Test problems with discontinuities arising from inserted plunge electrodes and naturally occurring myocardial discontinuities are considered. For these problems, we explore the advantages to using a more advanced multigrid method like Black Box Multigrid over conventional geometric multigrid. Results will indicate that for certain discontinuous bidomain problems Black Box Multigrid provides 60% faster simulations than using conventional geometric multigrid. Also, for the problems examined, it will be shown that a direct usage of conventional multigrid leads to faster simulations than an indirect usage of conventional multigrid as a preconditioner unless there are sharp discontinuities.

On the Computational Complexity of the Bidomain and the Monodomain Models of Electrophysiology

The bidomain model, coupled with accurate models of cell membrane kinetics, is generally believed to provide a reasonable basis for numerical simulations of cardiac electrophysiology. Because of changes occurring in very short time intervals and over small spatial domains, discretized versions of these models must be solved on fine computational grids, and small time-steps must be applied. This leads to huge computational challenges that have been addressed by several authors. One popular way of reducing the CPU demands is to approximate the bidomain model by the monodomain model, and thus reducing a two by two set of partial differential equations to one scalar partial differential equation; both of which are coupled to a set of ordinary differential equations modeling the cell membrane kinetics. A reduction in CPU time of two orders of magnitude has been reported. It is the purpose of the present paper to provide arguments that such a reduction is not present when order-optimal numerical methods are applied. Theoretical considerations and numerical experiments indicate that the reduction factor of the CPU requirements from bidomain to monodomain computations, using order-optimal methods, typically is about 10 for simple cell models and less than two for more complex cell models.

Accelerating large cardiac bidomain simulations by arnoldi preconditioning

Bidomain simulations of cardiac systems often in volve solving large, sparse, linear systems of the form Ax=b. These simulations are computationally very expensive in terms of run time and memory requirements. Therefore, efficient solvers are essential to keep simulations tractable. In this paper, an efficient preconditioner for the conjugate gradient (CG) method based on system order reduction using the Arnoldi method (A-PCG) is explained. Large order systems generated during cardiac bidomain simulations using a finite element method formulation, are solved using the A-PCG method. Its performance is compared with incomplete LU (ILU) preconditioning. Results indicate that the A-PCG estimates an approximate solution considerably faster than the ILU, often within a single iteration. To reduce the computational demands in terms of memory and run time, the use of a cascaded preconditioner is suggested. The A-PCG can be applied to quickly obtain an approximate solution, subsequently a cheap iterative method such as successive overrelaxation (SOR) is applied to further refine the solution to arrive at a desired accuracy. The memory requirements are less than direct LU but more than ILU method. The proposed scheme is shown to yield significant speedups when solving time evolving systems.